1
0
Fork 0
mirror of https://github.com/jashkenas/coffeescript.git synced 2022-11-09 12:23:24 -05:00

rebuilding the docs with new pygments, no errors on '@'

This commit is contained in:
Jeremy Ashkenas 2010-04-03 12:01:46 -04:00
parent 44765907b3
commit 9d2c81ea54
7 changed files with 566 additions and 565 deletions

View file

@ -30,7 +30,7 @@ then compile it by calling <code>.compile()</code> on the root, or traverse it b
<span class="nx">parser</span><span class="p">.</span><span class="nx">parse</span> <span class="nx">lexer</span><span class="p">.</span><span class="nx">tokenize</span> <span class="nx">code</span></pre></div> </td> </tr> <tr id="section-8"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-8">#</a> </div> <p>Compile and execute a string of CoffeeScript (on the server), correctly
setting <code>__filename</code>, <code>__dirname</code>, and relative <code>require()</code>.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">exports.run: </span><span class="p">((</span><span class="nx">code</span><span class="p">,</span> <span class="nx">options</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="nv">module.filename: __filename: </span><span class="nx">options</span><span class="p">.</span><span class="nx">source</span>
<span class="nv">__dirname: </span><span class="nx">path</span><span class="p">.</span><span class="nx">dirname</span> <span class="nx">__filename</span>
<span class="nv">__dirname: </span><span class="nx">path</span><span class="p">.</span><span class="nx">dirname</span><span class="p">(</span><span class="nx">__filename</span><span class="p">)</span>
<span class="nb">eval</span> <span class="nx">exports</span><span class="p">.</span><span class="nx">compile</span> <span class="nx">code</span><span class="p">,</span> <span class="nx">options</span>
<span class="p">)</span></pre></div> </td> </tr> <tr id="section-9"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-9">#</a> </div> <p>Extend CoffeeScript with a custom language extension. It should hook in to
the <strong>Lexer</strong> (as a peer of any of the lexer's tokenizing methods), and
@ -40,16 +40,16 @@ peer of the nodes in <a href="nodes.html">nodes.coffee</a>).</p> </t
thin wrapper around it, compatible with the Jison API. We can then pass it
directly as a "Jison lexer".</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">parser.lexer: </span><span class="p">{</span>
<span class="nv">lex: </span><span class="o">-&gt;</span>
<span class="nv">token: </span><span class="err">@</span><span class="nx">tokens</span><span class="p">[</span><span class="err">@</span><span class="nx">pos</span><span class="p">]</span> <span class="o">or</span> <span class="p">[</span><span class="s2">&quot;&quot;</span><span class="p">]</span>
<span class="err">@</span><span class="nv">pos: </span><span class="o">+</span> <span class="mi">1</span>
<span class="nv">token: </span><span class="nx">@tokens</span><span class="p">[</span><span class="nx">@pos</span><span class="p">]</span> <span class="o">or</span> <span class="p">[</span><span class="s2">&quot;&quot;</span><span class="p">]</span>
<span class="vi">@pos: </span><span class="o">+</span> <span class="mi">1</span>
<span class="k">this</span><span class="p">.</span><span class="nv">yylineno: </span><span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
<span class="k">this</span><span class="p">.</span><span class="nv">yytext: </span> <span class="nx">token</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="nv">setInput: </span><span class="p">(</span><span class="nx">tokens</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="err">@</span><span class="nv">tokens: </span><span class="nx">tokens</span>
<span class="err">@</span><span class="nv">pos: </span><span class="mi">0</span>
<span class="vi">@tokens: </span><span class="nx">tokens</span>
<span class="vi">@pos: </span><span class="mi">0</span>
<span class="nv">upcomingInput: </span><span class="o">-&gt;</span> <span class="s2">&quot;&quot;</span>
<span class="nv">showPosition: </span><span class="o">-&gt;</span> <span class="err">@</span><span class="nx">pos</span>
<span class="nv">showPosition: </span><span class="o">-&gt;</span> <span class="nx">@pos</span>
<span class="p">}</span></pre></div> </td> </tr> <tr id="section-11"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-11">#</a> </div> <p>Activate CoffeeScript in the browser by having it compile and evaluate
all script tags with a content-type of <code>text/coffeescript</code>. This happens
on page load. Unfortunately, the text contents of remote scripts cannot be

View file

@ -14,7 +14,7 @@ reduces into the <a href="http://en.wikipedia.org/wiki/Terminal_and_nonterminal_
<p>If you run the <code>cake build:parser</code> command, Jison constructs a parse table
from our rules and saves it into <code>lib/parser.js</code>.</p> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-2"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-2">#</a> </div> <p>The only dependency is on the <strong>Jison.Parser</strong>.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">Parser: </span><span class="nx">require</span><span class="p">(</span><span class="s1">&#39;jison&#39;</span><span class="p">).</span><span class="nx">Parser</span></pre></div> </td> </tr> <tr id="section-3"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-3">#</a> </div> <h2>Jison DSL</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-4"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-4">#</a> </div> <p>Since we're going to be wrapped in a function by Jison in any case, if our
action immediately returns a value, we can optimize by removing the function
wrapper and just returning the value directly.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">unwrap: </span><span class="sr">/function\s*\(\)\s*\{\s*return\s*([\s\S]*);\s*\}/</span></pre></div> </td> </tr> <tr id="section-5"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-5">#</a> </div> <p>Our handy DSL for Jison grammar generation, thanks to
wrapper and just returning the value directly.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">unwrap: </span><span class="o">/</span><span class="nx">function</span><span class="err">\</span><span class="nx">s</span><span class="o">*</span><span class="err">\</span><span class="p">(</span><span class="err">\</span><span class="p">)</span><span class="err">\</span><span class="nx">s</span><span class="o">*</span><span class="err">\</span><span class="p">{</span><span class="err">\</span><span class="nx">s</span><span class="o">*</span><span class="k">return</span><span class="err">\</span><span class="nx">s</span><span class="o">*</span><span class="p">([</span><span class="err">\</span><span class="nx">s</span><span class="err">\</span><span class="nx">S</span><span class="p">]</span><span class="o">*</span><span class="p">);</span><span class="err">\</span><span class="nx">s</span><span class="o">*</span><span class="err">\</span><span class="p">}</span><span class="o">/</span></pre></div> </td> </tr> <tr id="section-5"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-5">#</a> </div> <p>Our handy DSL for Jison grammar generation, thanks to
<a href="http://github.com/creationix">Tim Caswell</a>. For every rule in the grammar,
we pass the pattern-defining string, the action to run, and extra options,
optionally. If no action is specified, we simply pass the value of the

View file

@ -32,31 +32,31 @@ of source.</p>
unless explicitly asked not to.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tokenize: </span><span class="p">(</span><span class="nx">code</span><span class="p">,</span> <span class="nx">options</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="nx">code</span> <span class="o">:</span> <span class="nx">code</span><span class="p">.</span><span class="nx">replace</span> <span class="o">/</span><span class="p">(</span><span class="err">\</span><span class="nx">r</span><span class="o">|</span><span class="err">\</span><span class="nx">s</span><span class="o">+</span><span class="nx">$</span><span class="p">)</span><span class="o">/</span><span class="nx">g</span><span class="p">,</span> <span class="s1">&#39;&#39;</span>
<span class="nx">o</span> <span class="o">:</span> <span class="nx">options</span> <span class="o">or</span> <span class="p">{}</span>
<span class="err">@</span><span class="nx">code</span> <span class="o">:</span> <span class="nx">code</span> <span class="c1"># The remainder of the source code.</span>
<span class="err">@</span><span class="nx">i</span> <span class="o">:</span> <span class="mi">0</span> <span class="c1"># Current character position we&#39;re parsing.</span>
<span class="err">@</span><span class="nx">line</span> <span class="o">:</span> <span class="nx">o</span><span class="p">.</span><span class="nx">line</span> <span class="o">or</span> <span class="mi">0</span> <span class="c1"># The current line.</span>
<span class="err">@</span><span class="nx">indent</span> <span class="o">:</span> <span class="mi">0</span> <span class="c1"># The current indentation level.</span>
<span class="err">@</span><span class="nx">indents</span> <span class="o">:</span> <span class="p">[]</span> <span class="c1"># The stack of all current indentation levels.</span>
<span class="err">@</span><span class="nx">tokens</span> <span class="o">:</span> <span class="p">[]</span> <span class="c1"># Stream of parsed tokens in the form [&#39;TYPE&#39;, value, line]</span>
<span class="k">while</span> <span class="err">@</span><span class="nx">i</span> <span class="o">&lt;</span> <span class="err">@</span><span class="nx">code</span><span class="p">.</span><span class="nx">length</span>
<span class="err">@</span><span class="nv">chunk: </span><span class="err">@</span><span class="nx">code</span><span class="p">.</span><span class="nx">slice</span> <span class="err">@</span><span class="nx">i</span>
<span class="err">@</span><span class="nx">extract_next_token</span><span class="p">()</span>
<span class="err">@</span><span class="nx">close_indentation</span><span class="p">()</span>
<span class="k">return</span> <span class="err">@</span><span class="nx">tokens</span> <span class="k">if</span> <span class="nx">o</span><span class="p">.</span><span class="nx">rewrite</span> <span class="o">is</span> <span class="kc">off</span>
<span class="p">(</span><span class="k">new</span> <span class="nx">Rewriter</span><span class="p">()).</span><span class="nx">rewrite</span> <span class="err">@</span><span class="nx">tokens</span></pre></div> </td> </tr> <tr id="section-7"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-7">#</a> </div> <p>At every position, run through this list of attempted matches,
<span class="nx">@code</span> <span class="o">:</span> <span class="nx">code</span> <span class="c1"># The remainder of the source code.</span>
<span class="nx">@i</span> <span class="o">:</span> <span class="mi">0</span> <span class="c1"># Current character position we&#39;re parsing.</span>
<span class="nx">@line</span> <span class="o">:</span> <span class="nx">o</span><span class="p">.</span><span class="nx">line</span> <span class="o">or</span> <span class="mi">0</span> <span class="c1"># The current line.</span>
<span class="nx">@indent</span> <span class="o">:</span> <span class="mi">0</span> <span class="c1"># The current indentation level.</span>
<span class="nx">@indents</span> <span class="o">:</span> <span class="p">[]</span> <span class="c1"># The stack of all current indentation levels.</span>
<span class="nx">@tokens</span> <span class="o">:</span> <span class="p">[]</span> <span class="c1"># Stream of parsed tokens in the form [&#39;TYPE&#39;, value, line]</span>
<span class="k">while</span> <span class="nx">@i</span> <span class="o">&lt;</span> <span class="nx">@code</span><span class="p">.</span><span class="nx">length</span>
<span class="vi">@chunk: </span><span class="nx">@code</span><span class="p">.</span><span class="nx">slice</span> <span class="nx">@i</span>
<span class="nx">@extract_next_token</span><span class="p">()</span>
<span class="nx">@close_indentation</span><span class="p">()</span>
<span class="k">return</span> <span class="nx">@tokens</span> <span class="k">if</span> <span class="nx">o</span><span class="p">.</span><span class="nx">rewrite</span> <span class="o">is</span> <span class="kc">off</span>
<span class="p">(</span><span class="k">new</span> <span class="nx">Rewriter</span><span class="p">()).</span><span class="nx">rewrite</span> <span class="nx">@tokens</span></pre></div> </td> </tr> <tr id="section-7"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-7">#</a> </div> <p>At every position, run through this list of attempted matches,
short-circuiting if any of them succeed. Their order determines precedence:
<code>@literal_token</code> is the fallback catch-all.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">extract_next_token: </span><span class="o">-&gt;</span>
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">extension_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">identifier_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">number_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">heredoc_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">regex_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">comment_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">line_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">whitespace_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">js_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">string_token</span><span class="p">()</span>
<span class="k">return</span> <span class="err">@</span><span class="nx">literal_token</span><span class="p">()</span></pre></div> </td> </tr> <tr id="section-8"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-8">#</a> </div> <h2>Tokenizers</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-9"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-9">#</a> </div> <p>Language extensions get the highest priority, first chance to tag tokens
<span class="k">return</span> <span class="k">if</span> <span class="nx">@extension_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="nx">@identifier_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="nx">@number_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="nx">@heredoc_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="nx">@regex_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="nx">@comment_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="nx">@line_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="nx">@whitespace_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="nx">@js_token</span><span class="p">()</span>
<span class="k">return</span> <span class="k">if</span> <span class="nx">@string_token</span><span class="p">()</span>
<span class="k">return</span> <span class="nx">@literal_token</span><span class="p">()</span></pre></div> </td> </tr> <tr id="section-8"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-8">#</a> </div> <h2>Tokenizers</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-9"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-9">#</a> </div> <p>Language extensions get the highest priority, first chance to tag tokens
as something else.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">extension_token: </span><span class="o">-&gt;</span>
<span class="k">for</span> <span class="nx">extension</span> <span class="k">in</span> <span class="nx">Lexer</span><span class="p">.</span><span class="nx">extensions</span>
<span class="k">return</span> <span class="kc">true</span> <span class="k">if</span> <span class="nx">extension</span><span class="p">.</span><span class="nx">call</span> <span class="k">this</span>
@ -66,73 +66,73 @@ identifiers. Because CoffeeScript reserves a handful of keywords that are
allowed in JavaScript, we're careful not to tag them as keywords when
referenced as property names here, so you can still do <code>jQuery.is()</code> even
though <code>is</code> means <code>===</code> otherwise.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">identifier_token: </span><span class="o">-&gt;</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">id: </span><span class="err">@</span><span class="nx">match</span> <span class="nx">IDENTIFIER</span><span class="p">,</span> <span class="mi">1</span>
<span class="err">@</span><span class="nx">name_access_type</span><span class="p">()</span>
<span class="nv">accessed: </span><span class="nx">include</span> <span class="nx">ACCESSORS</span><span class="p">,</span> <span class="err">@</span><span class="nx">tag</span> <span class="mi">0</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">id: </span><span class="nx">@match</span> <span class="nx">IDENTIFIER</span><span class="p">,</span> <span class="mi">1</span>
<span class="nx">@name_access_type</span><span class="p">()</span>
<span class="nv">accessed: </span><span class="nx">include</span> <span class="nx">ACCESSORS</span><span class="p">,</span> <span class="nx">@tag</span> <span class="mi">0</span>
<span class="nv">tag: </span><span class="s1">&#39;IDENTIFIER&#39;</span>
<span class="nv">tag: </span><span class="nx">id</span><span class="p">.</span><span class="nx">toUpperCase</span><span class="p">()</span> <span class="k">if</span> <span class="o">not</span> <span class="nx">accessed</span> <span class="o">and</span> <span class="nx">include</span><span class="p">(</span><span class="nx">KEYWORDS</span><span class="p">,</span> <span class="nx">id</span><span class="p">)</span>
<span class="err">@</span><span class="nx">identifier_error</span> <span class="nx">id</span> <span class="k">if</span> <span class="nx">include</span> <span class="nx">RESERVED</span><span class="p">,</span> <span class="nx">id</span>
<span class="nv">tag: </span><span class="s1">&#39;LEADING_WHEN&#39;</span> <span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s1">&#39;WHEN&#39;</span> <span class="o">and</span> <span class="nx">include</span> <span class="nx">LINE_BREAK</span><span class="p">,</span> <span class="err">@</span><span class="nx">tag</span><span class="p">()</span>
<span class="err">@</span><span class="nv">i: </span><span class="o">+</span> <span class="nx">id</span><span class="p">.</span><span class="nx">length</span>
<span class="nx">@identifier_error</span> <span class="nx">id</span> <span class="k">if</span> <span class="nx">include</span> <span class="nx">RESERVED</span><span class="p">,</span> <span class="nx">id</span>
<span class="nv">tag: </span><span class="s1">&#39;LEADING_WHEN&#39;</span> <span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s1">&#39;WHEN&#39;</span> <span class="o">and</span> <span class="nx">include</span> <span class="nx">LINE_BREAK</span><span class="p">,</span> <span class="nx">@tag</span><span class="p">()</span>
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">id</span><span class="p">.</span><span class="nx">length</span>
<span class="k">if</span> <span class="o">not</span> <span class="nx">accessed</span>
<span class="nv">tag: id: </span><span class="nx">CONVERSIONS</span><span class="p">[</span><span class="nx">id</span><span class="p">]</span> <span class="k">if</span> <span class="nx">include</span> <span class="nx">COFFEE_ALIASES</span><span class="p">,</span> <span class="nx">id</span>
<span class="k">return</span> <span class="err">@</span><span class="nx">tag_half_assignment</span> <span class="nx">tag</span> <span class="k">if</span> <span class="err">@</span><span class="nx">prev</span><span class="p">()</span> <span class="o">and</span> <span class="err">@</span><span class="nx">prev</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">&#39;ASSIGN&#39;</span> <span class="o">and</span> <span class="nx">include</span> <span class="nx">HALF_ASSIGNMENTS</span><span class="p">,</span> <span class="nx">tag</span>
<span class="err">@</span><span class="nx">token</span> <span class="nx">tag</span><span class="p">,</span> <span class="nx">id</span>
<span class="k">return</span> <span class="nx">@tag_half_assignment</span> <span class="nx">tag</span> <span class="k">if</span> <span class="nx">@prev</span><span class="p">()</span> <span class="o">and</span> <span class="nx">@prev</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">&#39;ASSIGN&#39;</span> <span class="o">and</span> <span class="nx">include</span> <span class="nx">HALF_ASSIGNMENTS</span><span class="p">,</span> <span class="nx">tag</span>
<span class="nx">@token</span> <span class="nx">tag</span><span class="p">,</span> <span class="nx">id</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-11"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-11">#</a> </div> <p>Matches numbers, including decimals, hex, and exponential notation.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">number_token: </span><span class="o">-&gt;</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">number: </span><span class="err">@</span><span class="nx">match</span> <span class="nx">NUMBER</span><span class="p">,</span> <span class="mi">1</span>
<span class="err">@</span><span class="nx">token</span> <span class="s1">&#39;NUMBER&#39;</span><span class="p">,</span> <span class="nx">number</span>
<span class="err">@</span><span class="nv">i: </span><span class="o">+</span> <span class="nx">number</span><span class="p">.</span><span class="nx">length</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">number: </span><span class="nx">@match</span> <span class="nx">NUMBER</span><span class="p">,</span> <span class="mi">1</span>
<span class="nx">@token</span> <span class="s1">&#39;NUMBER&#39;</span><span class="p">,</span> <span class="nx">number</span>
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">number</span><span class="p">.</span><span class="nx">length</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-12"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-12">#</a> </div> <p>Matches strings, including multi-line strings. Ensures that quotation marks
are balanced within the string's contents, and within nested interpolations.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">string_token: </span><span class="o">-&gt;</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nx">starts</span><span class="p">(</span><span class="err">@</span><span class="nx">chunk</span><span class="p">,</span> <span class="s1">&#39;&quot;&#39;</span><span class="p">)</span> <span class="o">or</span> <span class="nx">starts</span><span class="p">(</span><span class="err">@</span><span class="nx">chunk</span><span class="p">,</span> <span class="s2">&quot;&#39;&quot;</span><span class="p">)</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nx">starts</span><span class="p">(</span><span class="nx">@chunk</span><span class="p">,</span> <span class="s1">&#39;&quot;&#39;</span><span class="p">)</span> <span class="o">or</span> <span class="nx">starts</span><span class="p">(</span><span class="nx">@chunk</span><span class="p">,</span> <span class="s2">&quot;&#39;&quot;</span><span class="p">)</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">string:</span>
<span class="err">@</span><span class="nx">balanced_token</span><span class="p">([</span><span class="s1">&#39;&quot;&#39;</span><span class="p">,</span> <span class="s1">&#39;&quot;&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;${&#39;</span><span class="p">,</span> <span class="s1">&#39;}&#39;</span><span class="p">])</span> <span class="o">or</span>
<span class="err">@</span><span class="nx">balanced_token</span> <span class="p">[</span><span class="s2">&quot;&#39;&quot;</span><span class="p">,</span> <span class="s2">&quot;&#39;&quot;</span><span class="p">]</span>
<span class="err">@</span><span class="nx">interpolate_string</span> <span class="nx">string</span><span class="p">.</span><span class="nx">replace</span> <span class="nx">STRING_NEWLINES</span><span class="p">,</span> <span class="s2">&quot; \\\n&quot;</span>
<span class="err">@</span><span class="nv">line: </span><span class="o">+</span> <span class="nx">count</span> <span class="nx">string</span><span class="p">,</span> <span class="s2">&quot;\n&quot;</span>
<span class="err">@</span><span class="nv">i: </span><span class="o">+</span> <span class="nx">string</span><span class="p">.</span><span class="nx">length</span>
<span class="nx">@balanced_token</span><span class="p">([</span><span class="s1">&#39;&quot;&#39;</span><span class="p">,</span> <span class="s1">&#39;&quot;&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;${&#39;</span><span class="p">,</span> <span class="s1">&#39;}&#39;</span><span class="p">])</span> <span class="o">or</span>
<span class="nx">@balanced_token</span> <span class="p">[</span><span class="s2">&quot;&#39;&quot;</span><span class="p">,</span> <span class="s2">&quot;&#39;&quot;</span><span class="p">]</span>
<span class="nx">@interpolate_string</span> <span class="nx">string</span><span class="p">.</span><span class="nx">replace</span> <span class="nx">STRING_NEWLINES</span><span class="p">,</span> <span class="s2">&quot; \\\n&quot;</span>
<span class="vi">@line: </span><span class="o">+</span> <span class="nx">count</span> <span class="nx">string</span><span class="p">,</span> <span class="s2">&quot;\n&quot;</span>
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">string</span><span class="p">.</span><span class="nx">length</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-13"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-13">#</a> </div> <p>Matches heredocs, adjusting indentation to the correct level, as heredocs
preserve whitespace, but ignore indentation to the left.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">heredoc_token: </span><span class="o">-&gt;</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">match: </span><span class="err">@</span><span class="nx">chunk</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">HEREDOC</span><span class="p">)</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">match: </span><span class="nx">@chunk</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">HEREDOC</span><span class="p">)</span>
<span class="nv">quote: </span><span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">].</span><span class="nx">substr</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span>
<span class="nv">doc: </span><span class="err">@</span><span class="nx">sanitize_heredoc</span> <span class="nx">match</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">or</span> <span class="nx">match</span><span class="p">[</span><span class="mi">4</span><span class="p">],</span> <span class="nx">quote</span>
<span class="err">@</span><span class="nx">interpolate_string</span> <span class="s2">&quot;$quote$doc$quote&quot;</span>
<span class="err">@</span><span class="nv">line: </span><span class="o">+</span> <span class="nx">count</span> <span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="s2">&quot;\n&quot;</span>
<span class="err">@</span><span class="nv">i: </span><span class="o">+</span> <span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">].</span><span class="nx">length</span>
<span class="nv">doc: </span><span class="nx">@sanitize_heredoc</span> <span class="nx">match</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">or</span> <span class="nx">match</span><span class="p">[</span><span class="mi">4</span><span class="p">],</span> <span class="nx">quote</span>
<span class="nx">@interpolate_string</span> <span class="s2">&quot;$quote$doc$quote&quot;</span>
<span class="vi">@line: </span><span class="o">+</span> <span class="nx">count</span> <span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="s2">&quot;\n&quot;</span>
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">].</span><span class="nx">length</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-14"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-14">#</a> </div> <p>Matches JavaScript interpolated directly into the source via backticks.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">js_token: </span><span class="o">-&gt;</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nx">starts</span> <span class="err">@</span><span class="nx">chunk</span><span class="p">,</span> <span class="s1">&#39;`&#39;</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">script: </span><span class="err">@</span><span class="nx">balanced_token</span> <span class="p">[</span><span class="s1">&#39;`&#39;</span><span class="p">,</span> <span class="s1">&#39;`&#39;</span><span class="p">]</span>
<span class="err">@</span><span class="nx">token</span> <span class="s1">&#39;JS&#39;</span><span class="p">,</span> <span class="nx">script</span><span class="p">.</span><span class="nx">replace</span> <span class="nx">JS_CLEANER</span><span class="p">,</span> <span class="s1">&#39;&#39;</span>
<span class="err">@</span><span class="nv">i: </span><span class="o">+</span> <span class="nx">script</span><span class="p">.</span><span class="nx">length</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nx">starts</span> <span class="nx">@chunk</span><span class="p">,</span> <span class="s1">&#39;`&#39;</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">script: </span><span class="nx">@balanced_token</span> <span class="p">[</span><span class="s1">&#39;`&#39;</span><span class="p">,</span> <span class="s1">&#39;`&#39;</span><span class="p">]</span>
<span class="nx">@token</span> <span class="s1">&#39;JS&#39;</span><span class="p">,</span> <span class="nx">script</span><span class="p">.</span><span class="nx">replace</span> <span class="nx">JS_CLEANER</span><span class="p">,</span> <span class="s1">&#39;&#39;</span>
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">script</span><span class="p">.</span><span class="nx">length</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-15"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-15">#</a> </div> <p>Matches regular expression literals. Lexing regular expressions is difficult
to distinguish from division, so we borrow some basic heuristics from
JavaScript and Ruby, borrow slash balancing from <code>@balanced_token</code>, and
borrow interpolation from <code>@interpolate_string</code>.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">regex_token: </span><span class="o">-&gt;</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="err">@</span><span class="nx">chunk</span><span class="p">.</span><span class="nx">match</span> <span class="nx">REGEX_START</span>
<span class="k">return</span> <span class="kc">false</span> <span class="k">if</span> <span class="nx">include</span> <span class="nx">NOT_REGEX</span><span class="p">,</span> <span class="err">@</span><span class="nx">tag</span><span class="p">()</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">regex: </span><span class="err">@</span><span class="nx">balanced_token</span> <span class="p">[</span><span class="s1">&#39;/&#39;</span><span class="p">,</span> <span class="s1">&#39;/&#39;</span><span class="p">]</span>
<span class="nv">regex: </span><span class="o">+</span> <span class="p">(</span><span class="nv">flags: </span><span class="err">@</span><span class="nx">chunk</span><span class="p">.</span><span class="nx">substr</span><span class="p">(</span><span class="nx">regex</span><span class="p">.</span><span class="nx">length</span><span class="p">).</span><span class="nx">match</span> <span class="nx">REGEX_FLAGS</span><span class="p">)</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nx">@chunk</span><span class="p">.</span><span class="nx">match</span> <span class="nx">REGEX_START</span>
<span class="k">return</span> <span class="kc">false</span> <span class="k">if</span> <span class="nx">include</span> <span class="nx">NOT_REGEX</span><span class="p">,</span> <span class="nx">@tag</span><span class="p">()</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">regex: </span><span class="nx">@balanced_token</span> <span class="p">[</span><span class="s1">&#39;/&#39;</span><span class="p">,</span> <span class="s1">&#39;/&#39;</span><span class="p">]</span>
<span class="nv">regex: </span><span class="o">+</span> <span class="p">(</span><span class="nv">flags: </span><span class="nx">@chunk</span><span class="p">.</span><span class="nx">substr</span><span class="p">(</span><span class="nx">regex</span><span class="p">.</span><span class="nx">length</span><span class="p">).</span><span class="nx">match</span> <span class="nx">REGEX_FLAGS</span><span class="p">)</span>
<span class="k">if</span> <span class="nx">regex</span><span class="p">.</span><span class="nx">match</span> <span class="nx">REGEX_INTERPOLATION</span>
<span class="nv">str: </span><span class="nx">regex</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">1</span><span class="p">).</span><span class="nx">split</span><span class="p">(</span><span class="s1">&#39;/&#39;</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
<span class="nv">str: </span><span class="nx">str</span><span class="p">.</span><span class="nx">replace</span> <span class="nx">REGEX_ESCAPE</span><span class="p">,</span> <span class="p">(</span><span class="nx">escaped</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="s1">&#39;\\&#39;</span> <span class="o">+</span> <span class="nx">escaped</span>
<span class="err">@</span><span class="nv">tokens: </span><span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">concat</span> <span class="p">[[</span><span class="s1">&#39;(&#39;</span><span class="p">,</span> <span class="s1">&#39;(&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;NEW&#39;</span><span class="p">,</span> <span class="s1">&#39;new&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;IDENTIFIER&#39;</span><span class="p">,</span> <span class="s1">&#39;RegExp&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;CALL_START&#39;</span><span class="p">,</span> <span class="s1">&#39;(&#39;</span><span class="p">]]</span>
<span class="err">@</span><span class="nx">interpolate_string</span> <span class="s2">&quot;\&quot;$str\&quot;&quot;</span><span class="p">,</span> <span class="kc">yes</span>
<span class="err">@</span><span class="nv">tokens: </span><span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">concat</span> <span class="p">[[</span><span class="s1">&#39;,&#39;</span><span class="p">,</span> <span class="s1">&#39;,&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;STRING&#39;</span><span class="p">,</span> <span class="s2">&quot;\&quot;$flags\&quot;&quot;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;)&#39;</span><span class="p">,</span> <span class="s1">&#39;)&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;)&#39;</span><span class="p">,</span> <span class="s1">&#39;)&#39;</span><span class="p">]]</span>
<span class="vi">@tokens: </span><span class="nx">@tokens</span><span class="p">.</span><span class="nx">concat</span> <span class="p">[[</span><span class="s1">&#39;(&#39;</span><span class="p">,</span> <span class="s1">&#39;(&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;NEW&#39;</span><span class="p">,</span> <span class="s1">&#39;new&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;IDENTIFIER&#39;</span><span class="p">,</span> <span class="s1">&#39;RegExp&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;CALL_START&#39;</span><span class="p">,</span> <span class="s1">&#39;(&#39;</span><span class="p">]]</span>
<span class="nx">@interpolate_string</span> <span class="s2">&quot;\&quot;$str\&quot;&quot;</span><span class="p">,</span> <span class="kc">yes</span>
<span class="vi">@tokens: </span><span class="nx">@tokens</span><span class="p">.</span><span class="nx">concat</span> <span class="p">[[</span><span class="s1">&#39;,&#39;</span><span class="p">,</span> <span class="s1">&#39;,&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;STRING&#39;</span><span class="p">,</span> <span class="s2">&quot;\&quot;$flags\&quot;&quot;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;)&#39;</span><span class="p">,</span> <span class="s1">&#39;)&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;)&#39;</span><span class="p">,</span> <span class="s1">&#39;)&#39;</span><span class="p">]]</span>
<span class="k">else</span>
<span class="err">@</span><span class="nx">token</span> <span class="s1">&#39;REGEX&#39;</span><span class="p">,</span> <span class="nx">regex</span>
<span class="err">@</span><span class="nv">i: </span><span class="o">+</span> <span class="nx">regex</span><span class="p">.</span><span class="nx">length</span>
<span class="nx">@token</span> <span class="s1">&#39;REGEX&#39;</span><span class="p">,</span> <span class="nx">regex</span>
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">regex</span><span class="p">.</span><span class="nx">length</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-16"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-16">#</a> </div> <p>Matches a token in which which the passed delimiter pairs must be correctly
balanced (ie. strings, JS literals).</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">balanced_token: </span><span class="p">(</span><span class="nx">delimited</span><span class="p">...)</span> <span class="o">-&gt;</span>
<span class="nx">balanced_string</span> <span class="err">@</span><span class="nx">chunk</span><span class="p">,</span> <span class="nx">delimited</span></pre></div> </td> </tr> <tr id="section-17"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-17">#</a> </div> <p>Matches and conumes comments. We pass through comments into JavaScript,
<span class="nx">balanced_string</span> <span class="nx">@chunk</span><span class="p">,</span> <span class="nx">delimited</span></pre></div> </td> </tr> <tr id="section-17"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-17">#</a> </div> <p>Matches and conumes comments. We pass through comments into JavaScript,
so they're treated as real tokens, like any other part of the language.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">comment_token: </span><span class="o">-&gt;</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">comment: </span><span class="err">@</span><span class="nx">match</span> <span class="nx">COMMENT</span><span class="p">,</span> <span class="mi">1</span>
<span class="err">@</span><span class="nv">line: </span><span class="o">+</span> <span class="p">(</span><span class="nx">comment</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">MULTILINER</span><span class="p">)</span> <span class="o">or</span> <span class="p">[]).</span><span class="nx">length</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">comment: </span><span class="nx">@match</span> <span class="nx">COMMENT</span><span class="p">,</span> <span class="mi">1</span>
<span class="vi">@line: </span><span class="o">+</span> <span class="p">(</span><span class="nx">comment</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">MULTILINER</span><span class="p">)</span> <span class="o">or</span> <span class="p">[]).</span><span class="nx">length</span>
<span class="nv">lines: </span><span class="nx">compact</span> <span class="nx">comment</span><span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="nx">COMMENT_CLEANER</span><span class="p">,</span> <span class="s1">&#39;&#39;</span><span class="p">).</span><span class="nx">split</span> <span class="nx">MULTILINER</span>
<span class="nv">i: </span><span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
<span class="k">if</span> <span class="err">@</span><span class="nx">unfinished</span><span class="p">()</span>
<span class="nv">i: </span><span class="o">-</span> <span class="mi">1</span> <span class="k">while</span> <span class="err">@</span><span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">]</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">include</span> <span class="nx">LINE_BREAK</span><span class="p">,</span> <span class="err">@</span><span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="p">[</span><span class="s1">&#39;COMMENT&#39;</span><span class="p">,</span> <span class="nx">lines</span><span class="p">,</span> <span class="err">@</span><span class="nx">line</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;TERMINATOR&#39;</span><span class="p">,</span> <span class="s1">&#39;\n&#39;</span><span class="p">,</span> <span class="err">@</span><span class="nx">line</span><span class="p">])</span>
<span class="err">@</span><span class="nv">i: </span><span class="o">+</span> <span class="nx">comment</span><span class="p">.</span><span class="nx">length</span>
<span class="nv">i: </span><span class="nx">@tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
<span class="k">if</span> <span class="nx">@unfinished</span><span class="p">()</span>
<span class="nv">i: </span><span class="o">-</span> <span class="mi">1</span> <span class="k">while</span> <span class="nx">@tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">]</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">include</span> <span class="nx">LINE_BREAK</span><span class="p">,</span> <span class="nx">@tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="p">[</span><span class="s1">&#39;COMMENT&#39;</span><span class="p">,</span> <span class="nx">lines</span><span class="p">,</span> <span class="nx">@line</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;TERMINATOR&#39;</span><span class="p">,</span> <span class="s1">&#39;\n&#39;</span><span class="p">,</span> <span class="nx">@line</span><span class="p">])</span>
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">comment</span><span class="p">.</span><span class="nx">length</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-18"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-18">#</a> </div> <p>Matches newlines, indents, and outdents, and determines which is which.
If we can detect that the current line is continued onto the the next line,
then the newline is suppressed:</p>
@ -144,103 +144,103 @@ then the newline is suppressed:</p>
<p>Keeps track of the level of indentation, because a single outdent token
can close multiple indents, so we need to know how far in we happen to be.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">line_token: </span><span class="o">-&gt;</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">indent: </span><span class="err">@</span><span class="nx">match</span> <span class="nx">MULTI_DENT</span><span class="p">,</span> <span class="mi">1</span>
<span class="err">@</span><span class="nv">line: </span><span class="o">+</span> <span class="nx">indent</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">MULTILINER</span><span class="p">).</span><span class="nx">length</span>
<span class="err">@</span><span class="nx">i</span> <span class="o">:</span> <span class="o">+</span> <span class="nx">indent</span><span class="p">.</span><span class="nx">length</span>
<span class="nv">prev: </span><span class="err">@</span><span class="nx">prev</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">indent: </span><span class="nx">@match</span> <span class="nx">MULTI_DENT</span><span class="p">,</span> <span class="mi">1</span>
<span class="vi">@line: </span><span class="o">+</span> <span class="nx">indent</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">MULTILINER</span><span class="p">).</span><span class="nx">length</span>
<span class="nx">@i</span> <span class="o">:</span> <span class="o">+</span> <span class="nx">indent</span><span class="p">.</span><span class="nx">length</span>
<span class="nv">prev: </span><span class="nx">@prev</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
<span class="nv">size: </span><span class="nx">indent</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">LAST_DENTS</span><span class="p">).</span><span class="nx">reverse</span><span class="p">()[</span><span class="mi">0</span><span class="p">].</span><span class="nx">match</span><span class="p">(</span><span class="nx">LAST_DENT</span><span class="p">)[</span><span class="mi">1</span><span class="p">].</span><span class="nx">length</span>
<span class="nv">next_character: </span><span class="err">@</span><span class="nx">chunk</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">MULTI_DENT</span><span class="p">)[</span><span class="mi">4</span><span class="p">]</span>
<span class="nv">no_newlines: </span><span class="nx">next_character</span> <span class="o">is</span> <span class="s1">&#39;.&#39;</span> <span class="o">or</span> <span class="err">@</span><span class="nx">unfinished</span><span class="p">()</span>
<span class="k">if</span> <span class="nx">size</span> <span class="o">is</span> <span class="err">@</span><span class="nx">indent</span>
<span class="k">return</span> <span class="err">@</span><span class="nx">suppress_newlines</span><span class="p">()</span> <span class="k">if</span> <span class="nx">no_newlines</span>
<span class="k">return</span> <span class="err">@</span><span class="nx">newline_token</span> <span class="nx">indent</span>
<span class="k">else</span> <span class="k">if</span> <span class="nx">size</span> <span class="o">&gt;</span> <span class="err">@</span><span class="nx">indent</span>
<span class="k">return</span> <span class="err">@</span><span class="nx">suppress_newlines</span><span class="p">()</span> <span class="k">if</span> <span class="nx">no_newlines</span>
<span class="nv">diff: </span><span class="nx">size</span> <span class="o">-</span> <span class="err">@</span><span class="nx">indent</span>
<span class="err">@</span><span class="nx">token</span> <span class="s1">&#39;INDENT&#39;</span><span class="p">,</span> <span class="nx">diff</span>
<span class="err">@</span><span class="nx">indents</span><span class="p">.</span><span class="nx">push</span> <span class="nx">diff</span>
<span class="nv">next_character: </span><span class="nx">@chunk</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">MULTI_DENT</span><span class="p">)[</span><span class="mi">4</span><span class="p">]</span>
<span class="nv">no_newlines: </span><span class="nx">next_character</span> <span class="o">is</span> <span class="s1">&#39;.&#39;</span> <span class="o">or</span> <span class="nx">@unfinished</span><span class="p">()</span>
<span class="k">if</span> <span class="nx">size</span> <span class="o">is</span> <span class="nx">@indent</span>
<span class="k">return</span> <span class="nx">@suppress_newlines</span><span class="p">()</span> <span class="k">if</span> <span class="nx">no_newlines</span>
<span class="k">return</span> <span class="nx">@newline_token</span> <span class="nx">indent</span>
<span class="k">else</span> <span class="k">if</span> <span class="nx">size</span> <span class="o">&gt;</span> <span class="nx">@indent</span>
<span class="k">return</span> <span class="nx">@suppress_newlines</span><span class="p">()</span> <span class="k">if</span> <span class="nx">no_newlines</span>
<span class="nv">diff: </span><span class="nx">size</span> <span class="o">-</span> <span class="nx">@indent</span>
<span class="nx">@token</span> <span class="s1">&#39;INDENT&#39;</span><span class="p">,</span> <span class="nx">diff</span>
<span class="nx">@indents</span><span class="p">.</span><span class="nx">push</span> <span class="nx">diff</span>
<span class="k">else</span>
<span class="err">@</span><span class="nx">outdent_token</span> <span class="err">@</span><span class="nx">indent</span> <span class="o">-</span> <span class="nx">size</span><span class="p">,</span> <span class="nx">no_newlines</span>
<span class="err">@</span><span class="nv">indent: </span><span class="nx">size</span>
<span class="nx">@outdent_token</span> <span class="nx">@indent</span> <span class="o">-</span> <span class="nx">size</span><span class="p">,</span> <span class="nx">no_newlines</span>
<span class="vi">@indent: </span><span class="nx">size</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-19"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-19">#</a> </div> <p>Record an outdent token or multiple tokens, if we happen to be moving back
inwards past several recorded indents.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">outdent_token: </span><span class="p">(</span><span class="nx">move_out</span><span class="p">,</span> <span class="nx">no_newlines</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="k">while</span> <span class="nx">move_out</span> <span class="o">&gt;</span> <span class="mi">0</span> <span class="o">and</span> <span class="err">@</span><span class="nx">indents</span><span class="p">.</span><span class="nx">length</span>
<span class="nv">last_indent: </span><span class="err">@</span><span class="nx">indents</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
<span class="err">@</span><span class="nx">token</span> <span class="s1">&#39;OUTDENT&#39;</span><span class="p">,</span> <span class="nx">last_indent</span>
<span class="k">while</span> <span class="nx">move_out</span> <span class="o">&gt;</span> <span class="mi">0</span> <span class="o">and</span> <span class="nx">@indents</span><span class="p">.</span><span class="nx">length</span>
<span class="nv">last_indent: </span><span class="nx">@indents</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
<span class="nx">@token</span> <span class="s1">&#39;OUTDENT&#39;</span><span class="p">,</span> <span class="nx">last_indent</span>
<span class="nv">move_out: </span><span class="o">-</span> <span class="nx">last_indent</span>
<span class="err">@</span><span class="nx">token</span> <span class="s1">&#39;TERMINATOR&#39;</span><span class="p">,</span> <span class="s2">&quot;\n&quot;</span> <span class="nx">unless</span> <span class="err">@</span><span class="nx">tag</span><span class="p">()</span> <span class="o">is</span> <span class="s1">&#39;TERMINATOR&#39;</span> <span class="o">or</span> <span class="nx">no_newlines</span>
<span class="nx">@token</span> <span class="s1">&#39;TERMINATOR&#39;</span><span class="p">,</span> <span class="s2">&quot;\n&quot;</span> <span class="nx">unless</span> <span class="nx">@tag</span><span class="p">()</span> <span class="o">is</span> <span class="s1">&#39;TERMINATOR&#39;</span> <span class="o">or</span> <span class="nx">no_newlines</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-20"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-20">#</a> </div> <p>Matches and consumes non-meaningful whitespace. Tag the previous token
as being "spaced", because there are some cases where it makes a difference.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">whitespace_token: </span><span class="o">-&gt;</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">space: </span><span class="err">@</span><span class="nx">match</span> <span class="nx">WHITESPACE</span><span class="p">,</span> <span class="mi">1</span>
<span class="nv">prev: </span><span class="err">@</span><span class="nx">prev</span><span class="p">()</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">space: </span><span class="nx">@match</span> <span class="nx">WHITESPACE</span><span class="p">,</span> <span class="mi">1</span>
<span class="nv">prev: </span><span class="nx">@prev</span><span class="p">()</span>
<span class="nv">prev.spaced: </span><span class="kc">true</span> <span class="k">if</span> <span class="nx">prev</span>
<span class="err">@</span><span class="nv">i: </span><span class="o">+</span> <span class="nx">space</span><span class="p">.</span><span class="nx">length</span>
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">space</span><span class="p">.</span><span class="nx">length</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-21"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-21">#</a> </div> <p>Generate a newline token. Consecutive newlines get merged together.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">newline_token: </span><span class="p">(</span><span class="nx">newlines</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="err">@</span><span class="nx">token</span> <span class="s1">&#39;TERMINATOR&#39;</span><span class="p">,</span> <span class="s2">&quot;\n&quot;</span> <span class="nx">unless</span> <span class="err">@</span><span class="nx">tag</span><span class="p">()</span> <span class="o">is</span> <span class="s1">&#39;TERMINATOR&#39;</span>
<span class="nx">@token</span> <span class="s1">&#39;TERMINATOR&#39;</span><span class="p">,</span> <span class="s2">&quot;\n&quot;</span> <span class="nx">unless</span> <span class="nx">@tag</span><span class="p">()</span> <span class="o">is</span> <span class="s1">&#39;TERMINATOR&#39;</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-22"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-22">#</a> </div> <p>Use a <code>\</code> at a line-ending to suppress the newline.
The slash is removed here once its job is done.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">suppress_newlines: </span><span class="o">-&gt;</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span> <span class="k">if</span> <span class="err">@</span><span class="nx">value</span><span class="p">()</span> <span class="o">is</span> <span class="s2">&quot;\\&quot;</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span> <span class="k">if</span> <span class="nx">@value</span><span class="p">()</span> <span class="o">is</span> <span class="s2">&quot;\\&quot;</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-23"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-23">#</a> </div> <p>We treat all other single characters as a token. Eg.: <code>( ) , . !</code>
Multi-character operators are also literal tokens, so that Jison can assign
the proper order of operations. There are some symbols that we tag specially
here. <code>;</code> and newlines are both treated as a <code>TERMINATOR</code>, we distinguish
parentheses that indicate a method call from regular parentheses, and so on.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">literal_token: </span><span class="o">-&gt;</span>
<span class="nv">match: </span><span class="err">@</span><span class="nx">chunk</span><span class="p">.</span><span class="nx">match</span> <span class="nx">OPERATOR</span>
<span class="nv">match: </span><span class="nx">@chunk</span><span class="p">.</span><span class="nx">match</span> <span class="nx">OPERATOR</span>
<span class="nv">value: </span><span class="nx">match</span> <span class="o">and</span> <span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="nv">space: </span><span class="nx">match</span> <span class="o">and</span> <span class="nx">match</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
<span class="err">@</span><span class="nx">tag_parameters</span><span class="p">()</span> <span class="k">if</span> <span class="nx">value</span> <span class="o">and</span> <span class="nx">value</span><span class="p">.</span><span class="nx">match</span> <span class="nx">CODE</span>
<span class="nv">value: </span><span class="o">or</span> <span class="err">@</span><span class="nx">chunk</span><span class="p">.</span><span class="nx">substr</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span>
<span class="nv">prev_spaced: </span><span class="err">@</span><span class="nx">prev</span><span class="p">()</span> <span class="o">and</span> <span class="err">@</span><span class="nx">prev</span><span class="p">().</span><span class="nx">spaced</span>
<span class="nx">@tag_parameters</span><span class="p">()</span> <span class="k">if</span> <span class="nx">value</span> <span class="o">and</span> <span class="nx">value</span><span class="p">.</span><span class="nx">match</span> <span class="nx">CODE</span>
<span class="nv">value: </span><span class="o">or</span> <span class="nx">@chunk</span><span class="p">.</span><span class="nx">substr</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span>
<span class="nv">prev_spaced: </span><span class="nx">@prev</span><span class="p">()</span> <span class="o">and</span> <span class="nx">@prev</span><span class="p">().</span><span class="nx">spaced</span>
<span class="nv">tag: </span><span class="nx">value</span>
<span class="k">if</span> <span class="nx">value</span><span class="p">.</span><span class="nx">match</span> <span class="nx">ASSIGNMENT</span>
<span class="nv">tag: </span><span class="s1">&#39;ASSIGN&#39;</span>
<span class="err">@</span><span class="nx">assignment_error</span><span class="p">()</span> <span class="k">if</span> <span class="nx">include</span> <span class="nx">JS_FORBIDDEN</span><span class="p">,</span> <span class="err">@</span><span class="nx">value</span>
<span class="nx">@assignment_error</span><span class="p">()</span> <span class="k">if</span> <span class="nx">include</span> <span class="nx">JS_FORBIDDEN</span><span class="p">,</span> <span class="nx">@value</span>
<span class="k">else</span> <span class="k">if</span> <span class="nx">value</span> <span class="o">is</span> <span class="s1">&#39;;&#39;</span>
<span class="nv">tag: </span><span class="s1">&#39;TERMINATOR&#39;</span>
<span class="k">else</span> <span class="k">if</span> <span class="nx">value</span> <span class="o">is</span> <span class="s1">&#39;[&#39;</span> <span class="o">and</span> <span class="err">@</span><span class="nx">tag</span><span class="p">()</span> <span class="o">is</span> <span class="s1">&#39;?&#39;</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">prev_spaced</span>
<span class="k">else</span> <span class="k">if</span> <span class="nx">value</span> <span class="o">is</span> <span class="s1">&#39;[&#39;</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">()</span> <span class="o">is</span> <span class="s1">&#39;?&#39;</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">prev_spaced</span>
<span class="nv">tag: </span><span class="s1">&#39;SOAKED_INDEX_START&#39;</span>
<span class="err">@</span><span class="nv">soaked_index: </span><span class="kc">true</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
<span class="k">else</span> <span class="k">if</span> <span class="nx">value</span> <span class="o">is</span> <span class="s1">&#39;]&#39;</span> <span class="o">and</span> <span class="err">@</span><span class="nx">soaked_index</span>
<span class="vi">@soaked_index: </span><span class="kc">true</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
<span class="k">else</span> <span class="k">if</span> <span class="nx">value</span> <span class="o">is</span> <span class="s1">&#39;]&#39;</span> <span class="o">and</span> <span class="nx">@soaked_index</span>
<span class="nv">tag: </span><span class="s1">&#39;SOAKED_INDEX_END&#39;</span>
<span class="err">@</span><span class="nv">soaked_index: </span><span class="kc">false</span>
<span class="k">else</span> <span class="k">if</span> <span class="nx">include</span><span class="p">(</span><span class="nx">CALLABLE</span><span class="p">,</span> <span class="err">@</span><span class="nx">tag</span><span class="p">())</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">prev_spaced</span>
<span class="vi">@soaked_index: </span><span class="kc">false</span>
<span class="k">else</span> <span class="k">if</span> <span class="nx">include</span><span class="p">(</span><span class="nx">CALLABLE</span><span class="p">,</span> <span class="nx">@tag</span><span class="p">())</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">prev_spaced</span>
<span class="nv">tag: </span><span class="s1">&#39;CALL_START&#39;</span> <span class="k">if</span> <span class="nx">value</span> <span class="o">is</span> <span class="s1">&#39;(&#39;</span>
<span class="nv">tag: </span><span class="s1">&#39;INDEX_START&#39;</span> <span class="k">if</span> <span class="nx">value</span> <span class="o">is</span> <span class="s1">&#39;[&#39;</span>
<span class="err">@</span><span class="nv">i: </span><span class="o">+</span> <span class="nx">value</span><span class="p">.</span><span class="nx">length</span>
<span class="k">return</span> <span class="err">@</span><span class="nx">tag_half_assignment</span> <span class="nx">tag</span> <span class="k">if</span> <span class="nx">space</span> <span class="o">and</span> <span class="nx">prev_spaced</span> <span class="o">and</span> <span class="err">@</span><span class="nx">prev</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">&#39;ASSIGN&#39;</span> <span class="o">and</span> <span class="nx">include</span> <span class="nx">HALF_ASSIGNMENTS</span><span class="p">,</span> <span class="nx">tag</span>
<span class="err">@</span><span class="nx">token</span> <span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span>
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">value</span><span class="p">.</span><span class="nx">length</span>
<span class="k">return</span> <span class="nx">@tag_half_assignment</span> <span class="nx">tag</span> <span class="k">if</span> <span class="nx">space</span> <span class="o">and</span> <span class="nx">prev_spaced</span> <span class="o">and</span> <span class="nx">@prev</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">&#39;ASSIGN&#39;</span> <span class="o">and</span> <span class="nx">include</span> <span class="nx">HALF_ASSIGNMENTS</span><span class="p">,</span> <span class="nx">tag</span>
<span class="nx">@token</span> <span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-24"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-24">#</a> </div> <h2>Token Manipulators</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-25"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-25">#</a> </div> <p>As we consume a new <code>IDENTIFIER</code>, look at the previous token to determine
if it's a special kind of accessor.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">name_access_type: </span><span class="o">-&gt;</span>
<span class="err">@</span><span class="nx">tag</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="s1">&#39;PROTOTYPE_ACCESS&#39;</span><span class="p">)</span> <span class="k">if</span> <span class="err">@</span><span class="nx">value</span><span class="p">()</span> <span class="o">is</span> <span class="s1">&#39;::&#39;</span>
<span class="k">if</span> <span class="err">@</span><span class="nx">value</span><span class="p">()</span> <span class="o">is</span> <span class="s1">&#39;.&#39;</span> <span class="o">and</span> <span class="o">not</span> <span class="p">(</span><span class="err">@</span><span class="nx">value</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="o">is</span> <span class="s1">&#39;.&#39;</span><span class="p">)</span>
<span class="k">if</span> <span class="err">@</span><span class="nx">tag</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="o">is</span> <span class="s1">&#39;?&#39;</span>
<span class="err">@</span><span class="nx">tag</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="s1">&#39;SOAK_ACCESS&#39;</span><span class="p">)</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span><span class="p">(</span><span class="o">-</span><span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="nx">@tag</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="s1">&#39;PROTOTYPE_ACCESS&#39;</span><span class="p">)</span> <span class="k">if</span> <span class="nx">@value</span><span class="p">()</span> <span class="o">is</span> <span class="s1">&#39;::&#39;</span>
<span class="k">if</span> <span class="nx">@value</span><span class="p">()</span> <span class="o">is</span> <span class="s1">&#39;.&#39;</span> <span class="o">and</span> <span class="o">not</span> <span class="p">(</span><span class="nx">@value</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="o">is</span> <span class="s1">&#39;.&#39;</span><span class="p">)</span>
<span class="k">if</span> <span class="nx">@tag</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="o">is</span> <span class="s1">&#39;?&#39;</span>
<span class="nx">@tag</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="s1">&#39;SOAK_ACCESS&#39;</span><span class="p">)</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span><span class="p">(</span><span class="o">-</span><span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="k">else</span>
<span class="err">@</span><span class="nx">tag</span> <span class="mi">1</span><span class="p">,</span> <span class="s1">&#39;PROPERTY_ACCESS&#39;</span></pre></div> </td> </tr> <tr id="section-26"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-26">#</a> </div> <p>Sanitize a heredoc by escaping internal double quotes and erasing all
<span class="nx">@tag</span> <span class="mi">1</span><span class="p">,</span> <span class="s1">&#39;PROPERTY_ACCESS&#39;</span></pre></div> </td> </tr> <tr id="section-26"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-26">#</a> </div> <p>Sanitize a heredoc by escaping internal double quotes and erasing all
external indentation on the left-hand side.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">sanitize_heredoc: </span><span class="p">(</span><span class="nx">doc</span><span class="p">,</span> <span class="nx">quote</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="nv">indent: </span><span class="p">(</span><span class="nx">doc</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">HEREDOC_INDENT</span><span class="p">)</span> <span class="o">or</span> <span class="p">[</span><span class="s1">&#39;&#39;</span><span class="p">]).</span><span class="nx">sort</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span>
<span class="nx">doc</span><span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="k">new</span> <span class="nb">RegExp</span><span class="p">(</span><span class="s2">&quot;^&quot;</span> <span class="o">+</span><span class="nx">indent</span><span class="p">,</span> <span class="s1">&#39;gm&#39;</span><span class="p">),</span> <span class="s1">&#39;&#39;</span><span class="p">)</span>
<span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="nx">MULTILINER</span><span class="p">,</span> <span class="s2">&quot;\\n&quot;</span><span class="p">)</span>
<span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="k">new</span> <span class="nb">RegExp</span><span class="p">(</span><span class="nx">quote</span><span class="p">,</span> <span class="s1">&#39;g&#39;</span><span class="p">),</span> <span class="s1">&#39;\\&quot;&#39;</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-27"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-27">#</a> </div> <p>Tag a half assignment.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tag_half_assignment: </span><span class="p">(</span><span class="nx">tag</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="nv">last: </span><span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s2">&quot;$tag=&quot;</span><span class="p">,</span> <span class="s2">&quot;$tag=&quot;</span><span class="p">,</span> <span class="nx">last</span><span class="p">[</span><span class="mi">2</span><span class="p">]]</span>
<span class="nv">last: </span><span class="nx">@tokens</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s2">&quot;$tag=&quot;</span><span class="p">,</span> <span class="s2">&quot;$tag=&quot;</span><span class="p">,</span> <span class="nx">last</span><span class="p">[</span><span class="mi">2</span><span class="p">]]</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-28"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-28">#</a> </div> <p>A source of ambiguity in our grammar used to be parameter lists in function
definitions versus argument lists in function calls. Walk backwards, tagging
parameters specially in order to make things easier for the parser.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tag_parameters: </span><span class="o">-&gt;</span>
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">tag</span><span class="p">()</span> <span class="o">isnt</span> <span class="s1">&#39;)&#39;</span>
<span class="k">return</span> <span class="k">if</span> <span class="nx">@tag</span><span class="p">()</span> <span class="o">isnt</span> <span class="s1">&#39;)&#39;</span>
<span class="nv">i: </span><span class="mi">0</span>
<span class="k">while</span> <span class="kc">true</span>
<span class="nv">i: </span><span class="o">+</span> <span class="mi">1</span>
<span class="nv">tok: </span><span class="err">@</span><span class="nx">prev</span> <span class="nx">i</span>
<span class="nv">tok: </span><span class="nx">@prev</span> <span class="nx">i</span>
<span class="k">return</span> <span class="k">if</span> <span class="o">not</span> <span class="nx">tok</span>
<span class="k">switch</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">when</span> <span class="s1">&#39;IDENTIFIER&#39;</span> <span class="k">then</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">:</span> <span class="s1">&#39;PARAM&#39;</span>
<span class="k">when</span> <span class="s1">&#39;)&#39;</span> <span class="k">then</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">:</span> <span class="s1">&#39;PARAM_END&#39;</span>
<span class="k">when</span> <span class="s1">&#39;(&#39;</span> <span class="k">then</span> <span class="k">return</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">:</span> <span class="s1">&#39;PARAM_START&#39;</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-29"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-29">#</a> </div> <p>Close up all remaining open blocks at the end of the file.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">close_indentation: </span><span class="o">-&gt;</span>
<span class="err">@</span><span class="nx">outdent_token</span> <span class="err">@</span><span class="nx">indent</span></pre></div> </td> </tr> <tr id="section-30"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-30">#</a> </div> <p>The error for when you try to use a forbidden word in JavaScript as
<span class="nx">@outdent_token</span> <span class="nx">@indent</span></pre></div> </td> </tr> <tr id="section-30"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-30">#</a> </div> <p>The error for when you try to use a forbidden word in JavaScript as
an identifier.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">identifier_error: </span><span class="p">(</span><span class="nx">word</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span> <span class="s2">&quot;SyntaxError: Reserved word \&quot;$word\&quot; on line ${@line + 1}&quot;</span></pre></div> </td> </tr> <tr id="section-31"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-31">#</a> </div> <p>The error for when you try to assign to a reserved word in JavaScript,
like "function" or "default".</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">assignment_error: </span><span class="o">-&gt;</span>
@ -256,7 +256,7 @@ for substitution of bare variables as well as arbitrary expressions.</p>
new Lexer, tokenize the interpolated contents, and merge them into the
token stream.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">interpolate_string: </span><span class="p">(</span><span class="nx">str</span><span class="p">,</span> <span class="nx">escape_quotes</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="k">if</span> <span class="nx">str</span><span class="p">.</span><span class="nx">length</span> <span class="o">&lt;</span> <span class="mi">3</span> <span class="o">or</span> <span class="o">not</span> <span class="nx">starts</span> <span class="nx">str</span><span class="p">,</span> <span class="s1">&#39;&quot;&#39;</span>
<span class="err">@</span><span class="nx">token</span> <span class="s1">&#39;STRING&#39;</span><span class="p">,</span> <span class="nx">str</span>
<span class="nx">@token</span> <span class="s1">&#39;STRING&#39;</span><span class="p">,</span> <span class="nx">str</span>
<span class="k">else</span>
<span class="nv">lexer: </span> <span class="k">new</span> <span class="nx">Lexer</span><span class="p">()</span>
<span class="nv">tokens: </span> <span class="p">[]</span>
@ -276,7 +276,7 @@ token stream.</p> </td> <td class="code">
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">&#39;STRING&#39;</span><span class="p">,</span> <span class="s2">&quot;$quote${ str.substring(pi, i) }$quote&quot;</span><span class="p">]</span> <span class="k">if</span> <span class="nx">pi</span> <span class="o">&lt;</span> <span class="nx">i</span>
<span class="nv">inner: </span><span class="nx">expr</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="nx">expr</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span>
<span class="k">if</span> <span class="nx">inner</span><span class="p">.</span><span class="nx">length</span>
<span class="nv">nested: </span><span class="nx">lexer</span><span class="p">.</span><span class="nx">tokenize</span> <span class="s2">&quot;($inner)&quot;</span><span class="p">,</span> <span class="p">{</span><span class="nv">rewrite: </span><span class="kc">no</span><span class="p">,</span> <span class="nv">line: </span><span class="err">@</span><span class="nx">line</span><span class="p">}</span>
<span class="nv">nested: </span><span class="nx">lexer</span><span class="p">.</span><span class="nx">tokenize</span> <span class="s2">&quot;($inner)&quot;</span><span class="p">,</span> <span class="p">{</span><span class="nv">rewrite: </span><span class="kc">no</span><span class="p">,</span> <span class="nv">line: </span><span class="nx">@line</span><span class="p">}</span>
<span class="nx">nested</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">&#39;TOKENS&#39;</span><span class="p">,</span> <span class="nx">nested</span><span class="p">]</span>
<span class="k">else</span>
@ -289,28 +289,28 @@ token stream.</p> </td> <td class="code">
<span class="k">for</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">i</span> <span class="k">in</span> <span class="nx">tokens</span>
<span class="p">[</span><span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span><span class="p">]</span><span class="o">:</span> <span class="nx">token</span>
<span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s1">&#39;TOKENS&#39;</span>
<span class="err">@</span><span class="nv">tokens: </span><span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">concat</span> <span class="nx">value</span>
<span class="vi">@tokens: </span><span class="nx">@tokens</span><span class="p">.</span><span class="nx">concat</span> <span class="nx">value</span>
<span class="k">else</span> <span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s1">&#39;STRING&#39;</span> <span class="o">and</span> <span class="nx">escape_quotes</span>
<span class="nv">escaped: </span><span class="nx">value</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nx">value</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">).</span><span class="nx">replace</span><span class="p">(</span><span class="sr">/&quot;/g</span><span class="p">,</span> <span class="s1">&#39;\\&quot;&#39;</span><span class="p">)</span>
<span class="err">@</span><span class="nx">token</span> <span class="nx">tag</span><span class="p">,</span> <span class="s2">&quot;\&quot;$escaped\&quot;&quot;</span>
<span class="nx">@token</span> <span class="nx">tag</span><span class="p">,</span> <span class="s2">&quot;\&quot;$escaped\&quot;&quot;</span>
<span class="k">else</span>
<span class="err">@</span><span class="nx">token</span> <span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span>
<span class="err">@</span><span class="nx">token</span> <span class="s1">&#39;+&#39;</span><span class="p">,</span> <span class="s1">&#39;+&#39;</span> <span class="k">if</span> <span class="nx">i</span> <span class="o">&lt;</span> <span class="nx">tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
<span class="nx">@token</span> <span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span>
<span class="nx">@token</span> <span class="s1">&#39;+&#39;</span><span class="p">,</span> <span class="s1">&#39;+&#39;</span> <span class="k">if</span> <span class="nx">i</span> <span class="o">&lt;</span> <span class="nx">tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
<span class="nx">tokens</span></pre></div> </td> </tr> <tr id="section-33"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-33">#</a> </div> <h2>Helpers</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-34"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-34">#</a> </div> <p>Add a token to the results, taking note of the line number.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">token: </span><span class="p">(</span><span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span><span class="p">,</span> <span class="err">@</span><span class="nx">line</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-35"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-35">#</a> </div> <p>Peek at a tag in the current token stream.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tag: </span><span class="p">(</span><span class="nx">index</span><span class="p">,</span> <span class="nx">tag</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="k">return</span> <span class="nx">unless</span> <span class="nv">tok: </span><span class="err">@</span><span class="nx">prev</span> <span class="nx">index</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span><span class="p">,</span> <span class="nx">@line</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-35"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-35">#</a> </div> <p>Peek at a tag in the current token stream.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tag: </span><span class="p">(</span><span class="nx">index</span><span class="p">,</span> <span class="nx">tag</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="k">return</span> <span class="nx">unless</span> <span class="nv">tok: </span><span class="nx">@prev</span> <span class="nx">index</span>
<span class="k">return</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">:</span> <span class="nx">tag</span> <span class="k">if</span> <span class="nx">tag</span><span class="o">?</span>
<span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-36"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-36">#</a> </div> <p>Peek at a value in the current token stream.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">value: </span><span class="p">(</span><span class="nx">index</span><span class="p">,</span> <span class="nx">val</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="k">return</span> <span class="nx">unless</span> <span class="nv">tok: </span><span class="err">@</span><span class="nx">prev</span> <span class="nx">index</span>
<span class="k">return</span> <span class="nx">unless</span> <span class="nv">tok: </span><span class="nx">@prev</span> <span class="nx">index</span>
<span class="k">return</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">:</span> <span class="nx">val</span> <span class="k">if</span> <span class="nx">val</span><span class="o">?</span>
<span class="nx">tok</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-37"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-37">#</a> </div> <p>Peek at a previous token, entire.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">prev: </span><span class="p">(</span><span class="nx">index</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">[</span><span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="p">(</span><span class="nx">index</span> <span class="o">or</span> <span class="mi">1</span><span class="p">)]</span></pre></div> </td> </tr> <tr id="section-38"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-38">#</a> </div> <p>Attempt to match a string against the current chunk, returning the indexed
<span class="nx">@tokens</span><span class="p">[</span><span class="nx">@tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="p">(</span><span class="nx">index</span> <span class="o">or</span> <span class="mi">1</span><span class="p">)]</span></pre></div> </td> </tr> <tr id="section-38"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-38">#</a> </div> <p>Attempt to match a string against the current chunk, returning the indexed
match if successful, and <code>false</code> otherwise.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">match: </span><span class="p">(</span><span class="nx">regex</span><span class="p">,</span> <span class="nx">index</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">m: </span><span class="err">@</span><span class="nx">chunk</span><span class="p">.</span><span class="nx">match</span> <span class="nx">regex</span>
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">m: </span><span class="nx">@chunk</span><span class="p">.</span><span class="nx">match</span> <span class="nx">regex</span>
<span class="k">if</span> <span class="nx">m</span> <span class="k">then</span> <span class="nx">m</span><span class="p">[</span><span class="nx">index</span><span class="p">]</span> <span class="k">else</span> <span class="kc">false</span></pre></div> </td> </tr> <tr id="section-39"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-39">#</a> </div> <p>Are we in the midst of an unfinished expression?</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">unfinished: </span><span class="o">-&gt;</span>
<span class="nv">prev: </span><span class="err">@</span><span class="nx">prev</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
<span class="err">@</span><span class="nx">value</span><span class="p">()</span> <span class="o">and</span> <span class="err">@</span><span class="nx">value</span><span class="p">().</span><span class="nx">match</span> <span class="o">and</span> <span class="err">@</span><span class="nx">value</span><span class="p">().</span><span class="nx">match</span><span class="p">(</span><span class="nx">NO_NEWLINE</span><span class="p">)</span> <span class="o">and</span>
<span class="nx">prev</span> <span class="o">and</span> <span class="p">(</span><span class="nx">prev</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">isnt</span> <span class="s1">&#39;.&#39;</span><span class="p">)</span> <span class="o">and</span> <span class="o">not</span> <span class="err">@</span><span class="nx">value</span><span class="p">().</span><span class="nx">match</span><span class="p">(</span><span class="nx">CODE</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-40"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-40">#</a> </div> <h2>Lexer Properties</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-41"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-41">#</a> </div> <p>There are no exensions to the core lexer by default.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="err">@</span><span class="nv">extensions: </span><span class="p">[]</span></pre></div> </td> </tr> <tr id="section-42"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-42">#</a> </div> <h2>Constants</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-43"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-43">#</a> </div> <p>Keywords that CoffeeScript shares in common with JavaScript.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">JS_KEYWORDS: </span><span class="p">[</span>
<span class="nv">prev: </span><span class="nx">@prev</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
<span class="nx">@value</span><span class="p">()</span> <span class="o">and</span> <span class="nx">@value</span><span class="p">().</span><span class="nx">match</span> <span class="o">and</span> <span class="nx">@value</span><span class="p">().</span><span class="nx">match</span><span class="p">(</span><span class="nx">NO_NEWLINE</span><span class="p">)</span> <span class="o">and</span>
<span class="nx">prev</span> <span class="o">and</span> <span class="p">(</span><span class="nx">prev</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">isnt</span> <span class="s1">&#39;.&#39;</span><span class="p">)</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">@value</span><span class="p">().</span><span class="nx">match</span><span class="p">(</span><span class="nx">CODE</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-40"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-40">#</a> </div> <h2>Lexer Properties</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-41"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-41">#</a> </div> <p>There are no exensions to the core lexer by default.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="vi">@extensions: </span><span class="p">[]</span></pre></div> </td> </tr> <tr id="section-42"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-42">#</a> </div> <h2>Constants</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-43"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-43">#</a> </div> <p>Keywords that CoffeeScript shares in common with JavaScript.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">JS_KEYWORDS: </span><span class="p">[</span>
<span class="s2">&quot;if&quot;</span><span class="p">,</span> <span class="s2">&quot;else&quot;</span><span class="p">,</span>
<span class="s2">&quot;true&quot;</span><span class="p">,</span> <span class="s2">&quot;false&quot;</span><span class="p">,</span>
<span class="s2">&quot;new&quot;</span><span class="p">,</span> <span class="s2">&quot;return&quot;</span><span class="p">,</span>
@ -345,7 +345,7 @@ be used as identifiers or properties.</p> </td> <td clas
<span class="nx">LAST_DENTS</span> <span class="o">:</span> <span class="sr">/\n([ \t]*)/g</span>
<span class="nx">LAST_DENT</span> <span class="o">:</span> <span class="sr">/\n([ \t]*)/</span>
<span class="nx">ASSIGNMENT</span> <span class="o">:</span> <span class="sr">/^(:|=)$/</span></pre></div> </td> </tr> <tr id="section-49"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-49">#</a> </div> <p>Regex-matching-regexes.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nx">REGEX_START</span> <span class="o">:</span> <span class="sr">/^\/[^\/ ]/</span>
<span class="nv">REGEX_INTERPOLATION: </span><span class="sr">/([^\\]\$[a-zA-Z_@]|[^\\]\$\{.*[^\\]\})/</span>
<span class="nv">REGEX_INTERPOLATION: </span><span class="o">/</span><span class="p">([</span><span class="o">^</span><span class="err">\\</span><span class="p">]</span><span class="err">\</span><span class="nx">$</span><span class="p">[</span><span class="nx">a</span><span class="o">-</span><span class="nx">zA</span><span class="o">-</span><span class="nx">Z_</span><span class="err">@</span><span class="p">]</span><span class="o">|</span><span class="p">[</span><span class="o">^</span><span class="err">\\</span><span class="p">]</span><span class="err">\</span><span class="nx">$</span><span class="err">\</span><span class="p">{.</span><span class="o">*</span><span class="p">[</span><span class="o">^</span><span class="err">\\</span><span class="p">]</span><span class="err">\</span><span class="p">})</span><span class="o">/</span>
<span class="nx">REGEX_FLAGS</span> <span class="o">:</span> <span class="sr">/^[imgy]{0,4}/</span>
<span class="nx">REGEX_ESCAPE</span> <span class="o">:</span> <span class="sr">/\\[^\$]/g</span></pre></div> </td> </tr> <tr id="section-50"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-50">#</a> </div> <p>Token cleaning regexes.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nx">JS_CLEANER</span> <span class="o">:</span> <span class="sr">/(^`|`$)/g</span>
<span class="nx">MULTILINER</span> <span class="o">:</span> <span class="sr">/\n/g</span>
@ -361,7 +361,7 @@ a division operator might.</p>
<span class="s1">&#39;NUMBER&#39;</span><span class="p">,</span> <span class="s1">&#39;REGEX&#39;</span><span class="p">,</span> <span class="s1">&#39;++&#39;</span><span class="p">,</span> <span class="s1">&#39;--&#39;</span><span class="p">,</span> <span class="s1">&#39;FALSE&#39;</span><span class="p">,</span> <span class="s1">&#39;NULL&#39;</span><span class="p">,</span> <span class="s1">&#39;TRUE&#39;</span>
<span class="p">]</span></pre></div> </td> </tr> <tr id="section-52"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-52">#</a> </div> <p>Tokens which could legitimately be invoked or indexed. A opening
parentheses or bracket following these tokens will be recorded as the start
of a function invocation or indexing operation.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">CALLABLE: </span><span class="p">[</span><span class="s1">&#39;IDENTIFIER&#39;</span><span class="p">,</span> <span class="s1">&#39;SUPER&#39;</span><span class="p">,</span> <span class="s1">&#39;)&#39;</span><span class="p">,</span> <span class="s1">&#39;]&#39;</span><span class="p">,</span> <span class="s1">&#39;}&#39;</span><span class="p">,</span> <span class="s1">&#39;STRING&#39;</span><span class="p">,</span> <span class="s1">&#39;@&#39;</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-53"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-53">#</a> </div> <p>Tokens that indicate an access -- keywords immediately following will be
of a function invocation or indexing operation.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">CALLABLE: </span><span class="p">[</span><span class="s1">&#39;IDENTIFIER&#39;</span><span class="p">,</span> <span class="s1">&#39;SUPER&#39;</span><span class="p">,</span> <span class="s1">&#39;)&#39;</span><span class="p">,</span> <span class="s1">&#39;]&#39;</span><span class="p">,</span> <span class="s1">&#39;}&#39;</span><span class="p">,</span> <span class="s1">&#39;STRING&#39;</span><span class="p">,</span> <span class="s1">&#39;@&#39;</span><span class="p">,</span> <span class="s1">&#39;THIS&#39;</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-53"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-53">#</a> </div> <p>Tokens that indicate an access -- keywords immediately following will be
treated as identifiers.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">ACCESSORS: </span><span class="p">[</span><span class="s1">&#39;PROPERTY_ACCESS&#39;</span><span class="p">,</span> <span class="s1">&#39;PROTOTYPE_ACCESS&#39;</span><span class="p">,</span> <span class="s1">&#39;SOAK_ACCESS&#39;</span><span class="p">,</span> <span class="s1">&#39;@&#39;</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-54"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-54">#</a> </div> <p>Tokens that, when immediately preceding a <code>WHEN</code>, indicate that the <code>WHEN</code>
occurs at the start of a line. We disambiguate these from trailing whens to
avoid an ambiguity in the grammar.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">LINE_BREAK: </span><span class="p">[</span><span class="s1">&#39;INDENT&#39;</span><span class="p">,</span> <span class="s1">&#39;OUTDENT&#39;</span><span class="p">,</span> <span class="s1">&#39;TERMINATOR&#39;</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-55"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-55">#</a> </div> <p>Half-assignments...</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">HALF_ASSIGNMENTS: </span><span class="p">[</span><span class="s1">&#39;-&#39;</span><span class="p">,</span> <span class="s1">&#39;+&#39;</span><span class="p">,</span> <span class="s1">&#39;/&#39;</span><span class="p">,</span> <span class="s1">&#39;*&#39;</span><span class="p">,</span> <span class="s1">&#39;%&#39;</span><span class="p">,</span> <span class="s1">&#39;||&#39;</span><span class="p">,</span> <span class="s1">&#39;&amp;&amp;&#39;</span><span class="p">,</span> <span class="s1">&#39;?&#39;</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-56"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-56">#</a> </div> <p>Conversions from CoffeeScript operators into JavaScript ones.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">CONVERSIONS: </span><span class="p">{</span>

File diff suppressed because it is too large Load diff

View file

@ -9,8 +9,8 @@ options: parser.parse process.argv
</code></pre>
<p>Along with an an optional banner for the usage help.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">constructor: </span><span class="p">(</span><span class="nx">rules</span><span class="p">,</span> <span class="nx">banner</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="err">@</span><span class="nv">banner: </span> <span class="nx">banner</span>
<span class="err">@</span><span class="nv">rules: </span> <span class="nx">build_rules</span><span class="p">(</span><span class="nx">rules</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-3"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-3">#</a> </div> <p>Parse the list of arguments, populating an <code>options</code> object with all of the
<span class="vi">@banner: </span> <span class="nx">banner</span>
<span class="vi">@rules: </span> <span class="nx">build_rules</span><span class="p">(</span><span class="nx">rules</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-3"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-3">#</a> </div> <p>Parse the list of arguments, populating an <code>options</code> object with all of the
specified options, and returning it. <code>options.arguments</code> will be an array
containing the remaning non-option arguments. This is a simpler API than
many option parsers that allow you to attach callback actions for every
@ -20,7 +20,7 @@ flag. Instead, you're responsible for interpreting the options object.</p>
<span class="k">while</span> <span class="p">(</span><span class="nv">arg: </span><span class="nx">args</span><span class="p">.</span><span class="nx">shift</span><span class="p">())</span>
<span class="nv">is_option: </span><span class="o">!!</span><span class="p">(</span><span class="nx">arg</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">LONG_FLAG</span><span class="p">)</span> <span class="o">or</span> <span class="nx">arg</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">SHORT_FLAG</span><span class="p">))</span>
<span class="nv">matched_rule: </span><span class="kc">no</span>
<span class="k">for</span> <span class="nx">rule</span> <span class="k">in</span> <span class="err">@</span><span class="nx">rules</span>
<span class="k">for</span> <span class="nx">rule</span> <span class="k">in</span> <span class="nx">@rules</span>
<span class="k">if</span> <span class="nx">rule</span><span class="p">.</span><span class="nx">short_flag</span> <span class="o">is</span> <span class="nx">arg</span> <span class="o">or</span> <span class="nx">rule</span><span class="p">.</span><span class="nx">long_flag</span> <span class="o">is</span> <span class="nx">arg</span>
<span class="nx">options</span><span class="p">[</span><span class="nx">rule</span><span class="p">.</span><span class="nx">name</span><span class="p">]</span><span class="o">:</span> <span class="k">if</span> <span class="nx">rule</span><span class="p">.</span><span class="nx">has_argument</span> <span class="k">then</span> <span class="nx">args</span><span class="p">.</span><span class="nx">shift</span><span class="p">()</span> <span class="k">else</span> <span class="kc">true</span>
<span class="nv">matched_rule: </span><span class="kc">yes</span>
@ -30,16 +30,16 @@ flag. Instead, you're responsible for interpreting the options object.</p>
<span class="nx">options</span></pre></div> </td> </tr> <tr id="section-4"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-4">#</a> </div> <p>Return the help text for this <strong>OptionParser</strong>, listing and describing all
of the valid options, for <code>--help</code> and such.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">help: </span><span class="o">-&gt;</span>
<span class="nv">lines: </span><span class="p">[</span><span class="s1">&#39;Available options:&#39;</span><span class="p">]</span>
<span class="nx">lines</span><span class="p">.</span><span class="nx">unshift</span> <span class="s2">&quot;$@banner\n&quot;</span> <span class="k">if</span> <span class="err">@</span><span class="nx">banner</span>
<span class="k">for</span> <span class="nx">rule</span> <span class="k">in</span> <span class="err">@</span><span class="nx">rules</span>
<span class="nx">lines</span><span class="p">.</span><span class="nx">unshift</span> <span class="s2">&quot;$@banner\n&quot;</span> <span class="k">if</span> <span class="nx">@banner</span>
<span class="k">for</span> <span class="nx">rule</span> <span class="k">in</span> <span class="nx">@rules</span>
<span class="nv">spaces: </span> <span class="mi">15</span> <span class="o">-</span> <span class="nx">rule</span><span class="p">.</span><span class="nx">long_flag</span><span class="p">.</span><span class="nx">length</span>
<span class="nv">spaces: </span> <span class="k">if</span> <span class="nx">spaces</span> <span class="o">&gt;</span> <span class="mi">0</span> <span class="k">then</span> <span class="p">(</span><span class="s1">&#39; &#39;</span> <span class="k">for</span> <span class="nx">i</span> <span class="k">in</span> <span class="p">[</span><span class="mi">0</span><span class="p">..</span><span class="nx">spaces</span><span class="p">]).</span><span class="nx">join</span><span class="p">(</span><span class="s1">&#39;&#39;</span><span class="p">)</span> <span class="k">else</span> <span class="s1">&#39;&#39;</span>
<span class="nv">let_part: </span><span class="k">if</span> <span class="nx">rule</span><span class="p">.</span><span class="nx">short_flag</span> <span class="k">then</span> <span class="nx">rule</span><span class="p">.</span><span class="nx">short_flag</span> <span class="o">+</span> <span class="s1">&#39;, &#39;</span> <span class="k">else</span> <span class="s1">&#39; &#39;</span>
<span class="nx">lines</span><span class="p">.</span><span class="nx">push</span> <span class="s2">&quot; $let_part${rule.long_flag}$spaces${rule.description}&quot;</span>
<span class="s2">&quot;\n${ lines.join(&#39;\n&#39;) }\n&quot;</span></pre></div> </td> </tr> <tr id="section-5"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-5">#</a> </div> <h2>Helpers</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-6"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-6">#</a> </div> <p>Regex matchers for option flags.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">LONG_FLAG: </span> <span class="sr">/^(--\w[\w\-]+)/</span>
<span class="nv">SHORT_FLAG: </span><span class="sr">/^(-\w)/</span>
<span class="nv">MULTI_FLAG: </span><span class="sr">/^-(\w{2,})/</span>
<span class="nv">OPTIONAL: </span> <span class="sr">/\[(.+)\]/</span></pre></div> </td> </tr> <tr id="section-7"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-7">#</a> </div> <p>Build and return the list of option rules. If the optional <em>short-flag</em> is
<span class="s2">&quot;\n${ lines.join(&#39;\n&#39;) }\n&quot;</span></pre></div> </td> </tr> <tr id="section-5"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-5">#</a> </div> <h2>Helpers</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-6"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-6">#</a> </div> <p>Regex matchers for option flags.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">LONG_FLAG: </span> <span class="o">/^</span><span class="p">(</span><span class="o">--</span><span class="err">\</span><span class="nx">w</span><span class="p">[</span><span class="err">\</span><span class="nx">w</span><span class="err">\</span><span class="o">-</span><span class="p">]</span><span class="o">+</span><span class="p">)</span><span class="o">/</span>
<span class="nv">SHORT_FLAG: </span><span class="o">/^</span><span class="p">(</span><span class="o">-</span><span class="err">\</span><span class="nx">w</span><span class="p">)</span><span class="o">/</span>
<span class="nv">MULTI_FLAG: </span><span class="o">/^-</span><span class="p">(</span><span class="err">\</span><span class="nx">w</span><span class="p">{</span><span class="mi">2</span><span class="p">,})</span><span class="o">/</span>
<span class="nv">OPTIONAL: </span> <span class="o">/</span><span class="err">\</span><span class="p">[(.</span><span class="o">+</span><span class="p">)</span><span class="err">\</span><span class="p">]</span><span class="o">/</span></pre></div> </td> </tr> <tr id="section-7"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-7">#</a> </div> <p>Build and return the list of option rules. If the optional <em>short-flag</em> is
unspecified, leave it out by padding with <code>null</code>.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">build_rules: </span><span class="p">(</span><span class="nx">rules</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="k">for</span> <span class="nx">tuple</span> <span class="k">in</span> <span class="nx">rules</span>
<span class="nx">tuple</span><span class="p">.</span><span class="nx">unshift</span> <span class="kc">null</span> <span class="k">if</span> <span class="nx">tuple</span><span class="p">.</span><span class="nx">length</span> <span class="o">&lt;</span> <span class="mi">3</span>

View file

@ -13,51 +13,51 @@ a time. This could certainly be changed into a single pass through the
stream, with a big ol' efficient switch, but it's much nicer to work with
like this. The order of these passes matters -- indentation must be
corrected before implicit parentheses can be wrapped around blocks of code.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">rewrite: </span><span class="p">(</span><span class="nx">tokens</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="err">@</span><span class="nv">tokens: </span><span class="nx">tokens</span>
<span class="err">@</span><span class="nx">adjust_comments</span><span class="p">()</span>
<span class="err">@</span><span class="nx">remove_leading_newlines</span><span class="p">()</span>
<span class="err">@</span><span class="nx">remove_mid_expression_newlines</span><span class="p">()</span>
<span class="err">@</span><span class="nx">close_open_calls_and_indexes</span><span class="p">()</span>
<span class="err">@</span><span class="nx">add_implicit_indentation</span><span class="p">()</span>
<span class="err">@</span><span class="nx">add_implicit_parentheses</span><span class="p">()</span>
<span class="err">@</span><span class="nx">ensure_balance</span> <span class="nx">BALANCED_PAIRS</span>
<span class="err">@</span><span class="nx">rewrite_closing_parens</span><span class="p">()</span>
<span class="err">@</span><span class="nx">tokens</span></pre></div> </td> </tr> <tr id="section-6"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-6">#</a> </div> <p>Rewrite the token stream, looking one token ahead and behind.
<span class="vi">@tokens: </span><span class="nx">tokens</span>
<span class="nx">@adjust_comments</span><span class="p">()</span>
<span class="nx">@remove_leading_newlines</span><span class="p">()</span>
<span class="nx">@remove_mid_expression_newlines</span><span class="p">()</span>
<span class="nx">@close_open_calls_and_indexes</span><span class="p">()</span>
<span class="nx">@add_implicit_indentation</span><span class="p">()</span>
<span class="nx">@add_implicit_parentheses</span><span class="p">()</span>
<span class="nx">@ensure_balance</span> <span class="nx">BALANCED_PAIRS</span>
<span class="nx">@rewrite_closing_parens</span><span class="p">()</span>
<span class="nx">@tokens</span></pre></div> </td> </tr> <tr id="section-6"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-6">#</a> </div> <p>Rewrite the token stream, looking one token ahead and behind.
Allow the return value of the block to tell us how many tokens to move
forwards (or backwards) in the stream, to make sure we don't miss anything
as tokens are inserted and removed, and the stream changes length under
our feet.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">scan_tokens: </span><span class="p">(</span><span class="nx">block</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="nv">i: </span><span class="mi">0</span>
<span class="k">while</span> <span class="kc">true</span>
<span class="k">break</span> <span class="nx">unless</span> <span class="err">@</span><span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">]</span>
<span class="nv">move: </span><span class="nx">block</span> <span class="err">@</span><span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">],</span> <span class="err">@</span><span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">],</span> <span class="err">@</span><span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">],</span> <span class="nx">i</span>
<span class="k">break</span> <span class="nx">unless</span> <span class="nx">@tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">]</span>
<span class="nv">move: </span><span class="nx">block</span> <span class="nx">@tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">],</span> <span class="nx">@tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">],</span> <span class="nx">@tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">],</span> <span class="nx">i</span>
<span class="nv">i: </span><span class="o">+</span> <span class="nx">move</span>
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-7"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-7">#</a> </div> <p>Massage newlines and indentations so that comments don't have to be
correctly indented, or appear on a line of their own.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">adjust_comments: </span><span class="o">-&gt;</span>
<span class="err">@</span><span class="nx">scan_tokens</span> <span class="p">(</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">post</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">=&gt;</span>
<span class="nx">@scan_tokens</span> <span class="p">(</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">post</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">=&gt;</span>
<span class="k">return</span> <span class="mi">1</span> <span class="nx">unless</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">&#39;COMMENT&#39;</span>
<span class="nv">after: </span> <span class="err">@</span><span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">2</span><span class="p">]</span>
<span class="nv">after: </span> <span class="nx">@tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">2</span><span class="p">]</span>
<span class="k">if</span> <span class="nx">after</span> <span class="o">and</span> <span class="nx">after</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">&#39;INDENT&#39;</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">after</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">after</span>
<span class="k">return</span> <span class="mi">1</span>
<span class="k">else</span> <span class="k">if</span> <span class="nx">prev</span> <span class="o">and</span> <span class="nx">prev</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">isnt</span> <span class="s1">&#39;TERMINATOR&#39;</span> <span class="o">and</span> <span class="nx">prev</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">isnt</span> <span class="s1">&#39;INDENT&#39;</span> <span class="o">and</span> <span class="nx">prev</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">isnt</span> <span class="s1">&#39;OUTDENT&#39;</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="p">[</span><span class="s1">&#39;TERMINATOR&#39;</span><span class="p">,</span> <span class="s2">&quot;\n&quot;</span><span class="p">,</span> <span class="nx">prev</span><span class="p">[</span><span class="mi">2</span><span class="p">]]</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="p">[</span><span class="s1">&#39;TERMINATOR&#39;</span><span class="p">,</span> <span class="s2">&quot;\n&quot;</span><span class="p">,</span> <span class="nx">prev</span><span class="p">[</span><span class="mi">2</span><span class="p">]]</span>
<span class="k">return</span> <span class="mi">2</span>
<span class="k">else</span>
<span class="k">return</span> <span class="mi">1</span></pre></div> </td> </tr> <tr id="section-8"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-8">#</a> </div> <p>Leading newlines would introduce an ambiguity in the grammar, so we
dispatch them here.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">remove_leading_newlines: </span><span class="o">-&gt;</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">shift</span><span class="p">()</span> <span class="k">while</span> <span class="err">@</span><span class="nx">tokens</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">and</span> <span class="err">@</span><span class="nx">tokens</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">&#39;TERMINATOR&#39;</span></pre></div> </td> </tr> <tr id="section-9"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-9">#</a> </div> <p>Some blocks occur in the middle of expressions -- when we're expecting
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">shift</span><span class="p">()</span> <span class="k">while</span> <span class="nx">@tokens</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">and</span> <span class="nx">@tokens</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">&#39;TERMINATOR&#39;</span></pre></div> </td> </tr> <tr id="section-9"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-9">#</a> </div> <p>Some blocks occur in the middle of expressions -- when we're expecting
this, remove their trailing newlines.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">remove_mid_expression_newlines: </span><span class="o">-&gt;</span>
<span class="err">@</span><span class="nx">scan_tokens</span> <span class="p">(</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">post</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">=&gt;</span>
<span class="nx">@scan_tokens</span> <span class="p">(</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">post</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">=&gt;</span>
<span class="k">return</span> <span class="mi">1</span> <span class="nx">unless</span> <span class="nx">post</span> <span class="o">and</span> <span class="nx">include</span><span class="p">(</span><span class="nx">EXPRESSION_CLOSE</span><span class="p">,</span> <span class="nx">post</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">and</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">&#39;TERMINATOR&#39;</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">1</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">1</span>
<span class="k">return</span> <span class="mi">0</span></pre></div> </td> </tr> <tr id="section-10"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-10">#</a> </div> <p>The lexer has tagged the opening parenthesis of a method call, and the
opening bracket of an indexing operation. Match them with their paired
close.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">close_open_calls_and_indexes: </span><span class="o">-&gt;</span>
<span class="nv">parens: </span> <span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="nv">brackets: </span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="err">@</span><span class="nx">scan_tokens</span> <span class="p">(</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">post</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">=&gt;</span>
<span class="nx">@scan_tokens</span> <span class="p">(</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">post</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">=&gt;</span>
<span class="k">switch</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">when</span> <span class="s1">&#39;CALL_START&#39;</span> <span class="k">then</span> <span class="nx">parens</span><span class="p">.</span><span class="nx">push</span> <span class="mi">0</span>
<span class="k">when</span> <span class="s1">&#39;INDEX_START&#39;</span> <span class="k">then</span> <span class="nx">brackets</span><span class="p">.</span><span class="nx">push</span> <span class="mi">0</span>
@ -81,7 +81,8 @@ deal with them.</p> </td> <td class="code">
<span class="nv">stack: </span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="nv">calls: </span> <span class="mi">0</span>
<span class="nv">parens: </span><span class="mi">0</span>
<span class="err">@</span><span class="nx">scan_tokens</span> <span class="p">(</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">post</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">=&gt;</span>
<span class="nv">start_parens: </span><span class="mi">0</span>
<span class="nx">@scan_tokens</span> <span class="p">(</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">post</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">=&gt;</span>
<span class="nv">tag: </span><span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">switch</span> <span class="nx">tag</span>
<span class="k">when</span> <span class="s1">&#39;CALL_START&#39;</span> <span class="k">then</span> <span class="nv">calls: </span><span class="o">+</span> <span class="mi">1</span>
@ -93,37 +94,38 @@ deal with them.</p> </td> <td class="code">
<span class="nv">last: </span><span class="nx">stack</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
<span class="nx">stack</span><span class="p">[</span><span class="nx">stack</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span><span class="o">:</span> <span class="o">+</span> <span class="nx">last</span>
<span class="nv">open: </span><span class="nx">stack</span><span class="p">[</span><span class="nx">stack</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span> <span class="o">&gt;</span> <span class="mi">0</span>
<span class="k">if</span> <span class="o">!</span><span class="nx">post</span><span class="o">?</span> <span class="o">or</span> <span class="p">(</span><span class="nx">parens</span> <span class="o">is</span> <span class="mi">0</span> <span class="o">and</span> <span class="nx">include</span> <span class="nx">IMPLICIT_END</span><span class="p">,</span> <span class="nx">tag</span><span class="p">)</span>
<span class="k">if</span> <span class="o">!</span><span class="nx">post</span><span class="o">?</span> <span class="o">or</span> <span class="p">(</span><span class="nx">start_parens</span> <span class="o">&gt;</span> <span class="nx">parens</span><span class="p">)</span> <span class="o">or</span> <span class="p">(</span><span class="nx">parens</span> <span class="o">is</span> <span class="mi">0</span> <span class="o">and</span> <span class="nx">include</span> <span class="nx">IMPLICIT_END</span><span class="p">,</span> <span class="nx">tag</span><span class="p">)</span>
<span class="k">return</span> <span class="mi">1</span> <span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s1">&#39;INDENT&#39;</span> <span class="o">and</span> <span class="nx">prev</span> <span class="o">and</span> <span class="nx">include</span> <span class="nx">IMPLICIT_BLOCK</span><span class="p">,</span> <span class="nx">prev</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">return</span> <span class="mi">1</span> <span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s1">&#39;OUTDENT&#39;</span> <span class="o">and</span> <span class="nx">token</span><span class="p">.</span><span class="nx">generated</span>
<span class="k">if</span> <span class="nx">open</span> <span class="o">or</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s1">&#39;INDENT&#39;</span>
<span class="nv">idx: </span><span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s1">&#39;OUTDENT&#39;</span> <span class="k">then</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span> <span class="k">else</span> <span class="nx">i</span>
<span class="nv">stack_pointer: </span><span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s1">&#39;INDENT&#39;</span> <span class="k">then</span> <span class="mi">2</span> <span class="k">else</span> <span class="mi">1</span>
<span class="k">for</span> <span class="nx">tmp</span> <span class="k">in</span> <span class="p">[</span><span class="mi">0</span><span class="p">...</span><span class="nx">stack</span><span class="p">[</span><span class="nx">stack</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="nx">stack_pointer</span><span class="p">]]</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span><span class="p">(</span><span class="nx">idx</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="p">[</span><span class="s1">&#39;CALL_END&#39;</span><span class="p">,</span> <span class="s1">&#39;)&#39;</span><span class="p">,</span> <span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]])</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span><span class="p">(</span><span class="nx">idx</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="p">[</span><span class="s1">&#39;CALL_END&#39;</span><span class="p">,</span> <span class="s1">&#39;)&#39;</span><span class="p">,</span> <span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]])</span>
<span class="nv">size: </span><span class="nx">stack</span><span class="p">[</span><span class="nx">stack</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="nx">stack_pointer</span><span class="p">]</span> <span class="o">+</span> <span class="mi">1</span>
<span class="nx">stack</span><span class="p">[</span><span class="nx">stack</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="nx">stack_pointer</span><span class="p">]</span><span class="o">:</span> <span class="mi">0</span>
<span class="k">return</span> <span class="nx">size</span>
<span class="k">return</span> <span class="mi">1</span> <span class="nx">unless</span> <span class="nx">prev</span> <span class="o">and</span> <span class="nx">include</span><span class="p">(</span><span class="nx">IMPLICIT_FUNC</span><span class="p">,</span> <span class="nx">prev</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">and</span> <span class="nx">include</span><span class="p">(</span><span class="nx">IMPLICIT_CALL</span><span class="p">,</span> <span class="nx">tag</span><span class="p">)</span>
<span class="nv">calls: </span><span class="mi">0</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="p">[</span><span class="s1">&#39;CALL_START&#39;</span><span class="p">,</span> <span class="s1">&#39;(&#39;</span><span class="p">,</span> <span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]]</span>
<span class="nv">start_parens: </span><span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s1">&#39;(&#39;</span> <span class="k">then</span> <span class="nx">parens</span> <span class="o">-</span> <span class="mi">1</span> <span class="k">else</span> <span class="nx">parens</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="p">[</span><span class="s1">&#39;CALL_START&#39;</span><span class="p">,</span> <span class="s1">&#39;(&#39;</span><span class="p">,</span> <span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]]</span>
<span class="nx">stack</span><span class="p">[</span><span class="nx">stack</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span><span class="o">:</span> <span class="o">+</span> <span class="mi">1</span>
<span class="k">return</span> <span class="mi">2</span></pre></div> </td> </tr> <tr id="section-12"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-12">#</a> </div> <p>Because our grammar is LALR(1), it can't handle some single-line
expressions that lack ending delimiters. The <strong>Rewriter</strong> adds the implicit
blocks, so it doesn't need to. ')' can close a single-line block,
but we need to make sure it's balanced.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">add_implicit_indentation: </span><span class="o">-&gt;</span>
<span class="err">@</span><span class="nx">scan_tokens</span> <span class="p">(</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">post</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">=&gt;</span>
<span class="nx">@scan_tokens</span> <span class="p">(</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">post</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">=&gt;</span>
<span class="k">return</span> <span class="mi">1</span> <span class="nx">unless</span> <span class="nx">include</span><span class="p">(</span><span class="nx">SINGLE_LINERS</span><span class="p">,</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">and</span>
<span class="nx">post</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">isnt</span> <span class="s1">&#39;INDENT&#39;</span> <span class="o">and</span>
<span class="o">not</span> <span class="p">(</span><span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">&#39;ELSE&#39;</span> <span class="o">and</span> <span class="nx">post</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">&#39;IF&#39;</span><span class="p">)</span>
<span class="nv">starter: </span><span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="p">[</span><span class="s1">&#39;INDENT&#39;</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]]</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="p">[</span><span class="s1">&#39;INDENT&#39;</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]]</span>
<span class="nv">idx: </span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span>
<span class="nv">parens: </span><span class="mi">0</span>
<span class="k">while</span> <span class="kc">true</span>
<span class="nv">idx: </span><span class="o">+</span> <span class="mi">1</span>
<span class="nv">tok: </span><span class="err">@</span><span class="nx">tokens</span><span class="p">[</span><span class="nx">idx</span><span class="p">]</span>
<span class="nv">pre: </span><span class="err">@</span><span class="nx">tokens</span><span class="p">[</span><span class="nx">idx</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span>
<span class="nv">tok: </span><span class="nx">@tokens</span><span class="p">[</span><span class="nx">idx</span><span class="p">]</span>
<span class="nv">pre: </span><span class="nx">@tokens</span><span class="p">[</span><span class="nx">idx</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span>
<span class="k">if</span> <span class="p">(</span><span class="o">not</span> <span class="nx">tok</span> <span class="o">or</span>
<span class="p">(</span><span class="nx">include</span><span class="p">(</span><span class="nx">SINGLE_CLOSERS</span><span class="p">,</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">and</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">isnt</span> <span class="s1">&#39;;&#39;</span><span class="p">)</span> <span class="o">or</span>
<span class="p">(</span><span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">&#39;)&#39;</span> <span class="o">and</span> <span class="nx">parens</span> <span class="o">is</span> <span class="mi">0</span><span class="p">))</span> <span class="o">and</span>
@ -131,17 +133,17 @@ but we need to make sure it's balanced.</p> </td> <td cl
<span class="nv">insertion: </span><span class="k">if</span> <span class="nx">pre</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s2">&quot;,&quot;</span> <span class="k">then</span> <span class="nx">idx</span> <span class="o">-</span> <span class="mi">1</span> <span class="k">else</span> <span class="nx">idx</span>
<span class="nv">outdent: </span><span class="p">[</span><span class="s1">&#39;OUTDENT&#39;</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]]</span>
<span class="nv">outdent.generated: </span><span class="kc">true</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">insertion</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">outdent</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">insertion</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">outdent</span>
<span class="k">break</span>
<span class="nv">parens: </span><span class="o">+</span> <span class="mi">1</span> <span class="k">if</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">&#39;(&#39;</span>
<span class="nv">parens: </span><span class="o">-</span> <span class="mi">1</span> <span class="k">if</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">&#39;)&#39;</span>
<span class="k">return</span> <span class="mi">1</span> <span class="nx">unless</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">&#39;THEN&#39;</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">1</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">1</span>
<span class="k">return</span> <span class="mi">0</span></pre></div> </td> </tr> <tr id="section-13"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-13">#</a> </div> <p>Ensure that all listed pairs of tokens are correctly balanced throughout
the course of the token stream.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">ensure_balance: </span><span class="p">(</span><span class="nx">pairs</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="nv">levels: </span><span class="p">{}</span>
<span class="nv">open_line: </span><span class="p">{}</span>
<span class="err">@</span><span class="nx">scan_tokens</span> <span class="p">(</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">post</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">=&gt;</span>
<span class="nx">@scan_tokens</span> <span class="p">(</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">post</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">=&gt;</span>
<span class="k">for</span> <span class="nx">pair</span> <span class="k">in</span> <span class="nx">pairs</span>
<span class="p">[</span><span class="nx">open</span><span class="p">,</span> <span class="nx">close</span><span class="p">]</span><span class="o">:</span> <span class="nx">pair</span>
<span class="nx">levels</span><span class="p">[</span><span class="nx">open</span><span class="p">]</span><span class="o">:</span> <span class="o">or</span> <span class="mi">0</span>
@ -175,7 +177,7 @@ up balanced in the end.</li>
<span class="nv">stack: </span><span class="p">[]</span>
<span class="nv">debt: </span> <span class="p">{}</span>
<span class="p">(</span><span class="nx">debt</span><span class="p">[</span><span class="nx">key</span><span class="p">]</span><span class="o">:</span> <span class="mi">0</span><span class="p">)</span> <span class="k">for</span> <span class="nx">key</span><span class="p">,</span> <span class="nx">val</span> <span class="k">of</span> <span class="nx">INVERSES</span>
<span class="err">@</span><span class="nx">scan_tokens</span> <span class="p">(</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">post</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">=&gt;</span>
<span class="nx">@scan_tokens</span> <span class="p">(</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">post</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">=&gt;</span>
<span class="nv">tag: </span><span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="nv">inv: </span><span class="nx">INVERSES</span><span class="p">[</span><span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]]</span>
<span class="k">if</span> <span class="nx">include</span> <span class="nx">EXPRESSION_START</span><span class="p">,</span> <span class="nx">tag</span>
@ -184,7 +186,7 @@ up balanced in the end.</li>
<span class="k">else</span> <span class="k">if</span> <span class="nx">include</span> <span class="nx">EXPRESSION_END</span><span class="p">,</span> <span class="nx">tag</span>
<span class="k">if</span> <span class="nx">debt</span><span class="p">[</span><span class="nx">inv</span><span class="p">]</span> <span class="o">&gt;</span> <span class="mi">0</span>
<span class="nx">debt</span><span class="p">[</span><span class="nx">inv</span><span class="p">]</span><span class="o">:</span> <span class="o">-</span> <span class="mi">1</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">1</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">1</span>
<span class="k">return</span> <span class="mi">0</span>
<span class="k">else</span>
<span class="nv">match: </span><span class="nx">stack</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
@ -192,7 +194,7 @@ up balanced in the end.</li>
<span class="k">return</span> <span class="mi">1</span> <span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="nx">INVERSES</span><span class="p">[</span><span class="nx">mtag</span><span class="p">]</span>
<span class="nx">debt</span><span class="p">[</span><span class="nx">mtag</span><span class="p">]</span><span class="o">:</span> <span class="o">+</span> <span class="mi">1</span>
<span class="nv">val: </span><span class="k">if</span> <span class="nx">mtag</span> <span class="o">is</span> <span class="s1">&#39;INDENT&#39;</span> <span class="k">then</span> <span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="k">else</span> <span class="nx">INVERSES</span><span class="p">[</span><span class="nx">mtag</span><span class="p">]</span>
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="p">[</span><span class="nx">INVERSES</span><span class="p">[</span><span class="nx">mtag</span><span class="p">],</span> <span class="nx">val</span><span class="p">]</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="p">[</span><span class="nx">INVERSES</span><span class="p">[</span><span class="nx">mtag</span><span class="p">],</span> <span class="nx">val</span><span class="p">]</span>
<span class="k">return</span> <span class="mi">1</span>
<span class="k">else</span>
<span class="k">return</span> <span class="mi">1</span></pre></div> </td> </tr> <tr id="section-15"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-15">#</a> </div> <h2>Constants</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-16"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-16">#</a> </div> <p>List of the token pairs that must be balanced.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">BALANCED_PAIRS: </span><span class="p">[[</span><span class="s1">&#39;(&#39;</span><span class="p">,</span> <span class="s1">&#39;)&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;[&#39;</span><span class="p">,</span> <span class="s1">&#39;]&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;{&#39;</span><span class="p">,</span> <span class="s1">&#39;}&#39;</span><span class="p">],</span> <span class="p">[</span><span class="s1">&#39;INDENT&#39;</span><span class="p">,</span> <span class="s1">&#39;OUTDENT&#39;</span><span class="p">],</span>

View file

@ -5,44 +5,44 @@ and has a reference to its parent enclosing scope. In this way, we know which
variables are new and need to be declared with <code>var</code>, and which are shared
with the outside.</p> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-2"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-2">#</a> </div> <p>Set up exported variables for both <strong>Node.js</strong> and the browser.</p> </td> <td class="code"> <div class="highlight"><pre><span class="k">this</span><span class="p">.</span><span class="nv">exports: </span><span class="k">this</span> <span class="nx">unless</span> <span class="nx">process</span><span class="o">?</span>
<span class="nv">exports.Scope: </span><span class="nx">class</span> <span class="nx">Scope</span></pre></div> </td> </tr> <tr id="section-3"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-3">#</a> </div> <p>The top-level <strong>Scope</strong> object.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="err">@</span><span class="nv">root: </span><span class="kc">null</span></pre></div> </td> </tr> <tr id="section-4"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-4">#</a> </div> <p>Initialize a scope with its parent, for lookups up the chain,
<span class="nv">exports.Scope: </span><span class="nx">class</span> <span class="nx">Scope</span></pre></div> </td> </tr> <tr id="section-3"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-3">#</a> </div> <p>The top-level <strong>Scope</strong> object.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="vi">@root: </span><span class="kc">null</span></pre></div> </td> </tr> <tr id="section-4"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-4">#</a> </div> <p>Initialize a scope with its parent, for lookups up the chain,
as well as a reference to the <strong>Expressions</strong> node is belongs to, which is
where it should declare its variables, and a reference to the function that
it wraps.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">constructor: </span><span class="p">(</span><span class="nx">parent</span><span class="p">,</span> <span class="nx">expressions</span><span class="p">,</span> <span class="nx">method</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="p">[</span><span class="err">@</span><span class="nx">parent</span><span class="p">,</span> <span class="err">@</span><span class="nx">expressions</span><span class="p">,</span> <span class="err">@</span><span class="nx">method</span><span class="p">]</span><span class="o">:</span> <span class="p">[</span><span class="nx">parent</span><span class="p">,</span> <span class="nx">expressions</span><span class="p">,</span> <span class="nx">method</span><span class="p">]</span>
<span class="err">@</span><span class="nv">variables: </span><span class="p">{}</span>
<span class="k">if</span> <span class="err">@</span><span class="nx">parent</span>
<span class="err">@</span><span class="nv">temp_var: </span><span class="err">@</span><span class="nx">parent</span><span class="p">.</span><span class="nx">temp_var</span>
<span class="p">[</span><span class="nx">@parent</span><span class="p">,</span> <span class="nx">@expressions</span><span class="p">,</span> <span class="nx">@method</span><span class="p">]</span><span class="o">:</span> <span class="p">[</span><span class="nx">parent</span><span class="p">,</span> <span class="nx">expressions</span><span class="p">,</span> <span class="nx">method</span><span class="p">]</span>
<span class="vi">@variables: </span><span class="p">{}</span>
<span class="k">if</span> <span class="nx">@parent</span>
<span class="vi">@temp_var: </span><span class="nx">@parent</span><span class="p">.</span><span class="nx">temp_var</span>
<span class="k">else</span>
<span class="nv">Scope.root: </span><span class="k">this</span>
<span class="err">@</span><span class="nv">temp_var: </span><span class="s1">&#39;_a&#39;</span></pre></div> </td> </tr> <tr id="section-5"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-5">#</a> </div> <p>Look up a variable name in lexical scope, and declare it if it does not
<span class="vi">@temp_var: </span><span class="s1">&#39;_a&#39;</span></pre></div> </td> </tr> <tr id="section-5"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-5">#</a> </div> <p>Look up a variable name in lexical scope, and declare it if it does not
already exist.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">find: </span><span class="p">(</span><span class="nx">name</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="k">return</span> <span class="kc">true</span> <span class="k">if</span> <span class="err">@</span><span class="nx">check</span> <span class="nx">name</span>
<span class="err">@</span><span class="nx">variables</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span><span class="o">:</span> <span class="s1">&#39;var&#39;</span>
<span class="k">return</span> <span class="kc">true</span> <span class="k">if</span> <span class="nx">@check</span> <span class="nx">name</span>
<span class="nx">@variables</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span><span class="o">:</span> <span class="s1">&#39;var&#39;</span>
<span class="kc">false</span></pre></div> </td> </tr> <tr id="section-6"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-6">#</a> </div> <p>Test variables and return true the first time fn(v, k) returns true</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">any: </span><span class="p">(</span><span class="nx">fn</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="k">for</span> <span class="nx">v</span><span class="p">,</span> <span class="nx">k</span> <span class="k">of</span> <span class="err">@</span><span class="nx">variables</span> <span class="k">when</span> <span class="nx">fn</span><span class="p">(</span><span class="nx">v</span><span class="p">,</span> <span class="nx">k</span><span class="p">)</span>
<span class="k">for</span> <span class="nx">v</span><span class="p">,</span> <span class="nx">k</span> <span class="k">of</span> <span class="nx">@variables</span> <span class="k">when</span> <span class="nx">fn</span><span class="p">(</span><span class="nx">v</span><span class="p">,</span> <span class="nx">k</span><span class="p">)</span>
<span class="k">return</span> <span class="kc">true</span>
<span class="k">return</span> <span class="kc">false</span></pre></div> </td> </tr> <tr id="section-7"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-7">#</a> </div> <p>Reserve a variable name as originating from a function parameter for this
scope. No <code>var</code> required for internal references.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">parameter: </span><span class="p">(</span><span class="nx">name</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="err">@</span><span class="nx">variables</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span><span class="o">:</span> <span class="s1">&#39;param&#39;</span></pre></div> </td> </tr> <tr id="section-8"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-8">#</a> </div> <p>Just check to see if a variable has already been declared, without reserving.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">check: </span><span class="p">(</span><span class="nx">name</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="k">return</span> <span class="kc">true</span> <span class="k">if</span> <span class="err">@</span><span class="nx">variables</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span>
<span class="o">!!</span><span class="p">(</span><span class="err">@</span><span class="nx">parent</span> <span class="o">and</span> <span class="err">@</span><span class="nx">parent</span><span class="p">.</span><span class="nx">check</span><span class="p">(</span><span class="nx">name</span><span class="p">))</span></pre></div> </td> </tr> <tr id="section-9"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-9">#</a> </div> <p>If we need to store an intermediate result, find an available name for a
<span class="nx">@variables</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span><span class="o">:</span> <span class="s1">&#39;param&#39;</span></pre></div> </td> </tr> <tr id="section-8"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-8">#</a> </div> <p>Just check to see if a variable has already been declared, without reserving.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">check: </span><span class="p">(</span><span class="nx">name</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="k">return</span> <span class="kc">true</span> <span class="k">if</span> <span class="nx">@variables</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span>
<span class="o">!!</span><span class="p">(</span><span class="nx">@parent</span> <span class="o">and</span> <span class="nx">@parent</span><span class="p">.</span><span class="nx">check</span><span class="p">(</span><span class="nx">name</span><span class="p">))</span></pre></div> </td> </tr> <tr id="section-9"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-9">#</a> </div> <p>If we need to store an intermediate result, find an available name for a
compiler-generated variable. <code>_a</code>, <code>_b</code>, and so on...</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">free_variable: </span><span class="o">-&gt;</span>
<span class="k">while</span> <span class="err">@</span><span class="nx">check</span> <span class="err">@</span><span class="nx">temp_var</span>
<span class="nv">ordinal: </span><span class="mi">1</span> <span class="o">+</span> <span class="nb">parseInt</span> <span class="err">@</span><span class="nx">temp_var</span><span class="p">.</span><span class="nx">substr</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> <span class="mi">36</span>
<span class="err">@</span><span class="nv">temp_var: </span><span class="s1">&#39;_&#39;</span> <span class="o">+</span> <span class="nx">ordinal</span><span class="p">.</span><span class="nx">toString</span><span class="p">(</span><span class="mi">36</span><span class="p">).</span><span class="nx">replace</span><span class="p">(</span><span class="sr">/\d/g</span><span class="p">,</span> <span class="s1">&#39;a&#39;</span><span class="p">)</span>
<span class="err">@</span><span class="nx">variables</span><span class="p">[</span><span class="err">@</span><span class="nx">temp_var</span><span class="p">]</span><span class="o">:</span> <span class="s1">&#39;var&#39;</span>
<span class="err">@</span><span class="nx">temp_var</span></pre></div> </td> </tr> <tr id="section-10"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-10">#</a> </div> <p>Ensure that an assignment is made at the top of this scope
<span class="k">while</span> <span class="nx">@check</span> <span class="nx">@temp_var</span>
<span class="nv">ordinal: </span><span class="mi">1</span> <span class="o">+</span> <span class="nb">parseInt</span> <span class="nx">@temp_var</span><span class="p">.</span><span class="nx">substr</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> <span class="mi">36</span>
<span class="vi">@temp_var: </span><span class="s1">&#39;_&#39;</span> <span class="o">+</span> <span class="nx">ordinal</span><span class="p">.</span><span class="nx">toString</span><span class="p">(</span><span class="mi">36</span><span class="p">).</span><span class="nx">replace</span><span class="p">(</span><span class="sr">/\d/g</span><span class="p">,</span> <span class="s1">&#39;a&#39;</span><span class="p">)</span>
<span class="nx">@variables</span><span class="p">[</span><span class="nx">@temp_var</span><span class="p">]</span><span class="o">:</span> <span class="s1">&#39;var&#39;</span>
<span class="nx">@temp_var</span></pre></div> </td> </tr> <tr id="section-10"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-10">#</a> </div> <p>Ensure that an assignment is made at the top of this scope
(or at the top-level scope, if requested).</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">assign: </span><span class="p">(</span><span class="nx">name</span><span class="p">,</span> <span class="nx">value</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="err">@</span><span class="nx">variables</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span><span class="o">:</span> <span class="p">{</span><span class="nv">value: </span><span class="nx">value</span><span class="p">,</span> <span class="nv">assigned: </span><span class="kc">true</span><span class="p">}</span></pre></div> </td> </tr> <tr id="section-11"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-11">#</a> </div> <p>Does this scope reference any variables that need to be declared in the
<span class="nx">@variables</span><span class="p">[</span><span class="nx">name</span><span class="p">]</span><span class="o">:</span> <span class="p">{</span><span class="nv">value: </span><span class="nx">value</span><span class="p">,</span> <span class="nv">assigned: </span><span class="kc">true</span><span class="p">}</span></pre></div> </td> </tr> <tr id="section-11"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-11">#</a> </div> <p>Does this scope reference any variables that need to be declared in the
given function body?</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">has_declarations: </span><span class="p">(</span><span class="nx">body</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="nx">body</span> <span class="o">is</span> <span class="err">@</span><span class="nx">expressions</span> <span class="o">and</span> <span class="err">@</span><span class="nx">any</span> <span class="p">(</span><span class="nx">k</span><span class="p">,</span> <span class="nx">val</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nx">val</span> <span class="o">is</span> <span class="s1">&#39;var&#39;</span></pre></div> </td> </tr> <tr id="section-12"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-12">#</a> </div> <p>Does this scope reference any assignments that need to be declared at the
<span class="nx">body</span> <span class="o">is</span> <span class="nx">@expressions</span> <span class="o">and</span> <span class="nx">@any</span> <span class="p">(</span><span class="nx">k</span><span class="p">,</span> <span class="nx">val</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nx">val</span> <span class="o">is</span> <span class="s1">&#39;var&#39;</span></pre></div> </td> </tr> <tr id="section-12"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-12">#</a> </div> <p>Does this scope reference any assignments that need to be declared at the
top of the given function body?</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">has_assignments: </span><span class="p">(</span><span class="nx">body</span><span class="p">)</span> <span class="o">-&gt;</span>
<span class="nx">body</span> <span class="o">is</span> <span class="err">@</span><span class="nx">expressions</span> <span class="o">and</span> <span class="err">@</span><span class="nx">any</span> <span class="p">(</span><span class="nx">k</span><span class="p">,</span> <span class="nx">val</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nx">val</span><span class="p">.</span><span class="nx">assigned</span></pre></div> </td> </tr> <tr id="section-13"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-13">#</a> </div> <p>Return the list of variables first declared in this scope.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">declared_variables: </span><span class="o">-&gt;</span>
<span class="p">(</span><span class="nx">key</span> <span class="k">for</span> <span class="nx">key</span><span class="p">,</span> <span class="nx">val</span> <span class="k">of</span> <span class="err">@</span><span class="nx">variables</span> <span class="k">when</span> <span class="nx">val</span> <span class="o">is</span> <span class="s1">&#39;var&#39;</span><span class="p">).</span><span class="nx">sort</span><span class="p">()</span></pre></div> </td> </tr> <tr id="section-14"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-14">#</a> </div> <p>Return the list of assignments that are supposed to be made at the top
<span class="nx">body</span> <span class="o">is</span> <span class="nx">@expressions</span> <span class="o">and</span> <span class="nx">@any</span> <span class="p">(</span><span class="nx">k</span><span class="p">,</span> <span class="nx">val</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nx">val</span><span class="p">.</span><span class="nx">assigned</span></pre></div> </td> </tr> <tr id="section-13"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-13">#</a> </div> <p>Return the list of variables first declared in this scope.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">declared_variables: </span><span class="o">-&gt;</span>
<span class="p">(</span><span class="nx">key</span> <span class="k">for</span> <span class="nx">key</span><span class="p">,</span> <span class="nx">val</span> <span class="k">of</span> <span class="nx">@variables</span> <span class="k">when</span> <span class="nx">val</span> <span class="o">is</span> <span class="s1">&#39;var&#39;</span><span class="p">).</span><span class="nx">sort</span><span class="p">()</span></pre></div> </td> </tr> <tr id="section-14"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-14">#</a> </div> <p>Return the list of assignments that are supposed to be made at the top
of this scope.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">assigned_variables: </span><span class="o">-&gt;</span>
<span class="s2">&quot;$key = ${val.value}&quot;</span> <span class="k">for</span> <span class="nx">key</span><span class="p">,</span> <span class="nx">val</span> <span class="k">of</span> <span class="err">@</span><span class="nx">variables</span> <span class="k">when</span> <span class="nx">val</span><span class="p">.</span><span class="nx">assigned</span></pre></div> </td> </tr> <tr id="section-15"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-15">#</a> </div> <p>Compile the JavaScript for all of the variable declarations in this scope.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">compiled_declarations: </span><span class="o">-&gt;</span>
<span class="err">@</span><span class="nx">declared_variables</span><span class="p">().</span><span class="nx">join</span> <span class="s1">&#39;, &#39;</span></pre></div> </td> </tr> <tr id="section-16"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-16">#</a> </div> <p>Compile the JavaScript for all of the variable assignments in this scope.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">compiled_assignments: </span><span class="o">-&gt;</span>
<span class="err">@</span><span class="nx">assigned_variables</span><span class="p">().</span><span class="nx">join</span> <span class="s1">&#39;, &#39;</span>
<span class="s2">&quot;$key = ${val.value}&quot;</span> <span class="k">for</span> <span class="nx">key</span><span class="p">,</span> <span class="nx">val</span> <span class="k">of</span> <span class="nx">@variables</span> <span class="k">when</span> <span class="nx">val</span><span class="p">.</span><span class="nx">assigned</span></pre></div> </td> </tr> <tr id="section-15"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-15">#</a> </div> <p>Compile the JavaScript for all of the variable declarations in this scope.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">compiled_declarations: </span><span class="o">-&gt;</span>
<span class="nx">@declared_variables</span><span class="p">().</span><span class="nx">join</span> <span class="s1">&#39;, &#39;</span></pre></div> </td> </tr> <tr id="section-16"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-16">#</a> </div> <p>Compile the JavaScript for all of the variable assignments in this scope.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">compiled_assignments: </span><span class="o">-&gt;</span>
<span class="nx">@assigned_variables</span><span class="p">().</span><span class="nx">join</span> <span class="s1">&#39;, &#39;</span>
</pre></div> </td> </tr> </tbody> </table> </div> </body> </html>