kotovalexarian-likes-github/jashkenas--coffeescript
1
0
Fork 0
mirror of https://github.com/jashkenas/coffeescript.git synced 2022-11-09 12:23:24 -05:00
Code Releases Activity
jashkenas--coffeescript/documentation/docs/rewriter.html
Jeremy Ashkenas 9f614fedec CoffeeScript 1.6.0
2013-03-05 08:19:08 +13:00

325 lines
No EOL
80 KiB
HTML
Raw Blame History

<!DOCTYPE html> <html> <head> <title>rewriter.coffee</title> <meta http-equiv="content-type" content="text/html; charset=UTF-8"> <link rel="stylesheet" media="all" href="docco.css" /> </head> <body> <div id="container"> <div id="background"></div> <div id="jump_to"> Jump To &hellip; <div id="jump_wrapper"> <div id="jump_page"> <a class="source" href="browser.html"> browser.coffee </a> <a class="source" href="cake.html"> cake.coffee </a> <a class="source" href="coffee-script.html"> coffee-script.coffee </a> <a class="source" href="command.html"> command.coffee </a> <a class="source" href="grammar.html"> grammar.coffee </a> <a class="source" href="helpers.html"> helpers.coffee </a> <a class="source" href="index.html"> index.coffee </a> <a class="source" href="lexer.html"> lexer.coffee </a> <a class="source" href="nodes.html"> nodes.coffee </a> <a class="source" href="optparse.html"> optparse.coffee </a> <a class="source" href="repl.html"> repl.coffee </a> <a class="source" href="rewriter.html"> rewriter.coffee </a> <a class="source" href="sourcemap.html"> sourcemap.coffee </a> </div> </div> </div> <table cellpadding="0" cellspacing="0"> <thead> <tr> <th class="docs"> <h1> rewriter.coffee </h1> </th> <th class="code"> </th> </tr> </thead> <tbody> <tr id="section-1"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-1">&#182;</a> </div> <p>The CoffeeScript language has a good deal of optional syntax, implicit syntax,
and shorthand syntax. This can greatly complicate a grammar and bloat
the resulting parse table. Instead of making the parser handle it all, we take
a series of passes over the token stream, using this <strong>Rewriter</strong> to convert
shorthand into the unambiguous long form, add implicit indentation and
parentheses, and generally clean things up.</p> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-2"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-2">&#182;</a> </div> <p>Create a generated token: one that exists due to a use of implicit syntax.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">generate = </span><span class="nf">(tag, value) -&gt;</span>
<span class="nv">tok = </span><span class="p">[</span><span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span><span class="p">]</span>
<span class="nv">tok.generated = </span><span class="kc">yes</span>
<span class="nx">tok</span></pre></div> </td> </tr> <tr id="section-3"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-3">&#182;</a> </div> <p>The <strong>Rewriter</strong> class is used by the <a href="lexer.html">Lexer</a>, directly against
its internal array of tokens.</p> </td> <td class="code"> <div class="highlight"><pre><span class="k">class</span> <span class="nx">exports</span><span class="p">.</span><span class="nx">Rewriter</span></pre></div> </td> </tr> <tr id="section-4"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-4">&#182;</a> </div> <p>Helpful snippet for debugging:
console.log (t[0] + '/' + t[1] for t in @tokens).join ' '</p> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-5"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-5">&#182;</a> </div> <p>Rewrite the token stream in multiple passes, one logical filter at
a time. This could certainly be changed into a single pass through the
stream, with a big ol' efficient switch, but it's much nicer to work with
like this. The order of these passes matters -- indentation must be
corrected before implicit parentheses can be wrapped around blocks of code.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">rewrite: </span><span class="nf">(@tokens) -&gt;</span>
<span class="nx">@removeLeadingNewlines</span><span class="p">()</span>
<span class="nx">@removeMidExpressionNewlines</span><span class="p">()</span>
<span class="nx">@closeOpenCalls</span><span class="p">()</span>
<span class="nx">@closeOpenIndexes</span><span class="p">()</span>
<span class="nx">@addImplicitIndentation</span><span class="p">()</span>
<span class="nx">@tagPostfixConditionals</span><span class="p">()</span>
<span class="nx">@addImplicitBracesAndParens</span><span class="p">()</span>
<span class="nx">@addLocationDataToGeneratedTokens</span><span class="p">()</span>
<span class="nx">@tokens</span></pre></div> </td> </tr> <tr id="section-6"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-6">&#182;</a> </div> <p>Rewrite the token stream, looking one token ahead and behind.
Allow the return value of the block to tell us how many tokens to move
forwards (or backwards) in the stream, to make sure we don't miss anything
as tokens are inserted and removed, and the stream changes length under
our feet.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">scanTokens: </span><span class="nf">(block) -&gt;</span>
<span class="p">{</span><span class="nx">tokens</span><span class="p">}</span> <span class="o">=</span> <span class="k">this</span>
<span class="nv">i = </span><span class="mi">0</span>
<span class="nx">i</span> <span class="o">+=</span> <span class="nx">block</span><span class="p">.</span><span class="nx">call</span> <span class="k">this</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">i</span><span class="p">,</span> <span class="nx">tokens</span> <span class="k">while</span> <span class="nv">token = </span><span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">]</span>
<span class="kc">true</span>
<span class="nv">detectEnd: </span><span class="nf">(i, condition, action) -&gt;</span>
<span class="p">{</span><span class="nx">tokens</span><span class="p">}</span> <span class="o">=</span> <span class="k">this</span>
<span class="nv">levels = </span><span class="mi">0</span>
<span class="k">while</span> <span class="nv">token = </span><span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">]</span>
<span class="k">return</span> <span class="nx">action</span><span class="p">.</span><span class="nx">call</span> <span class="k">this</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">i</span> <span class="k">if</span> <span class="nx">levels</span> <span class="o">is</span> <span class="mi">0</span> <span class="o">and</span> <span class="nx">condition</span><span class="p">.</span><span class="nx">call</span> <span class="k">this</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">i</span>
<span class="k">return</span> <span class="nx">action</span><span class="p">.</span><span class="nx">call</span> <span class="k">this</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span> <span class="k">if</span> <span class="o">not</span> <span class="nx">token</span> <span class="o">or</span> <span class="nx">levels</span> <span class="o">&lt;</span> <span class="mi">0</span>
<span class="k">if</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">in</span> <span class="nx">EXPRESSION_START</span>
<span class="nx">levels</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">else</span> <span class="k">if</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">in</span> <span class="nx">EXPRESSION_END</span>
<span class="nx">levels</span> <span class="o">-=</span> <span class="mi">1</span>
<span class="nx">i</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span></pre></div> </td> </tr> <tr id="section-7"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-7">&#182;</a> </div> <p>Leading newlines would introduce an ambiguity in the grammar, so we
dispatch them here.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">removeLeadingNewlines: </span><span class="nf">-&gt;</span>
<span class="k">break</span> <span class="k">for</span> <span class="p">[</span><span class="nx">tag</span><span class="p">],</span> <span class="nx">i</span> <span class="k">in</span> <span class="nx">@tokens</span> <span class="k">when</span> <span class="nx">tag</span> <span class="o">isnt</span> <span class="s">&#39;TERMINATOR&#39;</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">i</span> <span class="k">if</span> <span class="nx">i</span></pre></div> </td> </tr> <tr id="section-8"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-8">&#182;</a> </div> <p>Some blocks occur in the middle of expressions -- when we're expecting
this, remove their trailing newlines.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">removeMidExpressionNewlines: </span><span class="nf">-&gt;</span>
<span class="nx">@scanTokens</span> <span class="nf">(token, i, tokens) -&gt;</span>
<span class="k">return</span> <span class="mi">1</span> <span class="k">unless</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">&#39;TERMINATOR&#39;</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span> <span class="k">in</span> <span class="nx">EXPRESSION_CLOSE</span>
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">1</span>
<span class="mi">0</span></pre></div> </td> </tr> <tr id="section-9"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-9">&#182;</a> </div> <p>The lexer has tagged the opening parenthesis of a method call. Match it with
its paired close. We have the mis-nested outdent case included here for
calls that close on the same line, just before their outdent.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">closeOpenCalls: </span><span class="nf">-&gt;</span>
<span class="nv">condition = </span><span class="nf">(token, i) -&gt;</span>
<span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">in</span> <span class="p">[</span><span class="s">&#39;)&#39;</span><span class="p">,</span> <span class="s">&#39;CALL_END&#39;</span><span class="p">]</span> <span class="o">or</span>
<span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">&#39;OUTDENT&#39;</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">is</span> <span class="s">&#39;)&#39;</span>
<span class="nv">action = </span><span class="nf">(token, i) -&gt;</span>
<span class="nx">@tokens</span><span class="p">[</span><span class="k">if</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">&#39;OUTDENT&#39;</span> <span class="k">then</span> <span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span> <span class="k">else</span> <span class="nx">i</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="s">&#39;CALL_END&#39;</span>
<span class="nx">@scanTokens</span> <span class="nf">(token, i) -&gt;</span>
<span class="nx">@detectEnd</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="nx">condition</span><span class="p">,</span> <span class="nx">action</span> <span class="k">if</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">&#39;CALL_START&#39;</span>
<span class="mi">1</span></pre></div> </td> </tr> <tr id="section-10"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-10">&#182;</a> </div> <p>The lexer has tagged the opening parenthesis of an indexing operation call.
Match it with its paired close.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">closeOpenIndexes: </span><span class="nf">-&gt;</span>
<span class="nv">condition = </span><span class="nf">(token, i) -&gt;</span>
<span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">in</span> <span class="p">[</span><span class="s">&#39;]&#39;</span><span class="p">,</span> <span class="s">&#39;INDEX_END&#39;</span><span class="p">]</span>
<span class="nv">action = </span><span class="nf">(token, i) -&gt;</span>
<span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="s">&#39;INDEX_END&#39;</span>
<span class="nx">@scanTokens</span> <span class="nf">(token, i) -&gt;</span>
<span class="nx">@detectEnd</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="nx">condition</span><span class="p">,</span> <span class="nx">action</span> <span class="k">if</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">&#39;INDEX_START&#39;</span>
<span class="mi">1</span></pre></div> </td> </tr> <tr id="section-11"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-11">&#182;</a> </div> <p>Match tags in token stream starting at i with pattern, skipping HERECOMMENTs
Pattern may consist of strings (equality), an array of strings (one of)
or null (wildcard)</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">matchTags: </span><span class="nf">(i, pattern...) -&gt;</span>
<span class="nv">fuzz = </span><span class="mi">0</span>
<span class="k">for</span> <span class="nx">j</span> <span class="k">in</span> <span class="p">[</span><span class="mi">0</span> <span class="p">...</span> <span class="nx">pattern</span><span class="p">.</span><span class="nx">length</span><span class="p">]</span>
<span class="nx">fuzz</span> <span class="o">+=</span> <span class="mi">2</span> <span class="k">while</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="nx">j</span> <span class="o">+</span> <span class="nx">fuzz</span><span class="p">)</span> <span class="o">is</span> <span class="s">&#39;HERECOMMENT&#39;</span>
<span class="k">continue</span> <span class="k">if</span> <span class="o">not</span> <span class="nx">pattern</span><span class="p">[</span><span class="nx">j</span><span class="p">]</span><span class="o">?</span>
<span class="nx">pattern</span><span class="p">[</span><span class="nx">j</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="nx">pattern</span><span class="p">[</span><span class="nx">j</span><span class="p">]]</span> <span class="k">if</span> <span class="k">typeof</span> <span class="nx">pattern</span><span class="p">[</span><span class="nx">j</span><span class="p">]</span> <span class="o">is</span> <span class="s">&#39;string&#39;</span>
<span class="k">return</span> <span class="kc">no</span> <span class="k">if</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="nx">j</span> <span class="o">+</span> <span class="nx">fuzz</span><span class="p">)</span> <span class="o">not</span> <span class="k">in</span> <span class="nx">pattern</span><span class="p">[</span><span class="nx">j</span><span class="p">]</span>
<span class="kc">yes</span></pre></div> </td> </tr> <tr id="section-12"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-12">&#182;</a> </div> <p>yes iff standing in front of something looking like
@<x>: or <x>:, skipping over 'HERECOMMENT's</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">looksObjectish: </span><span class="nf">(j) -&gt;</span>
<span class="nx">@matchTags</span><span class="p">(</span><span class="nx">j</span><span class="p">,</span> <span class="s">&#39;@&#39;</span><span class="p">,</span> <span class="kc">null</span><span class="p">,</span> <span class="s">&#39;:&#39;</span><span class="p">)</span> <span class="o">or</span> <span class="nx">@matchTags</span><span class="p">(</span><span class="nx">j</span><span class="p">,</span> <span class="kc">null</span><span class="p">,</span> <span class="s">&#39;:&#39;</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-13"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-13">&#182;</a> </div> <p>yes iff current line of tokens contain an element of tags on same
expression level. Stop searching at LINEBREAKS or explicit start of
containing balanced expression.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">findTagsBackwards: </span><span class="nf">(i, tags) -&gt;</span>
<span class="nv">backStack = </span><span class="p">[]</span>
<span class="k">while</span> <span class="nx">i</span> <span class="o">&gt;=</span> <span class="mi">0</span> <span class="o">and</span> <span class="p">(</span><span class="nx">backStack</span><span class="p">.</span><span class="nx">length</span> <span class="o">or</span>
<span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span><span class="p">)</span> <span class="o">not</span> <span class="k">in</span> <span class="nx">tags</span> <span class="o">and</span>
<span class="p">(</span><span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span><span class="p">)</span> <span class="o">not</span> <span class="k">in</span> <span class="nx">EXPRESSION_START</span> <span class="o">or</span> <span class="nx">@tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">].</span><span class="nx">generated</span><span class="p">)</span> <span class="o">and</span>
<span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span><span class="p">)</span> <span class="o">not</span> <span class="k">in</span> <span class="nx">LINEBREAKS</span><span class="p">)</span>
<span class="nx">backStack</span><span class="p">.</span><span class="nx">push</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span><span class="p">)</span> <span class="k">if</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span><span class="p">)</span> <span class="k">in</span> <span class="nx">EXPRESSION_END</span>
<span class="nx">backStack</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span> <span class="k">if</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span><span class="p">)</span> <span class="k">in</span> <span class="nx">EXPRESSION_START</span> <span class="o">and</span> <span class="nx">backStack</span><span class="p">.</span><span class="nx">length</span>
<span class="nx">i</span> <span class="o">-=</span> <span class="mi">1</span>
<span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span><span class="p">)</span> <span class="k">in</span> <span class="nx">tags</span></pre></div> </td> </tr> <tr id="section-14"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-14">&#182;</a> </div> <p>Look for signs of implicit calls and objects in the token stream and
add them.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">addImplicitBracesAndParens: </span><span class="nf">-&gt;</span></pre></div> </td> </tr> <tr id="section-15"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-15">&#182;</a> </div> <p>Track current balancing depth (both implicit and explicit) on stack.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">stack = </span><span class="p">[]</span>
<span class="nx">@scanTokens</span> <span class="nf">(token, i, tokens) -&gt;</span>
<span class="p">[</span><span class="nx">tag</span><span class="p">]</span> <span class="o">=</span> <span class="nx">token</span>
<span class="p">[</span><span class="nx">prevTag</span><span class="p">]</span> <span class="o">=</span> <span class="k">if</span> <span class="nx">i</span> <span class="o">&gt;</span> <span class="mi">0</span> <span class="k">then</span> <span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span> <span class="k">else</span> <span class="p">[]</span>
<span class="p">[</span><span class="nx">nextTag</span><span class="p">]</span> <span class="o">=</span> <span class="k">if</span> <span class="nx">i</span> <span class="o">&lt;</span> <span class="nx">tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span> <span class="k">then</span> <span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">]</span> <span class="k">else</span> <span class="p">[]</span>
<span class="nv">stackTop = </span><span class="nf">-&gt;</span> <span class="nx">stack</span><span class="p">[</span><span class="nx">stack</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span>
<span class="nv">startIdx = </span><span class="nx">i</span></pre></div> </td> </tr> <tr id="section-16"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-16">&#182;</a> </div> <p>Helper function, used for keeping track of the number of tokens consumed
and spliced, when returning for getting a new token.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">forward = </span><span class="nf">(n) -&gt;</span> <span class="nx">i</span> <span class="o">-</span> <span class="nx">startIdx</span> <span class="o">+</span> <span class="nx">n</span></pre></div> </td> </tr> <tr id="section-17"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-17">&#182;</a> </div> <p>Helper functions</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">inImplicit = </span><span class="nf">-&gt;</span> <span class="nx">stackTop</span><span class="p">()</span><span class="o">?</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span><span class="o">?</span><span class="p">.</span><span class="nx">ours</span>
<span class="nv">inImplicitCall = </span><span class="nf">-&gt;</span> <span class="nx">inImplicit</span><span class="p">()</span> <span class="o">and</span> <span class="nx">stackTop</span><span class="p">()</span><span class="o">?</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">&#39;(&#39;</span>
<span class="nv">inImplicitObject = </span><span class="nf">-&gt;</span> <span class="nx">inImplicit</span><span class="p">()</span> <span class="o">and</span> <span class="nx">stackTop</span><span class="p">()</span><span class="o">?</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">&#39;{&#39;</span></pre></div> </td> </tr> <tr id="section-18"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-18">&#182;</a> </div> <p>Unclosed control statement inside implicit parens (like
class declaration or if-conditionals)</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">inImplicitControl = </span><span class="nf">-&gt;</span> <span class="nx">inImplicit</span> <span class="o">and</span> <span class="nx">stackTop</span><span class="p">()</span><span class="o">?</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">&#39;CONTROL&#39;</span>
<span class="nv">startImplicitCall = </span><span class="nf">(j) -&gt;</span>
<span class="nv">idx = </span><span class="nx">j</span> <span class="o">?</span> <span class="nx">i</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s">&#39;(&#39;</span><span class="p">,</span> <span class="nx">idx</span><span class="p">,</span> <span class="nv">ours: </span><span class="kc">yes</span><span class="p">]</span>
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">idx</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">generate</span> <span class="s">&#39;CALL_START&#39;</span><span class="p">,</span> <span class="s">&#39;(&#39;</span>
<span class="nx">i</span> <span class="o">+=</span> <span class="mi">1</span> <span class="k">if</span> <span class="o">not</span> <span class="nx">j</span><span class="o">?</span>
<span class="nv">endImplicitCall = </span><span class="nf">-&gt;</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">generate</span> <span class="s">&#39;CALL_END&#39;</span><span class="p">,</span> <span class="s">&#39;)&#39;</span>
<span class="nx">i</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="nv">startImplicitObject = </span><span class="nf">(j, startsLine = yes) -&gt;</span>
<span class="nv">idx = </span><span class="nx">j</span> <span class="o">?</span> <span class="nx">i</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s">&#39;{&#39;</span><span class="p">,</span> <span class="nx">idx</span><span class="p">,</span> <span class="nv">sameLine: </span><span class="kc">yes</span><span class="p">,</span> <span class="nv">startsLine: </span><span class="nx">startsLine</span><span class="p">,</span> <span class="nv">ours: </span><span class="kc">yes</span><span class="p">]</span>
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">idx</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">generate</span> <span class="s">&#39;{&#39;</span><span class="p">,</span> <span class="nx">generate</span><span class="p">(</span><span class="k">new</span> <span class="nb">String</span><span class="p">(</span><span class="s">&#39;{&#39;</span><span class="p">))</span>
<span class="nx">i</span> <span class="o">+=</span> <span class="mi">1</span> <span class="k">if</span> <span class="o">not</span> <span class="nx">j</span><span class="o">?</span>
<span class="nv">endImplicitObject = </span><span class="nf">(j) -&gt;</span>
<span class="nv">j = </span><span class="nx">j</span> <span class="o">?</span> <span class="nx">i</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">j</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">generate</span> <span class="s">&#39;}&#39;</span><span class="p">,</span> <span class="s">&#39;}&#39;</span>
<span class="nx">i</span> <span class="o">+=</span> <span class="mi">1</span></pre></div> </td> </tr> <tr id="section-19"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-19">&#182;</a> </div> <p>Don't end an implicit call on next indent if any of these are in an argument</p> </td> <td class="code"> <div class="highlight"><pre> <span class="k">if</span> <span class="nx">inImplicitCall</span><span class="p">()</span> <span class="o">and</span> <span class="nx">tag</span> <span class="k">in</span> <span class="p">[</span><span class="s">&#39;IF&#39;</span><span class="p">,</span> <span class="s">&#39;TRY&#39;</span><span class="p">,</span> <span class="s">&#39;FINALLY&#39;</span><span class="p">,</span> <span class="s">&#39;CATCH&#39;</span><span class="p">,</span>
<span class="s">&#39;CLASS&#39;</span><span class="p">,</span> <span class="s">&#39;SWITCH&#39;</span><span class="p">]</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s">&#39;CONTROL&#39;</span><span class="p">,</span> <span class="nx">i</span><span class="p">,</span> <span class="nv">ours: </span><span class="kc">true</span><span class="p">]</span>
<span class="k">return</span> <span class="nx">forward</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">&#39;INDENT&#39;</span> <span class="o">and</span> <span class="nx">inImplicit</span><span class="p">()</span></pre></div> </td> </tr> <tr id="section-20"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-20">&#182;</a> </div> <p>An INDENT closes an implicit call unless
1. We have seen a CONTROL argument on the line.
2. The last token before the indent is part of the list below</p> </td> <td class="code"> <div class="highlight"><pre> <span class="k">if</span> <span class="nx">prevTag</span> <span class="o">not</span> <span class="k">in</span> <span class="p">[</span><span class="s">&#39;=&gt;&#39;</span><span class="p">,</span> <span class="s">&#39;-&gt;&#39;</span><span class="p">,</span> <span class="s">&#39;[&#39;</span><span class="p">,</span> <span class="s">&#39;(&#39;</span><span class="p">,</span> <span class="s">&#39;,&#39;</span><span class="p">,</span> <span class="s">&#39;{&#39;</span><span class="p">,</span> <span class="s">&#39;TRY&#39;</span><span class="p">,</span> <span class="s">&#39;ELSE&#39;</span><span class="p">,</span> <span class="s">&#39;=&#39;</span><span class="p">]</span>
<span class="nx">endImplicitCall</span><span class="p">()</span> <span class="k">while</span> <span class="nx">inImplicitCall</span><span class="p">()</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span> <span class="k">if</span> <span class="nx">inImplicitControl</span><span class="p">()</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="nx">tag</span><span class="p">,</span> <span class="nx">i</span><span class="p">]</span>
<span class="k">return</span> <span class="nx">forward</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-21"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-21">&#182;</a> </div> <p>Straightforward start of explicit expression</p> </td> <td class="code"> <div class="highlight"><pre> <span class="k">if</span> <span class="nx">tag</span> <span class="k">in</span> <span class="nx">EXPRESSION_START</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="nx">tag</span><span class="p">,</span> <span class="nx">i</span><span class="p">]</span>
<span class="k">return</span> <span class="nx">forward</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-22"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-22">&#182;</a> </div> <p>Close all implicit expressions inside of explicitly closed expressions.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="k">if</span> <span class="nx">tag</span> <span class="k">in</span> <span class="nx">EXPRESSION_END</span>
<span class="k">while</span> <span class="nx">inImplicit</span><span class="p">()</span>
<span class="k">if</span> <span class="nx">inImplicitCall</span><span class="p">()</span>
<span class="nx">endImplicitCall</span><span class="p">()</span>
<span class="k">else</span> <span class="k">if</span> <span class="nx">inImplicitObject</span><span class="p">()</span>
<span class="nx">endImplicitObject</span><span class="p">()</span>
<span class="k">else</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span></pre></div> </td> </tr> <tr id="section-23"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-23">&#182;</a> </div> <p>Recognize standard implicit calls like
f a, f() b, f? c, h[0] d etc.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="k">if</span> <span class="p">(</span><span class="nx">tag</span> <span class="k">in</span> <span class="nx">IMPLICIT_FUNC</span> <span class="o">and</span> <span class="nx">token</span><span class="p">.</span><span class="nx">spaced</span> <span class="o">or</span>
<span class="nx">tag</span> <span class="o">is</span> <span class="s">&#39;?&#39;</span> <span class="o">and</span> <span class="nx">i</span> <span class="o">&gt;</span> <span class="mi">0</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">].</span><span class="nx">spaced</span><span class="p">)</span> <span class="o">and</span>
<span class="p">(</span><span class="nx">nextTag</span> <span class="k">in</span> <span class="nx">IMPLICIT_CALL</span> <span class="o">or</span>
<span class="nx">nextTag</span> <span class="k">in</span> <span class="nx">IMPLICIT_UNSPACED_CALL</span> <span class="o">and</span>
<span class="o">not</span> <span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">]</span><span class="o">?</span><span class="p">.</span><span class="nx">spaced</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">]</span><span class="o">?</span><span class="p">.</span><span class="nx">newLine</span><span class="p">)</span>
<span class="nv">tag = </span><span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="s">&#39;FUNC_EXIST&#39;</span> <span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">&#39;?&#39;</span>
<span class="nx">startImplicitCall</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span>
<span class="k">return</span> <span class="nx">forward</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-24"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-24">&#182;</a> </div> <p>Implicit call taking an implicit indented object as first argument.
f
a: b
c: d
and
f
1
a: b
b: c
Don't accept implicit calls of this type, when on the same line
as the control strucutures below as that may misinterpret constructs like:
if f
a: 1
as
if f(a: 1)
which is probably always unintended.
Furthermore don't allow this in literal arrays, as
that creates grammatical ambiguities.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="k">if</span> <span class="nx">@matchTags</span><span class="p">(</span><span class="nx">i</span><span class="p">,</span> <span class="nx">IMPLICIT_FUNC</span><span class="p">,</span> <span class="s">&#39;INDENT&#39;</span><span class="p">,</span> <span class="kc">null</span><span class="p">,</span> <span class="s">&#39;:&#39;</span><span class="p">)</span> <span class="o">and</span>
<span class="o">not</span> <span class="nx">@findTagsBackwards</span><span class="p">(</span><span class="nx">i</span><span class="p">,</span> <span class="p">[</span><span class="s">&#39;CLASS&#39;</span><span class="p">,</span> <span class="s">&#39;EXTENDS&#39;</span><span class="p">,</span> <span class="s">&#39;IF&#39;</span><span class="p">,</span> <span class="s">&#39;CATCH&#39;</span><span class="p">,</span>
<span class="s">&#39;SWITCH&#39;</span><span class="p">,</span> <span class="s">&#39;LEADING_WHEN&#39;</span><span class="p">,</span> <span class="s">&#39;FOR&#39;</span><span class="p">,</span> <span class="s">&#39;WHILE&#39;</span><span class="p">,</span> <span class="s">&#39;UNTIL&#39;</span><span class="p">])</span>
<span class="nx">startImplicitCall</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s">&#39;INDENT&#39;</span><span class="p">,</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">2</span><span class="p">]</span>
<span class="k">return</span> <span class="nx">forward</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-25"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-25">&#182;</a> </div> <p>Implicit objects start here</p> </td> <td class="code"> <div class="highlight"><pre> <span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">&#39;:&#39;</span></pre></div> </td> </tr> <tr id="section-26"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-26">&#182;</a> </div> <p>Go back to the (implicit) start of the object</p> </td> <td class="code"> <div class="highlight"><pre> <span class="k">if</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">2</span><span class="p">)</span> <span class="o">is</span> <span class="s">&#39;@&#39;</span> <span class="k">then</span> <span class="nv">s = </span><span class="nx">i</span> <span class="o">-</span> <span class="mi">2</span> <span class="k">else</span> <span class="nv">s = </span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span>
<span class="nx">s</span> <span class="o">-=</span> <span class="mi">2</span> <span class="k">while</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">s</span> <span class="o">-</span> <span class="mi">2</span><span class="p">)</span> <span class="o">is</span> <span class="s">&#39;HERECOMMENT&#39;</span>
<span class="nv">startsLine = </span><span class="nx">s</span> <span class="o">is</span> <span class="mi">0</span> <span class="o">or</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">s</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="k">in</span> <span class="nx">LINEBREAKS</span> <span class="o">or</span> <span class="nx">tokens</span><span class="p">[</span><span class="nx">s</span> <span class="o">-</span> <span class="mi">1</span><span class="p">].</span><span class="nx">newLine</span></pre></div> </td> </tr> <tr id="section-27"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-27">&#182;</a> </div> <p>Are we just continuing an already declared object?</p> </td> <td class="code"> <div class="highlight"><pre> <span class="k">if</span> <span class="nx">stackTop</span><span class="p">()</span>
<span class="p">[</span><span class="nx">stackTag</span><span class="p">,</span> <span class="nx">stackIdx</span><span class="p">]</span> <span class="o">=</span> <span class="nx">stackTop</span><span class="p">()</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">stackTag</span> <span class="o">is</span> <span class="s">&#39;{&#39;</span> <span class="o">or</span> <span class="nx">stackTag</span> <span class="o">is</span> <span class="s">&#39;INDENT&#39;</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">stackIdx</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">is</span> <span class="s">&#39;{&#39;</span><span class="p">)</span> <span class="o">and</span>
<span class="p">(</span><span class="nx">startsLine</span> <span class="o">or</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">s</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">is</span> <span class="s">&#39;,&#39;</span> <span class="o">or</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">s</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">is</span> <span class="s">&#39;{&#39;</span><span class="p">)</span>
<span class="k">return</span> <span class="nx">forward</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="nx">startImplicitObject</span><span class="p">(</span><span class="nx">s</span><span class="p">,</span> <span class="o">!!</span><span class="nx">startsLine</span><span class="p">)</span>
<span class="k">return</span> <span class="nx">forward</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-28"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-28">&#182;</a> </div> <p>End implicit calls when chaining method calls
like e.g.:
f ->
a
.g b, ->
c
.h a</p> </td> <td class="code"> <div class="highlight"><pre> <span class="k">if</span> <span class="nx">prevTag</span> <span class="o">is</span> <span class="s">&#39;OUTDENT&#39;</span> <span class="o">and</span> <span class="nx">inImplicitCall</span><span class="p">()</span> <span class="o">and</span> <span class="nx">tag</span> <span class="k">in</span> <span class="p">[</span><span class="s">&#39;.&#39;</span><span class="p">,</span> <span class="s">&#39;?.&#39;</span><span class="p">,</span> <span class="s">&#39;::&#39;</span><span class="p">,</span> <span class="s">&#39;?::&#39;</span><span class="p">]</span>
<span class="nx">endImplicitCall</span><span class="p">()</span>
<span class="k">return</span> <span class="nx">forward</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="nx">stackTop</span><span class="p">()[</span><span class="mi">2</span><span class="p">].</span><span class="nv">sameLine = </span><span class="kc">no</span> <span class="k">if</span> <span class="nx">inImplicitObject</span><span class="p">()</span> <span class="o">and</span> <span class="nx">tag</span> <span class="k">in</span> <span class="nx">LINEBREAKS</span>
<span class="k">if</span> <span class="nx">tag</span> <span class="k">in</span> <span class="nx">IMPLICIT_END</span>
<span class="k">while</span> <span class="nx">inImplicit</span><span class="p">()</span>
<span class="p">[</span><span class="nx">stackTag</span><span class="p">,</span> <span class="nx">stackIdx</span><span class="p">,</span> <span class="p">{</span><span class="nx">sameLine</span><span class="p">,</span> <span class="nx">startsLine</span><span class="p">}]</span> <span class="o">=</span> <span class="nx">stackTop</span><span class="p">()</span></pre></div> </td> </tr> <tr id="section-29"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-29">&#182;</a> </div> <p>Close implicit calls when reached end of argument list</p> </td> <td class="code"> <div class="highlight"><pre> <span class="k">if</span> <span class="nx">inImplicitCall</span><span class="p">()</span> <span class="o">and</span> <span class="nx">prevTag</span> <span class="o">isnt</span> <span class="s">&#39;,&#39;</span>
<span class="nx">endImplicitCall</span><span class="p">()</span></pre></div> </td> </tr> <tr id="section-30"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-30">&#182;</a> </div> <p>Close implicit objects such as:
return a: 1, b: 2 unless true</p> </td> <td class="code"> <div class="highlight"><pre> <span class="k">else</span> <span class="k">if</span> <span class="nx">inImplicitObject</span><span class="p">()</span> <span class="o">and</span> <span class="nx">sameLine</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">startsLine</span>
<span class="nx">endImplicitObject</span><span class="p">()</span></pre></div> </td> </tr> <tr id="section-31"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-31">&#182;</a> </div> <p>Close implicit objects when at end of line, line didn't end with a comma
and the implicit object didn't start the line or the next line doesn't look like
the continuation of an object.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="k">else</span> <span class="k">if</span> <span class="nx">inImplicitObject</span><span class="p">()</span> <span class="o">and</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">&#39;TERMINATOR&#39;</span> <span class="o">and</span> <span class="nx">prevTag</span> <span class="o">isnt</span> <span class="s">&#39;,&#39;</span> <span class="o">and</span>
<span class="o">not</span> <span class="p">(</span><span class="nx">startsLine</span> <span class="o">and</span> <span class="nx">@looksObjectish</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">))</span>
<span class="nx">endImplicitObject</span><span class="p">()</span>
<span class="k">else</span>
<span class="k">break</span></pre></div> </td> </tr> <tr id="section-32"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-32">&#182;</a> </div> <p>Close implicit object if comma is the last character
and what comes after doesn't look like it belongs.
This is used for trailing commas and calls, like:
x =
a: b,
c: d,
e = 2</p>
<p>and</p>
<p>f a, b: c, d: e, f, g: h: i, j</p> </td> <td class="code"> <div class="highlight"><pre> <span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">&#39;,&#39;</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">@looksObjectish</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span> <span class="o">and</span> <span class="nx">inImplicitObject</span><span class="p">()</span> <span class="o">and</span>
<span class="p">(</span><span class="nx">nextTag</span> <span class="o">isnt</span> <span class="s">&#39;TERMINATOR&#39;</span> <span class="o">or</span> <span class="o">not</span> <span class="nx">@looksObjectish</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">2</span><span class="p">))</span></pre></div> </td> </tr> <tr id="section-33"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-33">&#182;</a> </div> <p>When nextTag is OUTDENT the comma is insignificant and
should just be ignored so embed it in the implicit object.</p>
<p>When it isn't the comma go on to play a role in a call or
array further up the stack, so give it a chance.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">offset = </span><span class="k">if</span> <span class="nx">nextTag</span> <span class="o">is</span> <span class="s">&#39;OUTDENT&#39;</span> <span class="k">then</span> <span class="mi">1</span> <span class="k">else</span> <span class="mi">0</span>
<span class="k">while</span> <span class="nx">inImplicitObject</span><span class="p">()</span>
<span class="nx">endImplicitObject</span> <span class="nx">i</span> <span class="o">+</span> <span class="nx">offset</span>
<span class="k">return</span> <span class="nx">forward</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-34"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-34">&#182;</a> </div> <p>Add location data to all tokens generated by the rewriter.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">addLocationDataToGeneratedTokens: </span><span class="nf">-&gt;</span>
<span class="nx">@scanTokens</span> <span class="nf">(token, i, tokens) -&gt;</span>
<span class="k">return</span> <span class="mi">1</span> <span class="k">if</span> <span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
<span class="k">return</span> <span class="mi">1</span> <span class="k">unless</span> <span class="nx">token</span><span class="p">.</span><span class="nx">generated</span> <span class="o">or</span> <span class="nx">token</span><span class="p">.</span><span class="nx">explicit</span>
<span class="p">{</span><span class="nx">last_line</span><span class="p">,</span> <span class="nx">last_column</span><span class="p">}</span> <span class="o">=</span> <span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span><span class="o">?</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">?</span> <span class="nv">last_line: </span><span class="mi">0</span><span class="p">,</span> <span class="nv">last_column: </span><span class="mi">0</span>
<span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">=</span>
<span class="nv">first_line: </span> <span class="nx">last_line</span>
<span class="nv">first_column: </span><span class="nx">last_column</span>
<span class="nv">last_line: </span> <span class="nx">last_line</span>
<span class="nv">last_column: </span> <span class="nx">last_column</span>
<span class="mi">1</span></pre></div> </td> </tr> <tr id="section-35"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-35">&#182;</a> </div> <p>Because our grammar is LALR(1), it can't handle some single-line
expressions that lack ending delimiters. The <strong>Rewriter</strong> adds the implicit
blocks, so it doesn't need to. ')' can close a single-line block,
but we need to make sure it's balanced.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">addImplicitIndentation: </span><span class="nf">-&gt;</span>
<span class="nv">starter = indent = outdent = </span><span class="kc">null</span>
<span class="nv">condition = </span><span class="nf">(token, i) -&gt;</span>
<span class="nx">token</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">isnt</span> <span class="s">&#39;;&#39;</span> <span class="o">and</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">in</span> <span class="nx">SINGLE_CLOSERS</span> <span class="o">and</span>
<span class="o">not</span> <span class="p">(</span><span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">&#39;ELSE&#39;</span> <span class="o">and</span> <span class="nx">starter</span> <span class="o">not</span> <span class="k">in</span> <span class="p">[</span><span class="s">&#39;IF&#39;</span><span class="p">,</span> <span class="s">&#39;THEN&#39;</span><span class="p">])</span>
<span class="nv">action = </span><span class="nf">(token, i) -&gt;</span>
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="p">(</span><span class="k">if</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">is</span> <span class="s">&#39;,&#39;</span> <span class="k">then</span> <span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span> <span class="k">else</span> <span class="nx">i</span><span class="p">),</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">outdent</span>
<span class="nx">@scanTokens</span> <span class="nf">(token, i, tokens) -&gt;</span>
<span class="p">[</span><span class="nx">tag</span><span class="p">]</span> <span class="o">=</span> <span class="nx">token</span>
<span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">&#39;TERMINATOR&#39;</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span> <span class="o">is</span> <span class="s">&#39;THEN&#39;</span>
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">1</span>
<span class="k">return</span> <span class="mi">0</span>
<span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">&#39;ELSE&#39;</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">isnt</span> <span class="s">&#39;OUTDENT&#39;</span>
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">@indentation</span><span class="p">(</span><span class="nx">token</span><span class="p">)...</span>
<span class="k">return</span> <span class="mi">2</span>
<span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">&#39;CATCH&#39;</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">2</span><span class="p">)</span> <span class="k">in</span> <span class="p">[</span><span class="s">&#39;OUTDENT&#39;</span><span class="p">,</span> <span class="s">&#39;TERMINATOR&#39;</span><span class="p">,</span> <span class="s">&#39;FINALLY&#39;</span><span class="p">]</span>
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">@indentation</span><span class="p">(</span><span class="nx">token</span><span class="p">)...</span>
<span class="k">return</span> <span class="mi">4</span>
<span class="k">if</span> <span class="nx">tag</span> <span class="k">in</span> <span class="nx">SINGLE_LINERS</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span> <span class="o">isnt</span> <span class="s">&#39;INDENT&#39;</span> <span class="o">and</span>
<span class="o">not</span> <span class="p">(</span><span class="nx">tag</span> <span class="o">is</span> <span class="s">&#39;ELSE&#39;</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span> <span class="o">is</span> <span class="s">&#39;IF&#39;</span><span class="p">)</span>
<span class="nv">starter = </span><span class="nx">tag</span>
<span class="p">[</span><span class="nx">indent</span><span class="p">,</span> <span class="nx">outdent</span><span class="p">]</span> <span class="o">=</span> <span class="nx">@indentation</span> <span class="nx">token</span><span class="p">,</span> <span class="kc">yes</span>
<span class="nv">indent.fromThen = </span><span class="kc">true</span> <span class="k">if</span> <span class="nx">starter</span> <span class="o">is</span> <span class="s">&#39;THEN&#39;</span>
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">indent</span>
<span class="nx">@detectEnd</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">2</span><span class="p">,</span> <span class="nx">condition</span><span class="p">,</span> <span class="nx">action</span>
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">1</span> <span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">&#39;THEN&#39;</span>
<span class="k">return</span> <span class="mi">1</span>
<span class="k">return</span> <span class="mi">1</span></pre></div> </td> </tr> <tr id="section-36"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-36">&#182;</a> </div> <p>Tag postfix conditionals as such, so that we can parse them with a
different precedence.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tagPostfixConditionals: </span><span class="nf">-&gt;</span>
<span class="nv">original = </span><span class="kc">null</span>
<span class="nv">condition = </span><span class="nf">(token, i) -&gt;</span>
<span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">in</span> <span class="p">[</span><span class="s">&#39;TERMINATOR&#39;</span><span class="p">,</span> <span class="s">&#39;INDENT&#39;</span><span class="p">]</span>
<span class="nv">action = </span><span class="nf">(token, i) -&gt;</span>
<span class="k">if</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">isnt</span> <span class="s">&#39;INDENT&#39;</span> <span class="o">or</span> <span class="p">(</span><span class="nx">token</span><span class="p">.</span><span class="nx">generated</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">token</span><span class="p">.</span><span class="nx">fromThen</span><span class="p">)</span>
<span class="nx">original</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="s">&#39;POST_&#39;</span> <span class="o">+</span> <span class="nx">original</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="nx">@scanTokens</span> <span class="nf">(token, i) -&gt;</span>
<span class="k">return</span> <span class="mi">1</span> <span class="k">unless</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">&#39;IF&#39;</span>
<span class="nv">original = </span><span class="nx">token</span>
<span class="nx">@detectEnd</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="nx">condition</span><span class="p">,</span> <span class="nx">action</span>
<span class="mi">1</span></pre></div> </td> </tr> <tr id="section-37"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-37">&#182;</a> </div> <p>Generate the indentation tokens, based on another token on the same line.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">indentation: </span><span class="nf">(token, implicit = no) -&gt;</span>
<span class="nv">indent = </span><span class="p">[</span><span class="s">&#39;INDENT&#39;</span><span class="p">,</span> <span class="mi">2</span><span class="p">]</span>
<span class="nv">outdent = </span><span class="p">[</span><span class="s">&#39;OUTDENT&#39;</span><span class="p">,</span> <span class="mi">2</span><span class="p">]</span>
<span class="nv">indent.generated = outdent.generated = </span><span class="kc">yes</span> <span class="k">if</span> <span class="nx">implicit</span>
<span class="nv">indent.explicit = outdent.explicit = </span><span class="kc">yes</span> <span class="k">if</span> <span class="o">not</span> <span class="nx">implicit</span>
<span class="p">[</span><span class="nx">indent</span><span class="p">,</span> <span class="nx">outdent</span><span class="p">]</span>
<span class="nv">generate: </span><span class="nx">generate</span></pre></div> </td> </tr> <tr id="section-38"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-38">&#182;</a> </div> <p>Look up a tag by token index.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tag: </span><span class="nf">(i) -&gt;</span> <span class="nx">@tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">]</span><span class="o">?</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-39"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-39">&#182;</a> </div> <h2>Constants</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-40"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-40">&#182;</a> </div> <p>List of the token pairs that must be balanced.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">BALANCED_PAIRS = </span><span class="p">[</span>
<span class="p">[</span><span class="s">&#39;(&#39;</span><span class="p">,</span> <span class="s">&#39;)&#39;</span><span class="p">]</span>
<span class="p">[</span><span class="s">&#39;[&#39;</span><span class="p">,</span> <span class="s">&#39;]&#39;</span><span class="p">]</span>
<span class="p">[</span><span class="s">&#39;{&#39;</span><span class="p">,</span> <span class="s">&#39;}&#39;</span><span class="p">]</span>
<span class="p">[</span><span class="s">&#39;INDENT&#39;</span><span class="p">,</span> <span class="s">&#39;OUTDENT&#39;</span><span class="p">],</span>
<span class="p">[</span><span class="s">&#39;CALL_START&#39;</span><span class="p">,</span> <span class="s">&#39;CALL_END&#39;</span><span class="p">]</span>
<span class="p">[</span><span class="s">&#39;PARAM_START&#39;</span><span class="p">,</span> <span class="s">&#39;PARAM_END&#39;</span><span class="p">]</span>
<span class="p">[</span><span class="s">&#39;INDEX_START&#39;</span><span class="p">,</span> <span class="s">&#39;INDEX_END&#39;</span><span class="p">]</span>
<span class="p">]</span></pre></div> </td> </tr> <tr id="section-41"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-41">&#182;</a> </div> <p>The inverse mappings of <code>BALANCED_PAIRS</code> we're trying to fix up, so we can
look things up from either end.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">exports.INVERSES = INVERSES = </span><span class="p">{}</span></pre></div> </td> </tr> <tr id="section-42"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-42">&#182;</a> </div> <p>The tokens that signal the start/end of a balanced pair.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">EXPRESSION_START = </span><span class="p">[]</span>
<span class="nv">EXPRESSION_END = </span><span class="p">[]</span>
<span class="k">for</span> <span class="p">[</span><span class="nx">left</span><span class="p">,</span> <span class="nx">rite</span><span class="p">]</span> <span class="k">in</span> <span class="nx">BALANCED_PAIRS</span>
<span class="nx">EXPRESSION_START</span><span class="p">.</span><span class="nx">push</span> <span class="nx">INVERSES</span><span class="p">[</span><span class="nx">rite</span><span class="p">]</span> <span class="o">=</span> <span class="nx">left</span>
<span class="nx">EXPRESSION_END</span> <span class="p">.</span><span class="nx">push</span> <span class="nx">INVERSES</span><span class="p">[</span><span class="nx">left</span><span class="p">]</span> <span class="o">=</span> <span class="nx">rite</span></pre></div> </td> </tr> <tr id="section-43"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-43">&#182;</a> </div> <p>Tokens that indicate the close of a clause of an expression.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">EXPRESSION_CLOSE = </span><span class="p">[</span><span class="s">&#39;CATCH&#39;</span><span class="p">,</span> <span class="s">&#39;WHEN&#39;</span><span class="p">,</span> <span class="s">&#39;ELSE&#39;</span><span class="p">,</span> <span class="s">&#39;FINALLY&#39;</span><span class="p">].</span><span class="nx">concat</span> <span class="nx">EXPRESSION_END</span></pre></div> </td> </tr> <tr id="section-44"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-44">&#182;</a> </div> <p>Tokens that, if followed by an <code>IMPLICIT_CALL</code>, indicate a function invocation.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">IMPLICIT_FUNC = </span><span class="p">[</span><span class="s">&#39;IDENTIFIER&#39;</span><span class="p">,</span> <span class="s">&#39;SUPER&#39;</span><span class="p">,</span> <span class="s">&#39;)&#39;</span><span class="p">,</span> <span class="s">&#39;CALL_END&#39;</span><span class="p">,</span> <span class="s">&#39;]&#39;</span><span class="p">,</span> <span class="s">&#39;INDEX_END&#39;</span><span class="p">,</span> <span class="s">&#39;@&#39;</span><span class="p">,</span> <span class="s">&#39;THIS&#39;</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-45"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-45">&#182;</a> </div> <p>If preceded by an <code>IMPLICIT_FUNC</code>, indicates a function invocation.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">IMPLICIT_CALL = </span><span class="p">[</span>
<span class="s">&#39;IDENTIFIER&#39;</span><span class="p">,</span> <span class="s">&#39;NUMBER&#39;</span><span class="p">,</span> <span class="s">&#39;STRING&#39;</span><span class="p">,</span> <span class="s">&#39;JS&#39;</span><span class="p">,</span> <span class="s">&#39;REGEX&#39;</span><span class="p">,</span> <span class="s">&#39;NEW&#39;</span><span class="p">,</span> <span class="s">&#39;PARAM_START&#39;</span><span class="p">,</span> <span class="s">&#39;CLASS&#39;</span>
<span class="s">&#39;IF&#39;</span><span class="p">,</span> <span class="s">&#39;TRY&#39;</span><span class="p">,</span> <span class="s">&#39;SWITCH&#39;</span><span class="p">,</span> <span class="s">&#39;THIS&#39;</span><span class="p">,</span> <span class="s">&#39;BOOL&#39;</span><span class="p">,</span> <span class="s">&#39;NULL&#39;</span><span class="p">,</span> <span class="s">&#39;UNDEFINED&#39;</span><span class="p">,</span> <span class="s">&#39;UNARY&#39;</span><span class="p">,</span> <span class="s">&#39;SUPER&#39;</span>
<span class="s">&#39;@&#39;</span><span class="p">,</span> <span class="s">&#39;-&gt;&#39;</span><span class="p">,</span> <span class="s">&#39;=&gt;&#39;</span><span class="p">,</span> <span class="s">&#39;[&#39;</span><span class="p">,</span> <span class="s">&#39;(&#39;</span><span class="p">,</span> <span class="s">&#39;{&#39;</span><span class="p">,</span> <span class="s">&#39;--&#39;</span><span class="p">,</span> <span class="s">&#39;++&#39;</span>
<span class="p">]</span>
<span class="nv">IMPLICIT_UNSPACED_CALL = </span><span class="p">[</span><span class="s">&#39;+&#39;</span><span class="p">,</span> <span class="s">&#39;-&#39;</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-46"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-46">&#182;</a> </div> <p>Tokens indicating that the implicit call must enclose a block of expressions.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">IMPLICIT_BLOCK = </span><span class="p">[</span><span class="s">&#39;-&gt;&#39;</span><span class="p">,</span> <span class="s">&#39;=&gt;&#39;</span><span class="p">,</span> <span class="s">&#39;{&#39;</span><span class="p">,</span> <span class="s">&#39;[&#39;</span><span class="p">,</span> <span class="s">&#39;,&#39;</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-47"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-47">&#182;</a> </div> <p>Tokens that always mark the end of an implicit call for single-liners.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">IMPLICIT_END = </span><span class="p">[</span><span class="s">&#39;POST_IF&#39;</span><span class="p">,</span> <span class="s">&#39;FOR&#39;</span><span class="p">,</span> <span class="s">&#39;WHILE&#39;</span><span class="p">,</span> <span class="s">&#39;UNTIL&#39;</span><span class="p">,</span> <span class="s">&#39;WHEN&#39;</span><span class="p">,</span> <span class="s">&#39;BY&#39;</span><span class="p">,</span>
<span class="s">&#39;LOOP&#39;</span><span class="p">,</span> <span class="s">&#39;TERMINATOR&#39;</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-48"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-48">&#182;</a> </div> <p>Single-line flavors of block expressions that have unclosed endings.
The grammar can't disambiguate them, so we insert the implicit indentation.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">SINGLE_LINERS = </span><span class="p">[</span><span class="s">&#39;ELSE&#39;</span><span class="p">,</span> <span class="s">&#39;-&gt;&#39;</span><span class="p">,</span> <span class="s">&#39;=&gt;&#39;</span><span class="p">,</span> <span class="s">&#39;TRY&#39;</span><span class="p">,</span> <span class="s">&#39;FINALLY&#39;</span><span class="p">,</span> <span class="s">&#39;THEN&#39;</span><span class="p">]</span>
<span class="nv">SINGLE_CLOSERS = </span><span class="p">[</span><span class="s">&#39;TERMINATOR&#39;</span><span class="p">,</span> <span class="s">&#39;CATCH&#39;</span><span class="p">,</span> <span class="s">&#39;FINALLY&#39;</span><span class="p">,</span> <span class="s">&#39;ELSE&#39;</span><span class="p">,</span> <span class="s">&#39;OUTDENT&#39;</span><span class="p">,</span> <span class="s">&#39;LEADING_WHEN&#39;</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-49"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-49">&#182;</a> </div> <p>Tokens that end a line.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">LINEBREAKS = </span><span class="p">[</span><span class="s">&#39;TERMINATOR&#39;</span><span class="p">,</span> <span class="s">&#39;INDENT&#39;</span><span class="p">,</span> <span class="s">&#39;OUTDENT&#39;</span><span class="p">]</span>
</pre></div> </td> </tr> </tbody> </table> </div> </body> </html>
View git blame Copy permalink