243 lines
60 KiB
HTML
243 lines
60 KiB
HTML
<!DOCTYPE html> <html> <head> <title>rewriter.coffee</title> <meta http-equiv="content-type" content="text/html; charset=UTF-8"> <link rel="stylesheet" media="all" href="docco.css" /> </head> <body> <div id="container"> <div id="background"></div> <div id="jump_to"> Jump To … <div id="jump_wrapper"> <div id="jump_page"> <a class="source" href="browser.html"> browser.coffee </a> <a class="source" href="cake.html"> cake.coffee </a> <a class="source" href="coffee-script.html"> coffee-script.coffee </a> <a class="source" href="command.html"> command.coffee </a> <a class="source" href="grammar.html"> grammar.coffee </a> <a class="source" href="helpers.html"> helpers.coffee </a> <a class="source" href="index.html"> index.coffee </a> <a class="source" href="lexer.html"> lexer.coffee </a> <a class="source" href="nodes.html"> nodes.coffee </a> <a class="source" href="optparse.html"> optparse.coffee </a> <a class="source" href="repl.html"> repl.coffee </a> <a class="source" href="rewriter.html"> rewriter.coffee </a> <a class="source" href="scope.html"> scope.coffee </a> </div> </div> </div> <table cellpadding="0" cellspacing="0"> <thead> <tr> <th class="docs"> <h1> rewriter.coffee </h1> </th> <th class="code"> </th> </tr> </thead> <tbody> <tr id="section-1"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-1">¶</a> </div> <p>The CoffeeScript language has a good deal of optional syntax, implicit syntax,
|
|
and shorthand syntax. This can greatly complicate a grammar and bloat
|
|
the resulting parse table. Instead of making the parser handle it all, we take
|
|
a series of passes over the token stream, using this <strong>Rewriter</strong> to convert
|
|
shorthand into the unambiguous long form, add implicit indentation and
|
|
parentheses, and generally clean things up.</p> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-2"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-2">¶</a> </div> <p>The <strong>Rewriter</strong> class is used by the <a href="lexer.html">Lexer</a>, directly against
|
|
its internal array of tokens.</p> </td> <td class="code"> <div class="highlight"><pre><span class="k">class</span> <span class="nx">exports</span><span class="p">.</span><span class="nx">Rewriter</span></pre></div> </td> </tr> <tr id="section-3"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-3">¶</a> </div> <p>Helpful snippet for debugging:
|
|
console.log (t[0] + '/' + t[1] for t in @tokens).join ' '</p> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-4"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-4">¶</a> </div> <p>Rewrite the token stream in multiple passes, one logical filter at
|
|
a time. This could certainly be changed into a single pass through the
|
|
stream, with a big ol' efficient switch, but it's much nicer to work with
|
|
like this. The order of these passes matters -- indentation must be
|
|
corrected before implicit parentheses can be wrapped around blocks of code.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">rewrite: </span><span class="nf">(@tokens) -></span>
|
|
<span class="nx">@removeLeadingNewlines</span><span class="p">()</span>
|
|
<span class="nx">@removeMidExpressionNewlines</span><span class="p">()</span>
|
|
<span class="nx">@closeOpenCalls</span><span class="p">()</span>
|
|
<span class="nx">@closeOpenIndexes</span><span class="p">()</span>
|
|
<span class="nx">@addImplicitIndentation</span><span class="p">()</span>
|
|
<span class="nx">@tagPostfixConditionals</span><span class="p">()</span>
|
|
<span class="nx">@addImplicitBraces</span><span class="p">()</span>
|
|
<span class="nx">@addImplicitParentheses</span><span class="p">()</span>
|
|
<span class="nx">@tokens</span></pre></div> </td> </tr> <tr id="section-5"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-5">¶</a> </div> <p>Rewrite the token stream, looking one token ahead and behind.
|
|
Allow the return value of the block to tell us how many tokens to move
|
|
forwards (or backwards) in the stream, to make sure we don't miss anything
|
|
as tokens are inserted and removed, and the stream changes length under
|
|
our feet.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">scanTokens: </span><span class="nf">(block) -></span>
|
|
<span class="p">{</span><span class="nx">tokens</span><span class="p">}</span> <span class="o">=</span> <span class="k">this</span>
|
|
<span class="nv">i = </span><span class="mi">0</span>
|
|
<span class="nx">i</span> <span class="o">+=</span> <span class="nx">block</span><span class="p">.</span><span class="nx">call</span> <span class="k">this</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">i</span><span class="p">,</span> <span class="nx">tokens</span> <span class="k">while</span> <span class="nv">token = </span><span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">]</span>
|
|
<span class="kc">true</span>
|
|
|
|
<span class="nv">detectEnd: </span><span class="nf">(i, condition, action) -></span>
|
|
<span class="p">{</span><span class="nx">tokens</span><span class="p">}</span> <span class="o">=</span> <span class="k">this</span>
|
|
<span class="nv">levels = </span><span class="mi">0</span>
|
|
<span class="k">while</span> <span class="nv">token = </span><span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">]</span>
|
|
<span class="k">return</span> <span class="nx">action</span><span class="p">.</span><span class="nx">call</span> <span class="k">this</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">i</span> <span class="k">if</span> <span class="nx">levels</span> <span class="o">is</span> <span class="mi">0</span> <span class="o">and</span> <span class="nx">condition</span><span class="p">.</span><span class="nx">call</span> <span class="k">this</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">i</span>
|
|
<span class="k">return</span> <span class="nx">action</span><span class="p">.</span><span class="nx">call</span> <span class="k">this</span><span class="p">,</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span> <span class="k">if</span> <span class="o">not</span> <span class="nx">token</span> <span class="o">or</span> <span class="nx">levels</span> <span class="o"><</span> <span class="mi">0</span>
|
|
<span class="k">if</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">in</span> <span class="nx">EXPRESSION_START</span>
|
|
<span class="nx">levels</span> <span class="o">+=</span> <span class="mi">1</span>
|
|
<span class="k">else</span> <span class="k">if</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">in</span> <span class="nx">EXPRESSION_END</span>
|
|
<span class="nx">levels</span> <span class="o">-=</span> <span class="mi">1</span>
|
|
<span class="nx">i</span> <span class="o">+=</span> <span class="mi">1</span>
|
|
<span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span></pre></div> </td> </tr> <tr id="section-6"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-6">¶</a> </div> <p>Leading newlines would introduce an ambiguity in the grammar, so we
|
|
dispatch them here.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">removeLeadingNewlines: </span><span class="nf">-></span>
|
|
<span class="k">break</span> <span class="k">for</span> <span class="p">[</span><span class="nx">tag</span><span class="p">],</span> <span class="nx">i</span> <span class="k">in</span> <span class="nx">@tokens</span> <span class="k">when</span> <span class="nx">tag</span> <span class="o">isnt</span> <span class="s">'TERMINATOR'</span>
|
|
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">i</span> <span class="k">if</span> <span class="nx">i</span></pre></div> </td> </tr> <tr id="section-7"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-7">¶</a> </div> <p>Some blocks occur in the middle of expressions -- when we're expecting
|
|
this, remove their trailing newlines.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">removeMidExpressionNewlines: </span><span class="nf">-></span>
|
|
<span class="nx">@scanTokens</span> <span class="nf">(token, i, tokens) -></span>
|
|
<span class="k">return</span> <span class="mi">1</span> <span class="k">unless</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">'TERMINATOR'</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span> <span class="k">in</span> <span class="nx">EXPRESSION_CLOSE</span>
|
|
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">1</span>
|
|
<span class="mi">0</span></pre></div> </td> </tr> <tr id="section-8"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-8">¶</a> </div> <p>The lexer has tagged the opening parenthesis of a method call. Match it with
|
|
its paired close. We have the mis-nested outdent case included here for
|
|
calls that close on the same line, just before their outdent.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">closeOpenCalls: </span><span class="nf">-></span>
|
|
|
|
<span class="nv">condition = </span><span class="nf">(token, i) -></span>
|
|
<span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">in</span> <span class="p">[</span><span class="s">')'</span><span class="p">,</span> <span class="s">'CALL_END'</span><span class="p">]</span> <span class="o">or</span>
|
|
<span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">'OUTDENT'</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">is</span> <span class="s">')'</span>
|
|
|
|
<span class="nv">action = </span><span class="nf">(token, i) -></span>
|
|
<span class="nx">@tokens</span><span class="p">[</span><span class="k">if</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">'OUTDENT'</span> <span class="k">then</span> <span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span> <span class="k">else</span> <span class="nx">i</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="s">'CALL_END'</span>
|
|
|
|
<span class="nx">@scanTokens</span> <span class="nf">(token, i) -></span>
|
|
<span class="nx">@detectEnd</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="nx">condition</span><span class="p">,</span> <span class="nx">action</span> <span class="k">if</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">'CALL_START'</span>
|
|
<span class="mi">1</span></pre></div> </td> </tr> <tr id="section-9"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-9">¶</a> </div> <p>The lexer has tagged the opening parenthesis of an indexing operation call.
|
|
Match it with its paired close.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">closeOpenIndexes: </span><span class="nf">-></span>
|
|
|
|
<span class="nv">condition = </span><span class="nf">(token, i) -></span>
|
|
<span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">in</span> <span class="p">[</span><span class="s">']'</span><span class="p">,</span> <span class="s">'INDEX_END'</span><span class="p">]</span>
|
|
|
|
<span class="nv">action = </span><span class="nf">(token, i) -></span>
|
|
<span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="s">'INDEX_END'</span>
|
|
|
|
<span class="nx">@scanTokens</span> <span class="nf">(token, i) -></span>
|
|
<span class="nx">@detectEnd</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="nx">condition</span><span class="p">,</span> <span class="nx">action</span> <span class="k">if</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">'INDEX_START'</span>
|
|
<span class="mi">1</span></pre></div> </td> </tr> <tr id="section-10"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-10">¶</a> </div> <p>Object literals may be written with implicit braces, for simple cases.
|
|
Insert the missing braces here, so that the parser doesn't have to.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">addImplicitBraces: </span><span class="nf">-></span>
|
|
|
|
<span class="nv">stack = </span><span class="p">[]</span>
|
|
<span class="nv">start = </span><span class="kc">null</span>
|
|
<span class="nv">startsLine = </span><span class="kc">null</span>
|
|
<span class="nv">sameLine = </span><span class="kc">yes</span>
|
|
<span class="nv">startIndent = </span><span class="mi">0</span>
|
|
<span class="nv">startIndex = </span><span class="mi">0</span>
|
|
|
|
<span class="nv">condition = </span><span class="nf">(token, i) -></span>
|
|
<span class="p">[</span><span class="nx">one</span><span class="p">,</span> <span class="nx">two</span><span class="p">,</span> <span class="nx">three</span><span class="p">]</span> <span class="o">=</span> <span class="nx">@tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span> <span class="p">..</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">3</span><span class="p">]</span>
|
|
<span class="k">return</span> <span class="kc">no</span> <span class="k">if</span> <span class="s">'HERECOMMENT'</span> <span class="o">is</span> <span class="nx">one</span><span class="o">?</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
|
<span class="p">[</span><span class="nx">tag</span><span class="p">]</span> <span class="o">=</span> <span class="nx">token</span>
|
|
<span class="nv">sameLine = </span><span class="kc">no</span> <span class="k">if</span> <span class="nx">tag</span> <span class="k">in</span> <span class="nx">LINEBREAKS</span>
|
|
<span class="k">return</span> <span class="p">(</span>
|
|
<span class="p">(</span><span class="nx">tag</span> <span class="k">in</span> <span class="p">[</span><span class="s">'TERMINATOR'</span><span class="p">,</span> <span class="s">'OUTDENT'</span><span class="p">]</span> <span class="o">or</span>
|
|
<span class="p">(</span><span class="nx">tag</span> <span class="k">in</span> <span class="nx">IMPLICIT_END</span> <span class="o">and</span> <span class="nx">sameLine</span> <span class="o">and</span> <span class="o">not</span> <span class="p">(</span><span class="nx">i</span> <span class="o">-</span> <span class="nx">startIndex</span> <span class="o">is</span> <span class="mi">1</span><span class="p">)))</span> <span class="o">and</span>
|
|
<span class="p">((</span><span class="o">!</span><span class="nx">startsLine</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">isnt</span> <span class="s">','</span><span class="p">)</span> <span class="o">or</span>
|
|
<span class="o">not</span> <span class="p">(</span><span class="nx">two</span><span class="o">?</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">':'</span> <span class="o">or</span> <span class="nx">one</span><span class="o">?</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">'@'</span> <span class="o">and</span> <span class="nx">three</span><span class="o">?</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">':'</span><span class="p">)))</span> <span class="o">or</span>
|
|
<span class="p">(</span><span class="nx">tag</span> <span class="o">is</span> <span class="s">','</span> <span class="o">and</span> <span class="nx">one</span> <span class="o">and</span>
|
|
<span class="nx">one</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">not</span> <span class="k">in</span> <span class="p">[</span><span class="s">'IDENTIFIER'</span><span class="p">,</span> <span class="s">'NUMBER'</span><span class="p">,</span> <span class="s">'STRING'</span><span class="p">,</span> <span class="s">'@'</span><span class="p">,</span> <span class="s">'TERMINATOR'</span><span class="p">,</span> <span class="s">'OUTDENT'</span><span class="p">]</span>
|
|
<span class="p">)</span>
|
|
|
|
<span class="nv">action = </span><span class="nf">(token, i) -></span>
|
|
<span class="nv">tok = </span><span class="nx">@generate</span> <span class="s">'}'</span><span class="p">,</span> <span class="s">'}'</span><span class="p">,</span> <span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
|
|
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">tok</span>
|
|
|
|
<span class="nx">@scanTokens</span> <span class="nf">(token, i, tokens) -></span>
|
|
<span class="k">if</span> <span class="p">(</span><span class="nv">tag = </span><span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="k">in</span> <span class="nx">EXPRESSION_START</span>
|
|
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span> <span class="p">[(</span><span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">'INDENT'</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">is</span> <span class="s">'{'</span> <span class="k">then</span> <span class="s">'{'</span> <span class="k">else</span> <span class="nx">tag</span><span class="p">),</span> <span class="nx">i</span><span class="p">]</span>
|
|
<span class="k">return</span> <span class="mi">1</span>
|
|
<span class="k">if</span> <span class="nx">tag</span> <span class="k">in</span> <span class="nx">EXPRESSION_END</span>
|
|
<span class="nv">start = </span><span class="nx">stack</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
|
|
<span class="k">return</span> <span class="mi">1</span>
|
|
<span class="k">return</span> <span class="mi">1</span> <span class="k">unless</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">':'</span> <span class="o">and</span>
|
|
<span class="p">((</span><span class="nv">ago = </span><span class="nx">@tag</span> <span class="nx">i</span> <span class="o">-</span> <span class="mi">2</span><span class="p">)</span> <span class="o">is</span> <span class="s">':'</span> <span class="o">or</span> <span class="nx">stack</span><span class="p">[</span><span class="nx">stack</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span><span class="o">?</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">isnt</span> <span class="s">'{'</span><span class="p">)</span>
|
|
<span class="nv">sameLine = </span><span class="kc">yes</span>
|
|
<span class="nv">startIndex = </span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span>
|
|
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s">'{'</span><span class="p">]</span>
|
|
<span class="nv">idx = </span> <span class="k">if</span> <span class="nx">ago</span> <span class="o">is</span> <span class="s">'@'</span> <span class="k">then</span> <span class="nx">i</span> <span class="o">-</span> <span class="mi">2</span> <span class="k">else</span> <span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span>
|
|
<span class="nx">idx</span> <span class="o">-=</span> <span class="mi">2</span> <span class="k">while</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">idx</span> <span class="o">-</span> <span class="mi">2</span><span class="p">)</span> <span class="o">is</span> <span class="s">'HERECOMMENT'</span>
|
|
<span class="nv">prevTag = </span><span class="nx">@tag</span><span class="p">(</span><span class="nx">idx</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span>
|
|
<span class="nv">startsLine = </span><span class="o">not</span> <span class="nx">prevTag</span> <span class="o">or</span> <span class="p">(</span><span class="nx">prevTag</span> <span class="k">in</span> <span class="nx">LINEBREAKS</span><span class="p">)</span>
|
|
<span class="nv">value = </span><span class="k">new</span> <span class="nb">String</span><span class="p">(</span><span class="s">'{'</span><span class="p">)</span>
|
|
<span class="nv">value.generated = </span><span class="kc">yes</span>
|
|
<span class="nv">tok = </span><span class="nx">@generate</span> <span class="s">'{'</span><span class="p">,</span> <span class="nx">value</span><span class="p">,</span> <span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
|
|
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">idx</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">tok</span>
|
|
<span class="nx">@detectEnd</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">2</span><span class="p">,</span> <span class="nx">condition</span><span class="p">,</span> <span class="nx">action</span>
|
|
<span class="mi">2</span></pre></div> </td> </tr> <tr id="section-11"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-11">¶</a> </div> <p>Methods may be optionally called without parentheses, for simple cases.
|
|
Insert the implicit parentheses here, so that the parser doesn't have to
|
|
deal with them.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">addImplicitParentheses: </span><span class="nf">-></span>
|
|
|
|
<span class="nv">noCall = seenSingle = seenControl = </span><span class="kc">no</span>
|
|
|
|
<span class="nv">condition = </span><span class="nf">(token, i) -></span>
|
|
<span class="p">[</span><span class="nx">tag</span><span class="p">]</span> <span class="o">=</span> <span class="nx">token</span>
|
|
<span class="k">return</span> <span class="kc">yes</span> <span class="k">if</span> <span class="o">not</span> <span class="nx">seenSingle</span> <span class="o">and</span> <span class="nx">token</span><span class="p">.</span><span class="nx">fromThen</span>
|
|
<span class="nv">seenSingle = </span><span class="kc">yes</span> <span class="k">if</span> <span class="nx">tag</span> <span class="k">in</span> <span class="p">[</span><span class="s">'IF'</span><span class="p">,</span> <span class="s">'ELSE'</span><span class="p">,</span> <span class="s">'CATCH'</span><span class="p">,</span> <span class="s">'->'</span><span class="p">,</span> <span class="s">'=>'</span><span class="p">,</span> <span class="s">'CLASS'</span><span class="p">]</span>
|
|
<span class="nv">seenControl = </span><span class="kc">yes</span> <span class="k">if</span> <span class="nx">tag</span> <span class="k">in</span> <span class="p">[</span><span class="s">'IF'</span><span class="p">,</span> <span class="s">'ELSE'</span><span class="p">,</span> <span class="s">'SWITCH'</span><span class="p">,</span> <span class="s">'TRY'</span><span class="p">,</span> <span class="s">'='</span><span class="p">]</span>
|
|
<span class="k">return</span> <span class="kc">yes</span> <span class="k">if</span> <span class="nx">tag</span> <span class="k">in</span> <span class="p">[</span><span class="s">'.'</span><span class="p">,</span> <span class="s">'?.'</span><span class="p">,</span> <span class="s">'::'</span><span class="p">]</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">is</span> <span class="s">'OUTDENT'</span>
|
|
<span class="o">not</span> <span class="nx">token</span><span class="p">.</span><span class="nx">generated</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">isnt</span> <span class="s">','</span> <span class="o">and</span> <span class="p">(</span><span class="nx">tag</span> <span class="k">in</span> <span class="nx">IMPLICIT_END</span> <span class="o">or</span>
|
|
<span class="p">(</span><span class="nx">tag</span> <span class="o">is</span> <span class="s">'INDENT'</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">seenControl</span><span class="p">))</span> <span class="o">and</span>
|
|
<span class="p">(</span><span class="nx">tag</span> <span class="o">isnt</span> <span class="s">'INDENT'</span> <span class="o">or</span>
|
|
<span class="p">(</span><span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">2</span><span class="p">)</span> <span class="o">not</span> <span class="k">in</span> <span class="p">[</span><span class="s">'CLASS'</span><span class="p">,</span> <span class="s">'EXTENDS'</span><span class="p">]</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">not</span> <span class="k">in</span> <span class="nx">IMPLICIT_BLOCK</span> <span class="o">and</span>
|
|
<span class="o">not</span> <span class="p">((</span><span class="nv">post = </span><span class="nx">@tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">])</span> <span class="o">and</span> <span class="nx">post</span><span class="p">.</span><span class="nx">generated</span> <span class="o">and</span> <span class="nx">post</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">'{'</span><span class="p">)))</span>
|
|
|
|
<span class="nv">action = </span><span class="nf">(token, i) -></span>
|
|
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">@generate</span> <span class="s">'CALL_END'</span><span class="p">,</span> <span class="s">')'</span><span class="p">,</span> <span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
|
|
|
|
<span class="nx">@scanTokens</span> <span class="nf">(token, i, tokens) -></span>
|
|
<span class="nv">tag = </span><span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
|
<span class="nv">noCall = </span><span class="kc">yes</span> <span class="k">if</span> <span class="nx">tag</span> <span class="k">in</span> <span class="p">[</span><span class="s">'CLASS'</span><span class="p">,</span> <span class="s">'IF'</span><span class="p">,</span> <span class="s">'FOR'</span><span class="p">,</span> <span class="s">'WHILE'</span><span class="p">]</span>
|
|
<span class="p">[</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">current</span><span class="p">,</span> <span class="nx">next</span><span class="p">]</span> <span class="o">=</span> <span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span> <span class="p">..</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">]</span>
|
|
<span class="nv">callObject = </span><span class="o">not</span> <span class="nx">noCall</span> <span class="o">and</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">'INDENT'</span> <span class="o">and</span>
|
|
<span class="nx">next</span> <span class="o">and</span> <span class="nx">next</span><span class="p">.</span><span class="nx">generated</span> <span class="o">and</span> <span class="nx">next</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">'{'</span> <span class="o">and</span>
|
|
<span class="nx">prev</span> <span class="o">and</span> <span class="nx">prev</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">in</span> <span class="nx">IMPLICIT_FUNC</span>
|
|
<span class="nv">seenSingle = </span><span class="kc">no</span>
|
|
<span class="nv">seenControl = </span><span class="kc">no</span>
|
|
<span class="nv">noCall = </span><span class="kc">no</span> <span class="k">if</span> <span class="nx">tag</span> <span class="k">in</span> <span class="nx">LINEBREAKS</span>
|
|
<span class="nv">token.call = </span><span class="kc">yes</span> <span class="k">if</span> <span class="nx">prev</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">prev</span><span class="p">.</span><span class="nx">spaced</span> <span class="o">and</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">'?'</span>
|
|
<span class="k">return</span> <span class="mi">1</span> <span class="k">if</span> <span class="nx">token</span><span class="p">.</span><span class="nx">fromThen</span>
|
|
<span class="k">return</span> <span class="mi">1</span> <span class="k">unless</span> <span class="nx">callObject</span> <span class="o">or</span>
|
|
<span class="nx">prev</span><span class="o">?</span><span class="p">.</span><span class="nx">spaced</span> <span class="o">and</span> <span class="p">(</span><span class="nx">prev</span><span class="p">.</span><span class="nx">call</span> <span class="o">or</span> <span class="nx">prev</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">in</span> <span class="nx">IMPLICIT_FUNC</span><span class="p">)</span> <span class="o">and</span>
|
|
<span class="p">(</span><span class="nx">tag</span> <span class="k">in</span> <span class="nx">IMPLICIT_CALL</span> <span class="o">or</span> <span class="o">not</span> <span class="p">(</span><span class="nx">token</span><span class="p">.</span><span class="nx">spaced</span> <span class="o">or</span> <span class="nx">token</span><span class="p">.</span><span class="nx">newLine</span><span class="p">)</span> <span class="o">and</span> <span class="nx">tag</span> <span class="k">in</span> <span class="nx">IMPLICIT_UNSPACED_CALL</span><span class="p">)</span>
|
|
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">@generate</span> <span class="s">'CALL_START'</span><span class="p">,</span> <span class="s">'('</span><span class="p">,</span> <span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
|
|
<span class="nx">@detectEnd</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="nx">condition</span><span class="p">,</span> <span class="nx">action</span>
|
|
<span class="nx">prev</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="s">'FUNC_EXIST'</span> <span class="k">if</span> <span class="nx">prev</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">'?'</span>
|
|
<span class="mi">2</span></pre></div> </td> </tr> <tr id="section-12"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-12">¶</a> </div> <p>Because our grammar is LALR(1), it can't handle some single-line
|
|
expressions that lack ending delimiters. The <strong>Rewriter</strong> adds the implicit
|
|
blocks, so it doesn't need to. ')' can close a single-line block,
|
|
but we need to make sure it's balanced.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">addImplicitIndentation: </span><span class="nf">-></span>
|
|
|
|
<span class="nv">starter = indent = outdent = </span><span class="kc">null</span>
|
|
|
|
<span class="nv">condition = </span><span class="nf">(token, i) -></span>
|
|
<span class="nx">token</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">isnt</span> <span class="s">';'</span> <span class="o">and</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">in</span> <span class="nx">SINGLE_CLOSERS</span> <span class="o">and</span>
|
|
<span class="o">not</span> <span class="p">(</span><span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">'ELSE'</span> <span class="o">and</span> <span class="nx">starter</span> <span class="o">not</span> <span class="k">in</span> <span class="p">[</span><span class="s">'IF'</span><span class="p">,</span> <span class="s">'THEN'</span><span class="p">])</span>
|
|
|
|
<span class="nv">action = </span><span class="nf">(token, i) -></span>
|
|
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="p">(</span><span class="k">if</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">is</span> <span class="s">','</span> <span class="k">then</span> <span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span> <span class="k">else</span> <span class="nx">i</span><span class="p">),</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">outdent</span>
|
|
|
|
<span class="nx">@scanTokens</span> <span class="nf">(token, i, tokens) -></span>
|
|
<span class="p">[</span><span class="nx">tag</span><span class="p">]</span> <span class="o">=</span> <span class="nx">token</span>
|
|
<span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">'TERMINATOR'</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span> <span class="o">is</span> <span class="s">'THEN'</span>
|
|
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">1</span>
|
|
<span class="k">return</span> <span class="mi">0</span>
|
|
<span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">'ELSE'</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">isnt</span> <span class="s">'OUTDENT'</span>
|
|
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">@indentation</span><span class="p">(</span><span class="nx">token</span><span class="p">)...</span>
|
|
<span class="k">return</span> <span class="mi">2</span>
|
|
<span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">'CATCH'</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">2</span><span class="p">)</span> <span class="k">in</span> <span class="p">[</span><span class="s">'OUTDENT'</span><span class="p">,</span> <span class="s">'TERMINATOR'</span><span class="p">,</span> <span class="s">'FINALLY'</span><span class="p">]</span>
|
|
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">@indentation</span><span class="p">(</span><span class="nx">token</span><span class="p">)...</span>
|
|
<span class="k">return</span> <span class="mi">4</span>
|
|
<span class="k">if</span> <span class="nx">tag</span> <span class="k">in</span> <span class="nx">SINGLE_LINERS</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span> <span class="o">isnt</span> <span class="s">'INDENT'</span> <span class="o">and</span>
|
|
<span class="o">not</span> <span class="p">(</span><span class="nx">tag</span> <span class="o">is</span> <span class="s">'ELSE'</span> <span class="o">and</span> <span class="nx">@tag</span><span class="p">(</span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span> <span class="o">is</span> <span class="s">'IF'</span><span class="p">)</span>
|
|
<span class="nv">starter = </span><span class="nx">tag</span>
|
|
<span class="p">[</span><span class="nx">indent</span><span class="p">,</span> <span class="nx">outdent</span><span class="p">]</span> <span class="o">=</span> <span class="nx">@indentation</span> <span class="nx">token</span><span class="p">,</span> <span class="kc">yes</span>
|
|
<span class="nv">indent.fromThen = </span><span class="kc">true</span> <span class="k">if</span> <span class="nx">starter</span> <span class="o">is</span> <span class="s">'THEN'</span>
|
|
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">indent</span>
|
|
<span class="nx">@detectEnd</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">2</span><span class="p">,</span> <span class="nx">condition</span><span class="p">,</span> <span class="nx">action</span>
|
|
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span><span class="p">,</span> <span class="mi">1</span> <span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s">'THEN'</span>
|
|
<span class="k">return</span> <span class="mi">1</span>
|
|
<span class="k">return</span> <span class="mi">1</span></pre></div> </td> </tr> <tr id="section-13"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-13">¶</a> </div> <p>Tag postfix conditionals as such, so that we can parse them with a
|
|
different precedence.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tagPostfixConditionals: </span><span class="nf">-></span>
|
|
|
|
<span class="nv">original = </span><span class="kc">null</span>
|
|
|
|
<span class="nv">condition = </span><span class="nf">(token, i) -></span>
|
|
<span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">in</span> <span class="p">[</span><span class="s">'TERMINATOR'</span><span class="p">,</span> <span class="s">'INDENT'</span><span class="p">]</span>
|
|
|
|
<span class="nv">action = </span><span class="nf">(token, i) -></span>
|
|
<span class="k">if</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">isnt</span> <span class="s">'INDENT'</span> <span class="o">or</span> <span class="p">(</span><span class="nx">token</span><span class="p">.</span><span class="nx">generated</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">token</span><span class="p">.</span><span class="nx">fromThen</span><span class="p">)</span>
|
|
<span class="nx">original</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="s">'POST_'</span> <span class="o">+</span> <span class="nx">original</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
|
|
|
<span class="nx">@scanTokens</span> <span class="nf">(token, i) -></span>
|
|
<span class="k">return</span> <span class="mi">1</span> <span class="k">unless</span> <span class="nx">token</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s">'IF'</span>
|
|
<span class="nv">original = </span><span class="nx">token</span>
|
|
<span class="nx">@detectEnd</span> <span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="nx">condition</span><span class="p">,</span> <span class="nx">action</span>
|
|
<span class="mi">1</span></pre></div> </td> </tr> <tr id="section-14"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-14">¶</a> </div> <p>Generate the indentation tokens, based on another token on the same line.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">indentation: </span><span class="nf">(token, implicit = no) -></span>
|
|
<span class="nv">indent = </span><span class="p">[</span><span class="s">'INDENT'</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]]</span>
|
|
<span class="nv">outdent = </span><span class="p">[</span><span class="s">'OUTDENT'</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="nx">token</span><span class="p">[</span><span class="mi">2</span><span class="p">]]</span>
|
|
<span class="nv">indent.generated = outdent.generated = </span><span class="kc">yes</span> <span class="k">if</span> <span class="nx">implicit</span>
|
|
<span class="p">[</span><span class="nx">indent</span><span class="p">,</span> <span class="nx">outdent</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-15"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-15">¶</a> </div> <p>Create a generated token: one that exists due to a use of implicit syntax.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">generate: </span><span class="nf">(tag, value, line) -></span>
|
|
<span class="nv">tok = </span><span class="p">[</span><span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span><span class="p">,</span> <span class="nx">line</span><span class="p">]</span>
|
|
<span class="nv">tok.generated = </span><span class="kc">yes</span>
|
|
<span class="nx">tok</span></pre></div> </td> </tr> <tr id="section-16"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-16">¶</a> </div> <p>Look up a tag by token index.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tag: </span><span class="nf">(i) -></span> <span class="nx">@tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">]</span><span class="o">?</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-17"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-17">¶</a> </div> <h2>Constants</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-18"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-18">¶</a> </div> <p>List of the token pairs that must be balanced.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">BALANCED_PAIRS = </span><span class="p">[</span>
|
|
<span class="p">[</span><span class="s">'('</span><span class="p">,</span> <span class="s">')'</span><span class="p">]</span>
|
|
<span class="p">[</span><span class="s">'['</span><span class="p">,</span> <span class="s">']'</span><span class="p">]</span>
|
|
<span class="p">[</span><span class="s">'{'</span><span class="p">,</span> <span class="s">'}'</span><span class="p">]</span>
|
|
<span class="p">[</span><span class="s">'INDENT'</span><span class="p">,</span> <span class="s">'OUTDENT'</span><span class="p">],</span>
|
|
<span class="p">[</span><span class="s">'CALL_START'</span><span class="p">,</span> <span class="s">'CALL_END'</span><span class="p">]</span>
|
|
<span class="p">[</span><span class="s">'PARAM_START'</span><span class="p">,</span> <span class="s">'PARAM_END'</span><span class="p">]</span>
|
|
<span class="p">[</span><span class="s">'INDEX_START'</span><span class="p">,</span> <span class="s">'INDEX_END'</span><span class="p">]</span>
|
|
<span class="p">]</span></pre></div> </td> </tr> <tr id="section-19"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-19">¶</a> </div> <p>The inverse mappings of <code>BALANCED_PAIRS</code> we're trying to fix up, so we can
|
|
look things up from either end.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">exports.INVERSES = INVERSES = </span><span class="p">{}</span></pre></div> </td> </tr> <tr id="section-20"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-20">¶</a> </div> <p>The tokens that signal the start/end of a balanced pair.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">EXPRESSION_START = </span><span class="p">[]</span>
|
|
<span class="nv">EXPRESSION_END = </span><span class="p">[]</span>
|
|
|
|
<span class="k">for</span> <span class="p">[</span><span class="nx">left</span><span class="p">,</span> <span class="nx">rite</span><span class="p">]</span> <span class="k">in</span> <span class="nx">BALANCED_PAIRS</span>
|
|
<span class="nx">EXPRESSION_START</span><span class="p">.</span><span class="nx">push</span> <span class="nx">INVERSES</span><span class="p">[</span><span class="nx">rite</span><span class="p">]</span> <span class="o">=</span> <span class="nx">left</span>
|
|
<span class="nx">EXPRESSION_END</span> <span class="p">.</span><span class="nx">push</span> <span class="nx">INVERSES</span><span class="p">[</span><span class="nx">left</span><span class="p">]</span> <span class="o">=</span> <span class="nx">rite</span></pre></div> </td> </tr> <tr id="section-21"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-21">¶</a> </div> <p>Tokens that indicate the close of a clause of an expression.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">EXPRESSION_CLOSE = </span><span class="p">[</span><span class="s">'CATCH'</span><span class="p">,</span> <span class="s">'WHEN'</span><span class="p">,</span> <span class="s">'ELSE'</span><span class="p">,</span> <span class="s">'FINALLY'</span><span class="p">].</span><span class="nx">concat</span> <span class="nx">EXPRESSION_END</span></pre></div> </td> </tr> <tr id="section-22"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-22">¶</a> </div> <p>Tokens that, if followed by an <code>IMPLICIT_CALL</code>, indicate a function invocation.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">IMPLICIT_FUNC = </span><span class="p">[</span><span class="s">'IDENTIFIER'</span><span class="p">,</span> <span class="s">'SUPER'</span><span class="p">,</span> <span class="s">')'</span><span class="p">,</span> <span class="s">'CALL_END'</span><span class="p">,</span> <span class="s">']'</span><span class="p">,</span> <span class="s">'INDEX_END'</span><span class="p">,</span> <span class="s">'@'</span><span class="p">,</span> <span class="s">'THIS'</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-23"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-23">¶</a> </div> <p>If preceded by an <code>IMPLICIT_FUNC</code>, indicates a function invocation.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">IMPLICIT_CALL = </span><span class="p">[</span>
|
|
<span class="s">'IDENTIFIER'</span><span class="p">,</span> <span class="s">'NUMBER'</span><span class="p">,</span> <span class="s">'STRING'</span><span class="p">,</span> <span class="s">'JS'</span><span class="p">,</span> <span class="s">'REGEX'</span><span class="p">,</span> <span class="s">'NEW'</span><span class="p">,</span> <span class="s">'PARAM_START'</span><span class="p">,</span> <span class="s">'CLASS'</span>
|
|
<span class="s">'IF'</span><span class="p">,</span> <span class="s">'TRY'</span><span class="p">,</span> <span class="s">'SWITCH'</span><span class="p">,</span> <span class="s">'THIS'</span><span class="p">,</span> <span class="s">'BOOL'</span><span class="p">,</span> <span class="s">'NULL'</span><span class="p">,</span> <span class="s">'UNDEFINED'</span><span class="p">,</span> <span class="s">'UNARY'</span><span class="p">,</span> <span class="s">'SUPER'</span>
|
|
<span class="s">'@'</span><span class="p">,</span> <span class="s">'->'</span><span class="p">,</span> <span class="s">'=>'</span><span class="p">,</span> <span class="s">'['</span><span class="p">,</span> <span class="s">'('</span><span class="p">,</span> <span class="s">'{'</span><span class="p">,</span> <span class="s">'--'</span><span class="p">,</span> <span class="s">'++'</span>
|
|
<span class="p">]</span>
|
|
|
|
<span class="nv">IMPLICIT_UNSPACED_CALL = </span><span class="p">[</span><span class="s">'+'</span><span class="p">,</span> <span class="s">'-'</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-24"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-24">¶</a> </div> <p>Tokens indicating that the implicit call must enclose a block of expressions.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">IMPLICIT_BLOCK = </span><span class="p">[</span><span class="s">'->'</span><span class="p">,</span> <span class="s">'=>'</span><span class="p">,</span> <span class="s">'{'</span><span class="p">,</span> <span class="s">'['</span><span class="p">,</span> <span class="s">','</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-25"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-25">¶</a> </div> <p>Tokens that always mark the end of an implicit call for single-liners.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">IMPLICIT_END = </span><span class="p">[</span><span class="s">'POST_IF'</span><span class="p">,</span> <span class="s">'FOR'</span><span class="p">,</span> <span class="s">'WHILE'</span><span class="p">,</span> <span class="s">'UNTIL'</span><span class="p">,</span> <span class="s">'WHEN'</span><span class="p">,</span> <span class="s">'BY'</span><span class="p">,</span> <span class="s">'LOOP'</span><span class="p">,</span> <span class="s">'TERMINATOR'</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-26"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-26">¶</a> </div> <p>Single-line flavors of block expressions that have unclosed endings.
|
|
The grammar can't disambiguate them, so we insert the implicit indentation.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">SINGLE_LINERS = </span><span class="p">[</span><span class="s">'ELSE'</span><span class="p">,</span> <span class="s">'->'</span><span class="p">,</span> <span class="s">'=>'</span><span class="p">,</span> <span class="s">'TRY'</span><span class="p">,</span> <span class="s">'FINALLY'</span><span class="p">,</span> <span class="s">'THEN'</span><span class="p">]</span>
|
|
<span class="nv">SINGLE_CLOSERS = </span><span class="p">[</span><span class="s">'TERMINATOR'</span><span class="p">,</span> <span class="s">'CATCH'</span><span class="p">,</span> <span class="s">'FINALLY'</span><span class="p">,</span> <span class="s">'ELSE'</span><span class="p">,</span> <span class="s">'OUTDENT'</span><span class="p">,</span> <span class="s">'LEADING_WHEN'</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-27"> <td class="docs"> <div class="pilwrap"> <a class="pilcrow" href="#section-27">¶</a> </div> <p>Tokens that end a line.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">LINEBREAKS = </span><span class="p">[</span><span class="s">'TERMINATOR'</span><span class="p">,</span> <span class="s">'INDENT'</span><span class="p">,</span> <span class="s">'OUTDENT'</span><span class="p">]</span>
|
|
|
|
</pre></div> </td> </tr> </tbody> </table> </div> </body> </html> |