mirror of
https://github.com/jashkenas/coffeescript.git
synced 2022-11-09 12:23:24 -05:00
1212 lines
69 KiB
HTML
1212 lines
69 KiB
HTML
<!DOCTYPE html>
|
||
|
||
<html>
|
||
<head>
|
||
<title>rewriter.coffee</title>
|
||
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
|
||
<meta name="viewport" content="width=device-width, target-densitydpi=160dpi, initial-scale=1.0; maximum-scale=1.0; user-scalable=0;">
|
||
<link rel="stylesheet" media="all" href="docco.css" />
|
||
</head>
|
||
<body>
|
||
<div id="container">
|
||
<div id="background"></div>
|
||
|
||
<ul id="jump_to">
|
||
<li>
|
||
<a class="large" href="javascript:void(0);">Jump To …</a>
|
||
<a class="small" href="javascript:void(0);">+</a>
|
||
<div id="jump_wrapper">
|
||
<div id="jump_page_wrapper">
|
||
<div id="jump_page">
|
||
|
||
|
||
<a class="source" href="browser.html">
|
||
browser.coffee
|
||
</a>
|
||
|
||
|
||
<a class="source" href="cake.html">
|
||
cake.coffee
|
||
</a>
|
||
|
||
|
||
<a class="source" href="coffee-script.html">
|
||
coffee-script.coffee
|
||
</a>
|
||
|
||
|
||
<a class="source" href="command.html">
|
||
command.coffee
|
||
</a>
|
||
|
||
|
||
<a class="source" href="grammar.html">
|
||
grammar.coffee
|
||
</a>
|
||
|
||
|
||
<a class="source" href="helpers.html">
|
||
helpers.coffee
|
||
</a>
|
||
|
||
|
||
<a class="source" href="index.html">
|
||
index.coffee
|
||
</a>
|
||
|
||
|
||
<a class="source" href="lexer.html">
|
||
lexer.coffee
|
||
</a>
|
||
|
||
|
||
<a class="source" href="nodes.html">
|
||
nodes.coffee
|
||
</a>
|
||
|
||
|
||
<a class="source" href="optparse.html">
|
||
optparse.coffee
|
||
</a>
|
||
|
||
|
||
<a class="source" href="register.html">
|
||
register.coffee
|
||
</a>
|
||
|
||
|
||
<a class="source" href="repl.html">
|
||
repl.coffee
|
||
</a>
|
||
|
||
|
||
<a class="source" href="rewriter.html">
|
||
rewriter.coffee
|
||
</a>
|
||
|
||
|
||
<a class="source" href="scope.html">
|
||
scope.litcoffee
|
||
</a>
|
||
|
||
|
||
<a class="source" href="sourcemap.html">
|
||
sourcemap.litcoffee
|
||
</a>
|
||
|
||
</div>
|
||
</div>
|
||
</li>
|
||
</ul>
|
||
|
||
<ul class="sections">
|
||
|
||
<li id="title">
|
||
<div class="annotation">
|
||
<h1>rewriter.coffee</h1>
|
||
</div>
|
||
</li>
|
||
|
||
|
||
|
||
<li id="section-1">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-1">¶</a>
|
||
</div>
|
||
<p>The CoffeeScript language has a good deal of optional syntax, implicit syntax,
|
||
and shorthand syntax. This can greatly complicate a grammar and bloat
|
||
the resulting parse table. Instead of making the parser handle it all, we take
|
||
a series of passes over the token stream, using this <strong>Rewriter</strong> to convert
|
||
shorthand into the unambiguous long form, add implicit indentation and
|
||
parentheses, and generally clean things up.</p>
|
||
|
||
</div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-2">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-2">¶</a>
|
||
</div>
|
||
<p>Create a generated token: one that exists due to a use of implicit syntax.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre><span class="hljs-function"><span class="hljs-title">generate</span> = <span class="hljs-params">(tag, value, origin)</span> -></span>
|
||
tok = [tag, value]
|
||
tok.generated = <span class="hljs-literal">yes</span>
|
||
tok.origin = origin <span class="hljs-keyword">if</span> origin
|
||
tok</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-3">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-3">¶</a>
|
||
</div>
|
||
<p>The <strong>Rewriter</strong> class is used by the <a href="lexer.html">Lexer</a>, directly against
|
||
its internal array of tokens.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre><span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">exports</span>.<span class="hljs-title">Rewriter</span></span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-4">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-4">¶</a>
|
||
</div>
|
||
<p>Helpful snippet for debugging:</p>
|
||
<pre><code><span class="hljs-built_in">console</span>.log (t[<span class="hljs-number">0</span>] + <span class="hljs-string">'/'</span> + t[<span class="hljs-number">1</span>] <span class="hljs-keyword">for</span> t <span class="hljs-keyword">in</span> <span class="hljs-property">@tokens</span>).join <span class="hljs-string">' '</span>
|
||
</code></pre>
|
||
</div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-5">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-5">¶</a>
|
||
</div>
|
||
<p>Rewrite the token stream in multiple passes, one logical filter at
|
||
a time. This could certainly be changed into a single pass through the
|
||
stream, with a big ol’ efficient switch, but it’s much nicer to work with
|
||
like this. The order of these passes matters — indentation must be
|
||
corrected before implicit parentheses can be wrapped around blocks of code.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-attribute">rewrite</span>: <span class="hljs-function"><span class="hljs-params">(<span class="hljs-property">@tokens</span>)</span> -></span>
|
||
<span class="hljs-property">@removeLeadingNewlines</span>()
|
||
<span class="hljs-property">@closeOpenCalls</span>()
|
||
<span class="hljs-property">@closeOpenIndexes</span>()
|
||
<span class="hljs-property">@normalizeLines</span>()
|
||
<span class="hljs-property">@tagPostfixConditionals</span>()
|
||
<span class="hljs-property">@addImplicitBracesAndParens</span>()
|
||
<span class="hljs-property">@addLocationDataToGeneratedTokens</span>()
|
||
<span class="hljs-property">@tokens</span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-6">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-6">¶</a>
|
||
</div>
|
||
<p>Rewrite the token stream, looking one token ahead and behind.
|
||
Allow the return value of the block to tell us how many tokens to move
|
||
forwards (or backwards) in the stream, to make sure we don’t miss anything
|
||
as tokens are inserted and removed, and the stream changes length under
|
||
our feet.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-attribute">scanTokens</span>: <span class="hljs-function"><span class="hljs-params">(block)</span> -></span>
|
||
{tokens} = <span class="hljs-keyword">this</span>
|
||
i = <span class="hljs-number">0</span>
|
||
i += block.call <span class="hljs-keyword">this</span>, token, i, tokens <span class="hljs-keyword">while</span> token = tokens[i]
|
||
<span class="hljs-literal">true</span>
|
||
|
||
<span class="hljs-attribute">detectEnd</span>: <span class="hljs-function"><span class="hljs-params">(i, condition, action)</span> -></span>
|
||
{tokens} = <span class="hljs-keyword">this</span>
|
||
levels = <span class="hljs-number">0</span>
|
||
<span class="hljs-keyword">while</span> token = tokens[i]
|
||
<span class="hljs-keyword">return</span> action.call <span class="hljs-keyword">this</span>, token, i <span class="hljs-keyword">if</span> levels <span class="hljs-keyword">is</span> <span class="hljs-number">0</span> <span class="hljs-keyword">and</span> condition.call <span class="hljs-keyword">this</span>, token, i
|
||
<span class="hljs-keyword">return</span> action.call <span class="hljs-keyword">this</span>, token, i - <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> token <span class="hljs-keyword">or</span> levels < <span class="hljs-number">0</span>
|
||
<span class="hljs-keyword">if</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> EXPRESSION_START
|
||
levels += <span class="hljs-number">1</span>
|
||
<span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> EXPRESSION_END
|
||
levels -= <span class="hljs-number">1</span>
|
||
i += <span class="hljs-number">1</span>
|
||
i - <span class="hljs-number">1</span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-7">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-7">¶</a>
|
||
</div>
|
||
<p>Leading newlines would introduce an ambiguity in the grammar, so we
|
||
dispatch them here.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-attribute">removeLeadingNewlines</span>:<span class="hljs-function"> -></span>
|
||
<span class="hljs-keyword">break</span> <span class="hljs-keyword">for</span> [tag], i <span class="hljs-keyword">in</span> <span class="hljs-property">@tokens</span> <span class="hljs-keyword">when</span> tag <span class="hljs-keyword">isnt</span> <span class="hljs-string">'TERMINATOR'</span>
|
||
<span class="hljs-property">@tokens</span>.splice <span class="hljs-number">0</span>, i <span class="hljs-keyword">if</span> i</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-8">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-8">¶</a>
|
||
</div>
|
||
<p>The lexer has tagged the opening parenthesis of a method call. Match it with
|
||
its paired close. We have the mis-nested outdent case included here for
|
||
calls that close on the same line, just before their outdent.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-attribute">closeOpenCalls</span>:<span class="hljs-function"> -></span>
|
||
<span class="hljs-function"><span class="hljs-title">condition</span> = <span class="hljs-params">(token, i)</span> -></span>
|
||
token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> [<span class="hljs-string">')'</span>, <span class="hljs-string">'CALL_END'</span>] <span class="hljs-keyword">or</span>
|
||
token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'OUTDENT'</span> <span class="hljs-keyword">and</span> <span class="hljs-property">@tag</span>(i - <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">')'</span>
|
||
|
||
<span class="hljs-function"><span class="hljs-title">action</span> = <span class="hljs-params">(token, i)</span> -></span>
|
||
<span class="hljs-property">@tokens</span>[<span class="hljs-keyword">if</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'OUTDENT'</span> <span class="hljs-keyword">then</span> i - <span class="hljs-number">1</span> <span class="hljs-keyword">else</span> i][<span class="hljs-number">0</span>] = <span class="hljs-string">'CALL_END'</span>
|
||
|
||
<span class="hljs-property">@scanTokens</span> <span class="hljs-function"><span class="hljs-params">(token, i)</span> -></span>
|
||
<span class="hljs-property">@detectEnd</span> i + <span class="hljs-number">1</span>, condition, action <span class="hljs-keyword">if</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'CALL_START'</span>
|
||
<span class="hljs-number">1</span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-9">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-9">¶</a>
|
||
</div>
|
||
<p>The lexer has tagged the opening parenthesis of an indexing operation call.
|
||
Match it with its paired close.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-attribute">closeOpenIndexes</span>:<span class="hljs-function"> -></span>
|
||
<span class="hljs-function"><span class="hljs-title">condition</span> = <span class="hljs-params">(token, i)</span> -></span>
|
||
token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> [<span class="hljs-string">']'</span>, <span class="hljs-string">'INDEX_END'</span>]
|
||
|
||
<span class="hljs-function"><span class="hljs-title">action</span> = <span class="hljs-params">(token, i)</span> -></span>
|
||
token[<span class="hljs-number">0</span>] = <span class="hljs-string">'INDEX_END'</span>
|
||
|
||
<span class="hljs-property">@scanTokens</span> <span class="hljs-function"><span class="hljs-params">(token, i)</span> -></span>
|
||
<span class="hljs-property">@detectEnd</span> i + <span class="hljs-number">1</span>, condition, action <span class="hljs-keyword">if</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'INDEX_START'</span>
|
||
<span class="hljs-number">1</span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-10">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-10">¶</a>
|
||
</div>
|
||
<p>Match tags in token stream starting at <code>i</code> with <code>pattern</code>, skipping ‘HERECOMMENT’s.
|
||
<code>pattern</code> may consist of strings (equality), an array of strings (one of)
|
||
or null (wildcard). Returns the index of the match or -1 if no match.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-attribute">indexOfTag</span>: <span class="hljs-function"><span class="hljs-params">(i, pattern...)</span> -></span>
|
||
fuzz = <span class="hljs-number">0</span>
|
||
<span class="hljs-keyword">for</span> j <span class="hljs-keyword">in</span> [<span class="hljs-number">0</span> ... pattern.length]
|
||
fuzz += <span class="hljs-number">2</span> <span class="hljs-keyword">while</span> <span class="hljs-property">@tag</span>(i + j + fuzz) <span class="hljs-keyword">is</span> <span class="hljs-string">'HERECOMMENT'</span>
|
||
<span class="hljs-keyword">continue</span> <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> pattern[j]?
|
||
pattern[j] = [pattern[j]] <span class="hljs-keyword">if</span> <span class="hljs-keyword">typeof</span> pattern[j] <span class="hljs-keyword">is</span> <span class="hljs-string">'string'</span>
|
||
<span class="hljs-keyword">return</span> -<span class="hljs-number">1</span> <span class="hljs-keyword">if</span> <span class="hljs-property">@tag</span>(i + j + fuzz) <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> pattern[j]
|
||
i + j + fuzz - <span class="hljs-number">1</span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-11">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-11">¶</a>
|
||
</div>
|
||
<p>Returns <code>yes</code> if standing in front of something looking like
|
||
<code>@<x>:</code>, <code><x>:</code> or <code><EXPRESSION_START><x>...<EXPRESSION_END>:</code>,
|
||
skipping over ‘HERECOMMENT’s.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-attribute">looksObjectish</span>: <span class="hljs-function"><span class="hljs-params">(j)</span> -></span>
|
||
<span class="hljs-keyword">return</span> <span class="hljs-literal">yes</span> <span class="hljs-keyword">if</span> <span class="hljs-property">@indexOfTag</span>(j, <span class="hljs-string">'@'</span>, <span class="hljs-literal">null</span>, <span class="hljs-string">':'</span>) > -<span class="hljs-number">1</span> <span class="hljs-keyword">or</span> <span class="hljs-property">@indexOfTag</span>(j, <span class="hljs-literal">null</span>, <span class="hljs-string">':'</span>) > -<span class="hljs-number">1</span>
|
||
index = <span class="hljs-property">@indexOfTag</span>(j, EXPRESSION_START)
|
||
<span class="hljs-keyword">if</span> index > -<span class="hljs-number">1</span>
|
||
end = <span class="hljs-literal">null</span>
|
||
<span class="hljs-property">@detectEnd</span> index + <span class="hljs-number">1</span>, <span class="hljs-function"><span class="hljs-params">((token) -> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> EXPRESSION_END)</span>, <span class="hljs-params">((token, i) -> end = i)</span>
|
||
<span class="hljs-title">return</span> <span class="hljs-title">yes</span> <span class="hljs-title">if</span> @<span class="hljs-title">tag</span><span class="hljs-params">(end + <span class="hljs-number">1</span>)</span> <span class="hljs-title">is</span> ':'
|
||
<span class="hljs-title">no</span>
|
||
|
||
</span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-12">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-12">¶</a>
|
||
</div>
|
||
<p>Returns <code>yes</code> if current line of tokens contain an element of tags on same
|
||
expression level. Stop searching at LINEBREAKS or explicit start of
|
||
containing balanced expression.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-attribute">findTagsBackwards</span>: <span class="hljs-function"><span class="hljs-params">(i, tags)</span> -></span>
|
||
backStack = []
|
||
<span class="hljs-keyword">while</span> i >= <span class="hljs-number">0</span> <span class="hljs-keyword">and</span> (backStack.length <span class="hljs-keyword">or</span>
|
||
<span class="hljs-property">@tag</span>(i) <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> tags <span class="hljs-keyword">and</span>
|
||
(<span class="hljs-property">@tag</span>(i) <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> EXPRESSION_START <span class="hljs-keyword">or</span> <span class="hljs-property">@tokens</span>[i].generated) <span class="hljs-keyword">and</span>
|
||
<span class="hljs-property">@tag</span>(i) <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> LINEBREAKS)
|
||
backStack.push <span class="hljs-property">@tag</span>(i) <span class="hljs-keyword">if</span> <span class="hljs-property">@tag</span>(i) <span class="hljs-keyword">in</span> EXPRESSION_END
|
||
backStack.pop() <span class="hljs-keyword">if</span> <span class="hljs-property">@tag</span>(i) <span class="hljs-keyword">in</span> EXPRESSION_START <span class="hljs-keyword">and</span> backStack.length
|
||
i -= <span class="hljs-number">1</span>
|
||
<span class="hljs-property">@tag</span>(i) <span class="hljs-keyword">in</span> tags</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-13">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-13">¶</a>
|
||
</div>
|
||
<p>Look for signs of implicit calls and objects in the token stream and
|
||
add them.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-attribute">addImplicitBracesAndParens</span>:<span class="hljs-function"> -></span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-14">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-14">¶</a>
|
||
</div>
|
||
<p>Track current balancing depth (both implicit and explicit) on stack.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> stack = []
|
||
start = <span class="hljs-literal">null</span>
|
||
|
||
<span class="hljs-property">@scanTokens</span> <span class="hljs-function"><span class="hljs-params">(token, i, tokens)</span> -></span>
|
||
[tag] = token
|
||
[prevTag] = prevToken = <span class="hljs-keyword">if</span> i > <span class="hljs-number">0</span> <span class="hljs-keyword">then</span> tokens[i - <span class="hljs-number">1</span>] <span class="hljs-keyword">else</span> []
|
||
[nextTag] = <span class="hljs-keyword">if</span> i < tokens.length - <span class="hljs-number">1</span> <span class="hljs-keyword">then</span> tokens[i + <span class="hljs-number">1</span>] <span class="hljs-keyword">else</span> []
|
||
<span class="hljs-function"><span class="hljs-title">stackTop</span> = -></span> stack[stack.length - <span class="hljs-number">1</span>]
|
||
startIdx = i</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-15">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-15">¶</a>
|
||
</div>
|
||
<p>Helper function, used for keeping track of the number of tokens consumed
|
||
and spliced, when returning for getting a new token.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-function"><span class="hljs-title">forward</span> = <span class="hljs-params">(n)</span> -></span> i - startIdx + n</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-16">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-16">¶</a>
|
||
</div>
|
||
<p>Helper functions</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-function"><span class="hljs-title">inImplicit</span> = -></span> stackTop()?[<span class="hljs-number">2</span>]?.ours
|
||
<span class="hljs-function"><span class="hljs-title">inImplicitCall</span> = -></span> inImplicit() <span class="hljs-keyword">and</span> stackTop()?[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'('</span>
|
||
<span class="hljs-function"><span class="hljs-title">inImplicitObject</span> = -></span> inImplicit() <span class="hljs-keyword">and</span> stackTop()?[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'{'</span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-17">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-17">¶</a>
|
||
</div>
|
||
<p>Unclosed control statement inside implicit parens (like
|
||
class declaration or if-conditionals)</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-function"><span class="hljs-title">inImplicitControl</span> = -></span> inImplicit <span class="hljs-keyword">and</span> stackTop()?[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'CONTROL'</span>
|
||
|
||
<span class="hljs-function"><span class="hljs-title">startImplicitCall</span> = <span class="hljs-params">(j)</span> -></span>
|
||
idx = j ? i
|
||
stack.push [<span class="hljs-string">'('</span>, idx, <span class="hljs-attribute">ours</span>: <span class="hljs-literal">yes</span>]
|
||
tokens.splice idx, <span class="hljs-number">0</span>, generate <span class="hljs-string">'CALL_START'</span>, <span class="hljs-string">'('</span>
|
||
i += <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> j?
|
||
|
||
<span class="hljs-function"><span class="hljs-title">endImplicitCall</span> = -></span>
|
||
stack.pop()
|
||
tokens.splice i, <span class="hljs-number">0</span>, generate <span class="hljs-string">'CALL_END'</span>, <span class="hljs-string">')'</span>, [<span class="hljs-string">''</span>, <span class="hljs-string">'end of input'</span>, token[<span class="hljs-number">2</span>]]
|
||
i += <span class="hljs-number">1</span>
|
||
|
||
<span class="hljs-function"><span class="hljs-title">startImplicitObject</span> = <span class="hljs-params">(j, startsLine = <span class="hljs-literal">yes</span>)</span> -></span>
|
||
idx = j ? i
|
||
stack.push [<span class="hljs-string">'{'</span>, idx, <span class="hljs-attribute">sameLine</span>: <span class="hljs-literal">yes</span>, <span class="hljs-attribute">startsLine</span>: startsLine, <span class="hljs-attribute">ours</span>: <span class="hljs-literal">yes</span>]
|
||
val = <span class="hljs-keyword">new</span> String <span class="hljs-string">'{'</span>
|
||
val.generated = <span class="hljs-literal">yes</span>
|
||
tokens.splice idx, <span class="hljs-number">0</span>, generate <span class="hljs-string">'{'</span>, val, token
|
||
i += <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> j?
|
||
|
||
<span class="hljs-function"><span class="hljs-title">endImplicitObject</span> = <span class="hljs-params">(j)</span> -></span>
|
||
j = j ? i
|
||
stack.pop()
|
||
tokens.splice j, <span class="hljs-number">0</span>, generate <span class="hljs-string">'}'</span>, <span class="hljs-string">'}'</span>, token
|
||
i += <span class="hljs-number">1</span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-18">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-18">¶</a>
|
||
</div>
|
||
<p>Don’t end an implicit call on next indent if any of these are in an argument</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> inImplicitCall() <span class="hljs-keyword">and</span> tag <span class="hljs-keyword">in</span> [<span class="hljs-string">'IF'</span>, <span class="hljs-string">'TRY'</span>, <span class="hljs-string">'FINALLY'</span>, <span class="hljs-string">'CATCH'</span>,
|
||
<span class="hljs-string">'CLASS'</span>, <span class="hljs-string">'SWITCH'</span>]
|
||
stack.push [<span class="hljs-string">'CONTROL'</span>, i, <span class="hljs-attribute">ours</span>: <span class="hljs-literal">true</span>]
|
||
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">1</span>)
|
||
|
||
<span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">'INDENT'</span> <span class="hljs-keyword">and</span> inImplicit()</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-19">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-19">¶</a>
|
||
</div>
|
||
<p>An <code>INDENT</code> closes an implicit call unless</p>
|
||
<ol>
|
||
<li>We have seen a <code>CONTROL</code> argument on the line.</li>
|
||
<li>The last token before the indent is part of the list below</li>
|
||
</ol>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> prevTag <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> [<span class="hljs-string">'=>'</span>, <span class="hljs-string">'->'</span>, <span class="hljs-string">'['</span>, <span class="hljs-string">'('</span>, <span class="hljs-string">','</span>, <span class="hljs-string">'{'</span>, <span class="hljs-string">'TRY'</span>, <span class="hljs-string">'ELSE'</span>, <span class="hljs-string">'='</span>]
|
||
endImplicitCall() <span class="hljs-keyword">while</span> inImplicitCall()
|
||
stack.pop() <span class="hljs-keyword">if</span> inImplicitControl()
|
||
stack.push [tag, i]
|
||
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">1</span>)</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-20">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-20">¶</a>
|
||
</div>
|
||
<p>Straightforward start of explicit expression</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">in</span> EXPRESSION_START
|
||
stack.push [tag, i]
|
||
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">1</span>)</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-21">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-21">¶</a>
|
||
</div>
|
||
<p>Close all implicit expressions inside of explicitly closed expressions.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">in</span> EXPRESSION_END
|
||
<span class="hljs-keyword">while</span> inImplicit()
|
||
<span class="hljs-keyword">if</span> inImplicitCall()
|
||
endImplicitCall()
|
||
<span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> inImplicitObject()
|
||
endImplicitObject()
|
||
<span class="hljs-keyword">else</span>
|
||
stack.pop()
|
||
start = stack.pop()</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-22">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-22">¶</a>
|
||
</div>
|
||
<p>Recognize standard implicit calls like
|
||
f a, f() b, f? c, h[0] d etc.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> (tag <span class="hljs-keyword">in</span> IMPLICIT_FUNC <span class="hljs-keyword">and</span> token.spaced <span class="hljs-keyword">or</span>
|
||
tag <span class="hljs-keyword">is</span> <span class="hljs-string">'?'</span> <span class="hljs-keyword">and</span> i > <span class="hljs-number">0</span> <span class="hljs-keyword">and</span> <span class="hljs-keyword">not</span> tokens[i - <span class="hljs-number">1</span>].spaced) <span class="hljs-keyword">and</span>
|
||
(nextTag <span class="hljs-keyword">in</span> IMPLICIT_CALL <span class="hljs-keyword">or</span>
|
||
nextTag <span class="hljs-keyword">in</span> IMPLICIT_UNSPACED_CALL <span class="hljs-keyword">and</span>
|
||
<span class="hljs-keyword">not</span> tokens[i + <span class="hljs-number">1</span>]?.spaced <span class="hljs-keyword">and</span> <span class="hljs-keyword">not</span> tokens[i + <span class="hljs-number">1</span>]?.newLine)
|
||
tag = token[<span class="hljs-number">0</span>] = <span class="hljs-string">'FUNC_EXIST'</span> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">'?'</span>
|
||
startImplicitCall i + <span class="hljs-number">1</span>
|
||
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">2</span>)</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-23">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-23">¶</a>
|
||
</div>
|
||
<p>Implicit call taking an implicit indented object as first argument.</p>
|
||
<pre><code>f
|
||
<span class="hljs-attribute">a</span>: b
|
||
<span class="hljs-attribute">c</span>: d
|
||
</code></pre><p>and</p>
|
||
<pre><code>f
|
||
<span class="hljs-number">1</span>
|
||
<span class="hljs-attribute">a</span>: b
|
||
<span class="hljs-attribute">b</span>: c
|
||
</code></pre><p>Don’t accept implicit calls of this type, when on the same line
|
||
as the control strucutures below as that may misinterpret constructs like:</p>
|
||
<pre><code><span class="hljs-keyword">if</span> f
|
||
<span class="hljs-attribute">a</span>: <span class="hljs-number">1</span>
|
||
</code></pre><p>as</p>
|
||
<pre><code><span class="hljs-keyword">if</span> f(<span class="hljs-attribute">a</span>: <span class="hljs-number">1</span>)
|
||
</code></pre><p>which is probably always unintended.
|
||
Furthermore don’t allow this in literal arrays, as
|
||
that creates grammatical ambiguities.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">in</span> IMPLICIT_FUNC <span class="hljs-keyword">and</span>
|
||
<span class="hljs-property">@indexOfTag</span>(i + <span class="hljs-number">1</span>, <span class="hljs-string">'INDENT'</span>, <span class="hljs-literal">null</span>, <span class="hljs-string">':'</span>) > -<span class="hljs-number">1</span> <span class="hljs-keyword">and</span>
|
||
<span class="hljs-keyword">not</span> <span class="hljs-property">@findTagsBackwards</span>(i, [<span class="hljs-string">'CLASS'</span>, <span class="hljs-string">'EXTENDS'</span>, <span class="hljs-string">'IF'</span>, <span class="hljs-string">'CATCH'</span>,
|
||
<span class="hljs-string">'SWITCH'</span>, <span class="hljs-string">'LEADING_WHEN'</span>, <span class="hljs-string">'FOR'</span>, <span class="hljs-string">'WHILE'</span>, <span class="hljs-string">'UNTIL'</span>])
|
||
startImplicitCall i + <span class="hljs-number">1</span>
|
||
stack.push [<span class="hljs-string">'INDENT'</span>, i + <span class="hljs-number">2</span>]
|
||
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">3</span>)</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-24">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-24">¶</a>
|
||
</div>
|
||
<p>Implicit objects start here</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">':'</span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-25">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-25">¶</a>
|
||
</div>
|
||
<p>Go back to the (implicit) start of the object</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> s = <span class="hljs-keyword">switch</span>
|
||
<span class="hljs-keyword">when</span> <span class="hljs-property">@tag</span>(i - <span class="hljs-number">1</span>) <span class="hljs-keyword">in</span> EXPRESSION_END <span class="hljs-keyword">then</span> start[<span class="hljs-number">1</span>]
|
||
<span class="hljs-keyword">when</span> <span class="hljs-property">@tag</span>(i - <span class="hljs-number">2</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">'@'</span> <span class="hljs-keyword">then</span> i - <span class="hljs-number">2</span>
|
||
<span class="hljs-keyword">else</span> i - <span class="hljs-number">1</span>
|
||
s -= <span class="hljs-number">2</span> <span class="hljs-keyword">while</span> <span class="hljs-property">@tag</span>(s - <span class="hljs-number">2</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">'HERECOMMENT'</span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-26">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-26">¶</a>
|
||
</div>
|
||
<p>Mark if the value is a for loop</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-property">@insideForDeclaration</span> = nextTag <span class="hljs-keyword">is</span> <span class="hljs-string">'FOR'</span>
|
||
|
||
startsLine = s <span class="hljs-keyword">is</span> <span class="hljs-number">0</span> <span class="hljs-keyword">or</span> <span class="hljs-property">@tag</span>(s - <span class="hljs-number">1</span>) <span class="hljs-keyword">in</span> LINEBREAKS <span class="hljs-keyword">or</span> tokens[s - <span class="hljs-number">1</span>].newLine</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-27">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-27">¶</a>
|
||
</div>
|
||
<p>Are we just continuing an already declared object?</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> stackTop()
|
||
[stackTag, stackIdx] = stackTop()
|
||
<span class="hljs-keyword">if</span> (stackTag <span class="hljs-keyword">is</span> <span class="hljs-string">'{'</span> <span class="hljs-keyword">or</span> stackTag <span class="hljs-keyword">is</span> <span class="hljs-string">'INDENT'</span> <span class="hljs-keyword">and</span> <span class="hljs-property">@tag</span>(stackIdx - <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">'{'</span>) <span class="hljs-keyword">and</span>
|
||
(startsLine <span class="hljs-keyword">or</span> <span class="hljs-property">@tag</span>(s - <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">','</span> <span class="hljs-keyword">or</span> <span class="hljs-property">@tag</span>(s - <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">'{'</span>)
|
||
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">1</span>)
|
||
|
||
startImplicitObject(s, !!startsLine)
|
||
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">2</span>)</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-28">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-28">¶</a>
|
||
</div>
|
||
<p>End implicit calls when chaining method calls
|
||
like e.g.:</p>
|
||
<pre><code>f<span class="hljs-function"> -></span>
|
||
a
|
||
.g b,<span class="hljs-function"> -></span>
|
||
c
|
||
.h a
|
||
</code></pre><p>and also</p>
|
||
<pre><code>f a
|
||
.g b
|
||
.h a
|
||
</code></pre>
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre>
|
||
stackTop()[<span class="hljs-number">2</span>].sameLine = <span class="hljs-literal">no</span> <span class="hljs-keyword">if</span> inImplicitObject() <span class="hljs-keyword">and</span> tag <span class="hljs-keyword">in</span> LINEBREAKS
|
||
|
||
newLine = prevTag <span class="hljs-keyword">is</span> <span class="hljs-string">'OUTDENT'</span> <span class="hljs-keyword">or</span> prevToken.newLine
|
||
<span class="hljs-keyword">if</span> tag <span class="hljs-keyword">in</span> IMPLICIT_END <span class="hljs-keyword">or</span> tag <span class="hljs-keyword">in</span> CALL_CLOSERS <span class="hljs-keyword">and</span> newLine
|
||
<span class="hljs-keyword">while</span> inImplicit()
|
||
[stackTag, stackIdx, {sameLine, startsLine}] = stackTop()</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-29">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-29">¶</a>
|
||
</div>
|
||
<p>Close implicit calls when reached end of argument list</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> inImplicitCall() <span class="hljs-keyword">and</span> prevTag <span class="hljs-keyword">isnt</span> <span class="hljs-string">','</span>
|
||
endImplicitCall()</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-30">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-30">¶</a>
|
||
</div>
|
||
<p>Close implicit objects such as:
|
||
return a: 1, b: 2 unless true</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> inImplicitObject() <span class="hljs-keyword">and</span> <span class="hljs-keyword">not</span> <span class="hljs-property">@insideForDeclaration</span> <span class="hljs-keyword">and</span> sameLine <span class="hljs-keyword">and</span>
|
||
tag <span class="hljs-keyword">isnt</span> <span class="hljs-string">'TERMINATOR'</span> <span class="hljs-keyword">and</span> prevTag <span class="hljs-keyword">isnt</span> <span class="hljs-string">':'</span>
|
||
endImplicitObject()</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-31">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-31">¶</a>
|
||
</div>
|
||
<p>Close implicit objects when at end of line, line didn’t end with a comma
|
||
and the implicit object didn’t start the line or the next line doesn’t look like
|
||
the continuation of an object.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> inImplicitObject() <span class="hljs-keyword">and</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">'TERMINATOR'</span> <span class="hljs-keyword">and</span> prevTag <span class="hljs-keyword">isnt</span> <span class="hljs-string">','</span> <span class="hljs-keyword">and</span>
|
||
<span class="hljs-keyword">not</span> (startsLine <span class="hljs-keyword">and</span> <span class="hljs-property">@looksObjectish</span>(i + <span class="hljs-number">1</span>))
|
||
<span class="hljs-keyword">return</span> forward <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> nextTag <span class="hljs-keyword">is</span> <span class="hljs-string">'HERECOMMENT'</span>
|
||
endImplicitObject()
|
||
<span class="hljs-keyword">else</span>
|
||
<span class="hljs-keyword">break</span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-32">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-32">¶</a>
|
||
</div>
|
||
<p>Close implicit object if comma is the last character
|
||
and what comes after doesn’t look like it belongs.
|
||
This is used for trailing commas and calls, like:</p>
|
||
<pre><code>x =
|
||
<span class="hljs-attribute">a</span>: b,
|
||
<span class="hljs-attribute">c</span>: d,
|
||
e = <span class="hljs-number">2</span>
|
||
</code></pre><p>and</p>
|
||
<pre><code>f a, <span class="hljs-attribute">b</span>: c, <span class="hljs-attribute">d</span>: e, f, <span class="hljs-attribute">g</span>: <span class="hljs-attribute">h</span>: i, j
|
||
</code></pre>
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">','</span> <span class="hljs-keyword">and</span> <span class="hljs-keyword">not</span> <span class="hljs-property">@looksObjectish</span>(i + <span class="hljs-number">1</span>) <span class="hljs-keyword">and</span> inImplicitObject() <span class="hljs-keyword">and</span>
|
||
<span class="hljs-keyword">not</span> <span class="hljs-property">@insideForDeclaration</span> <span class="hljs-keyword">and</span>
|
||
(nextTag <span class="hljs-keyword">isnt</span> <span class="hljs-string">'TERMINATOR'</span> <span class="hljs-keyword">or</span> <span class="hljs-keyword">not</span> <span class="hljs-property">@looksObjectish</span>(i + <span class="hljs-number">2</span>))</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-33">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-33">¶</a>
|
||
</div>
|
||
<p>When nextTag is OUTDENT the comma is insignificant and
|
||
should just be ignored so embed it in the implicit object.</p>
|
||
<p>When it isn’t the comma go on to play a role in a call or
|
||
array further up the stack, so give it a chance.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre>
|
||
offset = <span class="hljs-keyword">if</span> nextTag <span class="hljs-keyword">is</span> <span class="hljs-string">'OUTDENT'</span> <span class="hljs-keyword">then</span> <span class="hljs-number">1</span> <span class="hljs-keyword">else</span> <span class="hljs-number">0</span>
|
||
<span class="hljs-keyword">while</span> inImplicitObject()
|
||
endImplicitObject i + offset
|
||
<span class="hljs-keyword">return</span> forward(<span class="hljs-number">1</span>)</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-34">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-34">¶</a>
|
||
</div>
|
||
<p>Add location data to all tokens generated by the rewriter.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-attribute">addLocationDataToGeneratedTokens</span>:<span class="hljs-function"> -></span>
|
||
<span class="hljs-property">@scanTokens</span> <span class="hljs-function"><span class="hljs-params">(token, i, tokens)</span> -></span>
|
||
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> token[<span class="hljs-number">2</span>]
|
||
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span> <span class="hljs-keyword">unless</span> token.generated <span class="hljs-keyword">or</span> token.explicit
|
||
<span class="hljs-keyword">if</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'{'</span> <span class="hljs-keyword">and</span> nextLocation=tokens[i + <span class="hljs-number">1</span>]?[<span class="hljs-number">2</span>]
|
||
{<span class="hljs-attribute">first_line</span>: line, <span class="hljs-attribute">first_column</span>: column} = nextLocation
|
||
<span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> prevLocation = tokens[i - <span class="hljs-number">1</span>]?[<span class="hljs-number">2</span>]
|
||
{<span class="hljs-attribute">last_line</span>: line, <span class="hljs-attribute">last_column</span>: column} = prevLocation
|
||
<span class="hljs-keyword">else</span>
|
||
line = column = <span class="hljs-number">0</span>
|
||
token[<span class="hljs-number">2</span>] =
|
||
<span class="hljs-attribute">first_line</span>: line
|
||
<span class="hljs-attribute">first_column</span>: column
|
||
<span class="hljs-attribute">last_line</span>: line
|
||
<span class="hljs-attribute">last_column</span>: column
|
||
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-35">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-35">¶</a>
|
||
</div>
|
||
<p>Because our grammar is LALR(1), it can’t handle some single-line
|
||
expressions that lack ending delimiters. The <strong>Rewriter</strong> adds the implicit
|
||
blocks, so it doesn’t need to. To keep the grammar clean and tidy, trailing
|
||
newlines within expressions are removed and the indentation tokens of empty
|
||
blocks are added.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-attribute">normalizeLines</span>:<span class="hljs-function"> -></span>
|
||
starter = indent = outdent = <span class="hljs-literal">null</span>
|
||
|
||
<span class="hljs-function"><span class="hljs-title">condition</span> = <span class="hljs-params">(token, i)</span> -></span>
|
||
token[<span class="hljs-number">1</span>] <span class="hljs-keyword">isnt</span> <span class="hljs-string">';'</span> <span class="hljs-keyword">and</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> SINGLE_CLOSERS <span class="hljs-keyword">and</span>
|
||
<span class="hljs-keyword">not</span> (token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'TERMINATOR'</span> <span class="hljs-keyword">and</span> <span class="hljs-property">@tag</span>(i + <span class="hljs-number">1</span>) <span class="hljs-keyword">in</span> EXPRESSION_CLOSE) <span class="hljs-keyword">and</span>
|
||
<span class="hljs-keyword">not</span> (token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'ELSE'</span> <span class="hljs-keyword">and</span> starter <span class="hljs-keyword">isnt</span> <span class="hljs-string">'THEN'</span>) <span class="hljs-keyword">and</span>
|
||
<span class="hljs-keyword">not</span> (token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> [<span class="hljs-string">'CATCH'</span>, <span class="hljs-string">'FINALLY'</span>] <span class="hljs-keyword">and</span> starter <span class="hljs-keyword">in</span> [<span class="hljs-string">'->'</span>, <span class="hljs-string">'=>'</span>]) <span class="hljs-keyword">or</span>
|
||
token[<span class="hljs-number">0</span>] <span class="hljs-keyword">in</span> CALL_CLOSERS <span class="hljs-keyword">and</span> <span class="hljs-property">@tokens</span>[i - <span class="hljs-number">1</span>].newLine
|
||
|
||
<span class="hljs-function"><span class="hljs-title">action</span> = <span class="hljs-params">(token, i)</span> -></span>
|
||
<span class="hljs-property">@tokens</span>.splice (<span class="hljs-keyword">if</span> <span class="hljs-property">@tag</span>(i - <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">','</span> <span class="hljs-keyword">then</span> i - <span class="hljs-number">1</span> <span class="hljs-keyword">else</span> i), <span class="hljs-number">0</span>, outdent
|
||
|
||
<span class="hljs-property">@scanTokens</span> <span class="hljs-function"><span class="hljs-params">(token, i, tokens)</span> -></span>
|
||
[tag] = token
|
||
<span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">'TERMINATOR'</span>
|
||
<span class="hljs-keyword">if</span> <span class="hljs-property">@tag</span>(i + <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">'ELSE'</span> <span class="hljs-keyword">and</span> <span class="hljs-property">@tag</span>(i - <span class="hljs-number">1</span>) <span class="hljs-keyword">isnt</span> <span class="hljs-string">'OUTDENT'</span>
|
||
tokens.splice i, <span class="hljs-number">1</span>, <span class="hljs-property">@indentation</span>()...
|
||
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span>
|
||
<span class="hljs-keyword">if</span> <span class="hljs-property">@tag</span>(i + <span class="hljs-number">1</span>) <span class="hljs-keyword">in</span> EXPRESSION_CLOSE
|
||
tokens.splice i, <span class="hljs-number">1</span>
|
||
<span class="hljs-keyword">return</span> <span class="hljs-number">0</span>
|
||
<span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">'CATCH'</span>
|
||
<span class="hljs-keyword">for</span> j <span class="hljs-keyword">in</span> [<span class="hljs-number">1.</span><span class="hljs-number">.2</span>] <span class="hljs-keyword">when</span> <span class="hljs-property">@tag</span>(i + j) <span class="hljs-keyword">in</span> [<span class="hljs-string">'OUTDENT'</span>, <span class="hljs-string">'TERMINATOR'</span>, <span class="hljs-string">'FINALLY'</span>]
|
||
tokens.splice i + j, <span class="hljs-number">0</span>, <span class="hljs-property">@indentation</span>()...
|
||
<span class="hljs-keyword">return</span> <span class="hljs-number">2</span> + j
|
||
<span class="hljs-keyword">if</span> tag <span class="hljs-keyword">in</span> SINGLE_LINERS <span class="hljs-keyword">and</span> <span class="hljs-property">@tag</span>(i + <span class="hljs-number">1</span>) <span class="hljs-keyword">isnt</span> <span class="hljs-string">'INDENT'</span> <span class="hljs-keyword">and</span>
|
||
<span class="hljs-keyword">not</span> (tag <span class="hljs-keyword">is</span> <span class="hljs-string">'ELSE'</span> <span class="hljs-keyword">and</span> <span class="hljs-property">@tag</span>(i + <span class="hljs-number">1</span>) <span class="hljs-keyword">is</span> <span class="hljs-string">'IF'</span>)
|
||
starter = tag
|
||
[indent, outdent] = <span class="hljs-property">@indentation</span> tokens[i]
|
||
indent.fromThen = <span class="hljs-literal">true</span> <span class="hljs-keyword">if</span> starter <span class="hljs-keyword">is</span> <span class="hljs-string">'THEN'</span>
|
||
tokens.splice i + <span class="hljs-number">1</span>, <span class="hljs-number">0</span>, indent
|
||
<span class="hljs-property">@detectEnd</span> i + <span class="hljs-number">2</span>, condition, action
|
||
tokens.splice i, <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> tag <span class="hljs-keyword">is</span> <span class="hljs-string">'THEN'</span>
|
||
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span>
|
||
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-36">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-36">¶</a>
|
||
</div>
|
||
<p>Tag postfix conditionals as such, so that we can parse them with a
|
||
different precedence.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-attribute">tagPostfixConditionals</span>:<span class="hljs-function"> -></span>
|
||
|
||
original = <span class="hljs-literal">null</span>
|
||
|
||
<span class="hljs-function"><span class="hljs-title">condition</span> = <span class="hljs-params">(token, i)</span> -></span>
|
||
[tag] = token
|
||
[prevTag] = <span class="hljs-property">@tokens</span>[i - <span class="hljs-number">1</span>]
|
||
tag <span class="hljs-keyword">is</span> <span class="hljs-string">'TERMINATOR'</span> <span class="hljs-keyword">or</span> (tag <span class="hljs-keyword">is</span> <span class="hljs-string">'INDENT'</span> <span class="hljs-keyword">and</span> prevTag <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> SINGLE_LINERS)
|
||
|
||
<span class="hljs-function"><span class="hljs-title">action</span> = <span class="hljs-params">(token, i)</span> -></span>
|
||
<span class="hljs-keyword">if</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">isnt</span> <span class="hljs-string">'INDENT'</span> <span class="hljs-keyword">or</span> (token.generated <span class="hljs-keyword">and</span> <span class="hljs-keyword">not</span> token.fromThen)
|
||
original[<span class="hljs-number">0</span>] = <span class="hljs-string">'POST_'</span> + original[<span class="hljs-number">0</span>]
|
||
|
||
<span class="hljs-property">@scanTokens</span> <span class="hljs-function"><span class="hljs-params">(token, i)</span> -></span>
|
||
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span> <span class="hljs-keyword">unless</span> token[<span class="hljs-number">0</span>] <span class="hljs-keyword">is</span> <span class="hljs-string">'IF'</span>
|
||
original = token
|
||
<span class="hljs-property">@detectEnd</span> i + <span class="hljs-number">1</span>, condition, action
|
||
<span class="hljs-keyword">return</span> <span class="hljs-number">1</span></pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-37">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-37">¶</a>
|
||
</div>
|
||
<p>Generate the indentation tokens, based on another token on the same line.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-attribute">indentation</span>: <span class="hljs-function"><span class="hljs-params">(origin)</span> -></span>
|
||
indent = [<span class="hljs-string">'INDENT'</span>, <span class="hljs-number">2</span>]
|
||
outdent = [<span class="hljs-string">'OUTDENT'</span>, <span class="hljs-number">2</span>]
|
||
<span class="hljs-keyword">if</span> origin
|
||
indent.generated = outdent.generated = <span class="hljs-literal">yes</span>
|
||
indent.origin = outdent.origin = origin
|
||
<span class="hljs-keyword">else</span>
|
||
indent.explicit = outdent.explicit = <span class="hljs-literal">yes</span>
|
||
[indent, outdent]
|
||
|
||
<span class="hljs-attribute">generate</span>: generate</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-38">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-38">¶</a>
|
||
</div>
|
||
<p>Look up a tag by token index.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre> <span class="hljs-attribute">tag</span>: <span class="hljs-function"><span class="hljs-params">(i)</span> -></span> <span class="hljs-property">@tokens</span>[i]?[<span class="hljs-number">0</span>]</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-39">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-39">¶</a>
|
||
</div>
|
||
<h2 id="constants">Constants</h2>
|
||
|
||
</div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-40">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-40">¶</a>
|
||
</div>
|
||
|
||
</div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-41">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-41">¶</a>
|
||
</div>
|
||
<p>List of the token pairs that must be balanced.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre>BALANCED_PAIRS = [
|
||
[<span class="hljs-string">'('</span>, <span class="hljs-string">')'</span>]
|
||
[<span class="hljs-string">'['</span>, <span class="hljs-string">']'</span>]
|
||
[<span class="hljs-string">'{'</span>, <span class="hljs-string">'}'</span>]
|
||
[<span class="hljs-string">'INDENT'</span>, <span class="hljs-string">'OUTDENT'</span>],
|
||
[<span class="hljs-string">'CALL_START'</span>, <span class="hljs-string">'CALL_END'</span>]
|
||
[<span class="hljs-string">'PARAM_START'</span>, <span class="hljs-string">'PARAM_END'</span>]
|
||
[<span class="hljs-string">'INDEX_START'</span>, <span class="hljs-string">'INDEX_END'</span>]
|
||
[<span class="hljs-string">'STRING_START'</span>, <span class="hljs-string">'STRING_END'</span>]
|
||
[<span class="hljs-string">'REGEX_START'</span>, <span class="hljs-string">'REGEX_END'</span>]
|
||
]</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-42">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-42">¶</a>
|
||
</div>
|
||
<p>The inverse mappings of <code>BALANCED_PAIRS</code> we’re trying to fix up, so we can
|
||
look things up from either end.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre><span class="hljs-built_in">exports</span>.INVERSES = INVERSES = {}</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-43">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-43">¶</a>
|
||
</div>
|
||
<p>The tokens that signal the start/end of a balanced pair.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre>EXPRESSION_START = []
|
||
EXPRESSION_END = []
|
||
|
||
<span class="hljs-keyword">for</span> [left, rite] <span class="hljs-keyword">in</span> BALANCED_PAIRS
|
||
EXPRESSION_START.push INVERSES[rite] = left
|
||
EXPRESSION_END .push INVERSES[left] = rite</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-44">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-44">¶</a>
|
||
</div>
|
||
<p>Tokens that indicate the close of a clause of an expression.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre>EXPRESSION_CLOSE = [<span class="hljs-string">'CATCH'</span>, <span class="hljs-string">'THEN'</span>, <span class="hljs-string">'ELSE'</span>, <span class="hljs-string">'FINALLY'</span>].concat EXPRESSION_END</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-45">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-45">¶</a>
|
||
</div>
|
||
<p>Tokens that, if followed by an <code>IMPLICIT_CALL</code>, indicate a function invocation.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre>IMPLICIT_FUNC = [<span class="hljs-string">'IDENTIFIER'</span>, <span class="hljs-string">'SUPER'</span>, <span class="hljs-string">')'</span>, <span class="hljs-string">'CALL_END'</span>, <span class="hljs-string">']'</span>, <span class="hljs-string">'INDEX_END'</span>, <span class="hljs-string">'@'</span>, <span class="hljs-string">'THIS'</span>]</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-46">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-46">¶</a>
|
||
</div>
|
||
<p>If preceded by an <code>IMPLICIT_FUNC</code>, indicates a function invocation.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre>IMPLICIT_CALL = [
|
||
<span class="hljs-string">'IDENTIFIER'</span>, <span class="hljs-string">'NUMBER'</span>, <span class="hljs-string">'STRING'</span>, <span class="hljs-string">'STRING_START'</span>, <span class="hljs-string">'JS'</span>, <span class="hljs-string">'REGEX'</span>, <span class="hljs-string">'REGEX_START'</span>
|
||
<span class="hljs-string">'NEW'</span>, <span class="hljs-string">'PARAM_START'</span>, <span class="hljs-string">'CLASS'</span>, <span class="hljs-string">'IF'</span>, <span class="hljs-string">'TRY'</span>, <span class="hljs-string">'SWITCH'</span>, <span class="hljs-string">'THIS'</span>, <span class="hljs-string">'BOOL'</span>, <span class="hljs-string">'NULL'</span>
|
||
<span class="hljs-string">'UNDEFINED'</span>, <span class="hljs-string">'UNARY'</span>, <span class="hljs-string">'YIELD'</span>, <span class="hljs-string">'UNARY_MATH'</span>, <span class="hljs-string">'SUPER'</span>, <span class="hljs-string">'THROW'</span>
|
||
<span class="hljs-string">'@'</span>, <span class="hljs-string">'->'</span>, <span class="hljs-string">'=>'</span>, <span class="hljs-string">'['</span>, <span class="hljs-string">'('</span>, <span class="hljs-string">'{'</span>, <span class="hljs-string">'--'</span>, <span class="hljs-string">'++'</span>
|
||
]
|
||
|
||
IMPLICIT_UNSPACED_CALL = [<span class="hljs-string">'+'</span>, <span class="hljs-string">'-'</span>]</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-47">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-47">¶</a>
|
||
</div>
|
||
<p>Tokens that always mark the end of an implicit call for single-liners.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre>IMPLICIT_END = [<span class="hljs-string">'POST_IF'</span>, <span class="hljs-string">'FOR'</span>, <span class="hljs-string">'WHILE'</span>, <span class="hljs-string">'UNTIL'</span>, <span class="hljs-string">'WHEN'</span>, <span class="hljs-string">'BY'</span>,
|
||
<span class="hljs-string">'LOOP'</span>, <span class="hljs-string">'TERMINATOR'</span>]</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-48">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-48">¶</a>
|
||
</div>
|
||
<p>Single-line flavors of block expressions that have unclosed endings.
|
||
The grammar can’t disambiguate them, so we insert the implicit indentation.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre>SINGLE_LINERS = [<span class="hljs-string">'ELSE'</span>, <span class="hljs-string">'->'</span>, <span class="hljs-string">'=>'</span>, <span class="hljs-string">'TRY'</span>, <span class="hljs-string">'FINALLY'</span>, <span class="hljs-string">'THEN'</span>]
|
||
SINGLE_CLOSERS = [<span class="hljs-string">'TERMINATOR'</span>, <span class="hljs-string">'CATCH'</span>, <span class="hljs-string">'FINALLY'</span>, <span class="hljs-string">'ELSE'</span>, <span class="hljs-string">'OUTDENT'</span>, <span class="hljs-string">'LEADING_WHEN'</span>]</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-49">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-49">¶</a>
|
||
</div>
|
||
<p>Tokens that end a line.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre>LINEBREAKS = [<span class="hljs-string">'TERMINATOR'</span>, <span class="hljs-string">'INDENT'</span>, <span class="hljs-string">'OUTDENT'</span>]</pre></div></div>
|
||
|
||
</li>
|
||
|
||
|
||
<li id="section-50">
|
||
<div class="annotation">
|
||
|
||
<div class="pilwrap ">
|
||
<a class="pilcrow" href="#section-50">¶</a>
|
||
</div>
|
||
<p>Tokens that close open calls when they follow a newline.</p>
|
||
|
||
</div>
|
||
|
||
<div class="content"><div class='highlight'><pre>CALL_CLOSERS = [<span class="hljs-string">'.'</span>, <span class="hljs-string">'?.'</span>, <span class="hljs-string">'::'</span>, <span class="hljs-string">'?::'</span>]</pre></div></div>
|
||
|
||
</li>
|
||
|
||
</ul>
|
||
</div>
|
||
</body>
|
||
</html>
|