mirror of
https://github.com/jashkenas/coffeescript.git
synced 2022-11-09 12:23:24 -05:00
cleaned and commented the lexer (again) interpolate_string() continues to shrink
This commit is contained in:
Jeremy Ashkenas
parent
f74fae58e3
commit
4906cf1aff
4 changed files with 403 additions and 335 deletions
|
|
@ -36,7 +36,7 @@ to avoid having a JavaScript error at runtime.</p> </td>
|
|||
be used as identifiers or properties.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">JS_FORBIDDEN: </span><span class="nx">JS_KEYWORDS</span><span class="p">.</span><span class="nx">concat</span> <span class="nx">RESERVED</span></pre></div> </td> </tr> <tr id="section-9"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-9">#</a> </div> <p>Token matching regexes.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nx">IDENTIFIER</span> <span class="o">:</span> <span class="sr">/^([a-zA-Z$_](\w|\$)*)/</span>
|
||||
<span class="nx">NUMBER</span> <span class="o">:</span> <span class="sr">/^(\b((0(x|X)[0-9a-fA-F]+)|([0-9]+(\.[0-9]+)?(e[+\-]?[0-9]+)?)))\b/i</span>
|
||||
<span class="nx">HEREDOC</span> <span class="o">:</span> <span class="sr">/^("{6}|'{6}|"{3}\n?([\s\S]*?)\n?([ \t]*)"{3}|'{3}\n?([\s\S]*?)\n?([ \t]*)'{3})/</span>
|
||||
<span class="nx">INTERPOLATION</span> <span class="o">:</span> <span class="sr">/(^|[\s\S]*?(?:[\\]|\\\\)?)\$([a-zA-Z_@]\w*|{[\s\S]*?(?:[^\\]|\\\\)})/</span>
|
||||
<span class="nx">INTERPOLATION</span> <span class="o">:</span> <span class="sr">/^\$([a-zA-Z_@]\w*)/</span>
|
||||
<span class="nx">OPERATOR</span> <span class="o">:</span> <span class="sr">/^([+\*&|\/\-%=<>:!?]+)/</span>
|
||||
<span class="nx">WHITESPACE</span> <span class="o">:</span> <span class="sr">/^([ \t]+)/</span>
|
||||
<span class="nx">COMMENT</span> <span class="o">:</span> <span class="sr">/^(((\n?[ \t]*)?#[^\n]*)+)/</span>
|
||||
|
|
@ -64,31 +64,45 @@ treated as identifiers.</p> </td> <td class="code">
|
|||
occurs at the start of a line. We disambiguate these from trailing whens to
|
||||
avoid an ambiguity in the grammar.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">BEFORE_WHEN: </span><span class="p">[</span><span class="s1">'INDENT'</span><span class="p">,</span> <span class="s1">'OUTDENT'</span><span class="p">,</span> <span class="s1">'TERMINATOR'</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-15"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-15">#</a> </div> <h2>The Lexer Class</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-16"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-16">#</a> </div> <p>The Lexer class reads a stream of CoffeeScript and divvys it up into tagged
|
||||
tokens. A minor bit of the ambiguity in the grammar has been avoided by
|
||||
pushing some extra smarts into the Lexer.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">exports.Lexer: </span><span class="nx">class</span> <span class="nx">Lexer</span></pre></div> </td> </tr> <tr id="section-17"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-17">#</a> </div> <p>Scan by attempting to match tokens one at a time. Slow and steady.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tokenize: </span><span class="p">(</span><span class="nx">code</span><span class="p">,</span> <span class="nx">options</span><span class="p">)</span> <span class="o">-></span>
|
||||
pushing some extra smarts into the Lexer.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">exports.Lexer: </span><span class="nx">class</span> <span class="nx">Lexer</span></pre></div> </td> </tr> <tr id="section-17"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-17">#</a> </div> <p><strong>tokenize</strong> is the Lexer's main method. Scan by attempting to match tokens
|
||||
one at a time, using a regular expression anchored at the start of the
|
||||
remaining code, or a custom recursive token-matching method
|
||||
(for interpolations). When the next token has been recorded, we move forward
|
||||
within the code past the token, and begin again.</p>
|
||||
|
||||
<p>Each tokenizing method is responsible for incrementing <code>@i</code> by the number of
|
||||
characters it has consumed. <code>@i</code> can be thought of as our finger on the page
|
||||
of source.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tokenize: </span><span class="p">(</span><span class="nx">code</span><span class="p">,</span> <span class="nx">options</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="nx">o</span> <span class="o">:</span> <span class="nx">options</span> <span class="o">or</span> <span class="p">{}</span>
|
||||
<span class="err">@</span><span class="nx">code</span> <span class="o">:</span> <span class="nx">code</span> <span class="c1"># The remainder of the source code.</span>
|
||||
<span class="err">@</span><span class="nx">i</span> <span class="o">:</span> <span class="mi">0</span> <span class="c1"># Current character position we're parsing.</span>
|
||||
<span class="err">@</span><span class="nx">line</span> <span class="o">:</span> <span class="nx">o</span><span class="p">.</span><span class="nx">line</span> <span class="o">or</span> <span class="mi">0</span> <span class="c1"># The current line.</span>
|
||||
<span class="err">@</span><span class="nx">indent</span> <span class="o">:</span> <span class="mi">0</span> <span class="c1"># The current indent level.</span>
|
||||
<span class="err">@</span><span class="nx">indents</span> <span class="o">:</span> <span class="p">[]</span> <span class="c1"># The stack of all indent levels we are currently within.</span>
|
||||
<span class="err">@</span><span class="nx">tokens</span> <span class="o">:</span> <span class="p">[]</span> <span class="c1"># Collection of all parsed tokens in the form ['TOKEN_TYPE', value, line]</span>
|
||||
<span class="err">@</span><span class="nx">indent</span> <span class="o">:</span> <span class="mi">0</span> <span class="c1"># The current indentation level.</span>
|
||||
<span class="err">@</span><span class="nx">indents</span> <span class="o">:</span> <span class="p">[]</span> <span class="c1"># The stack of all current indentation levels.</span>
|
||||
<span class="err">@</span><span class="nx">tokens</span> <span class="o">:</span> <span class="p">[]</span> <span class="c1"># Stream of parsed tokens in the form ['TYPE', value, line]</span>
|
||||
<span class="k">while</span> <span class="err">@</span><span class="nx">i</span> <span class="o"><</span> <span class="err">@</span><span class="nx">code</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="err">@</span><span class="nv">chunk: </span><span class="err">@</span><span class="nx">code</span><span class="p">.</span><span class="nx">slice</span><span class="p">(</span><span class="err">@</span><span class="nx">i</span><span class="p">)</span>
|
||||
<span class="err">@</span><span class="nx">extract_next_token</span><span class="p">()</span>
|
||||
<span class="err">@</span><span class="nx">close_indentation</span><span class="p">()</span>
|
||||
<span class="k">return</span> <span class="err">@</span><span class="nx">tokens</span> <span class="k">if</span> <span class="nx">o</span><span class="p">.</span><span class="nx">rewrite</span> <span class="o">is</span> <span class="kc">no</span>
|
||||
<span class="k">return</span> <span class="err">@</span><span class="nx">tokens</span> <span class="k">if</span> <span class="nx">o</span><span class="p">.</span><span class="nx">rewrite</span> <span class="o">is</span> <span class="kc">off</span>
|
||||
<span class="p">(</span><span class="k">new</span> <span class="nx">Rewriter</span><span class="p">()).</span><span class="nx">rewrite</span> <span class="err">@</span><span class="nx">tokens</span></pre></div> </td> </tr> <tr id="section-18"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-18">#</a> </div> <p>At every position, run through this list of attempted matches,
|
||||
short-circuiting if any of them succeed.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">extract_next_token: </span><span class="o">-></span>
|
||||
short-circuiting if any of them succeed. Their order determines precedence:
|
||||
<code>@literal_token</code> is the fallback catch-all.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">extract_next_token: </span><span class="o">-></span>
|
||||
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">identifier_token</span><span class="p">()</span>
|
||||
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">number_token</span><span class="p">()</span>
|
||||
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">heredoc_token</span><span class="p">()</span>
|
||||
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">string_token</span><span class="p">()</span>
|
||||
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">js_token</span><span class="p">()</span>
|
||||
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">regex_token</span><span class="p">()</span>
|
||||
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">comment_token</span><span class="p">()</span>
|
||||
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">line_token</span><span class="p">()</span>
|
||||
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">whitespace_token</span><span class="p">()</span>
|
||||
<span class="k">return</span> <span class="err">@</span><span class="nx">literal_token</span><span class="p">()</span></pre></div> </td> </tr> <tr id="section-19"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-19">#</a> </div> <h2>Tokenizers</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-20"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-20">#</a> </div> <p>Matches identifying literals: variables, keywords, method names, etc.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">identifier_token: </span><span class="o">-></span>
|
||||
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">js_token</span><span class="p">()</span>
|
||||
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">string_token</span><span class="p">()</span>
|
||||
<span class="k">return</span> <span class="err">@</span><span class="nx">literal_token</span><span class="p">()</span></pre></div> </td> </tr> <tr id="section-19"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-19">#</a> </div> <h2>Tokenizers</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-20"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-20">#</a> </div> <p>Matches identifying literals: variables, keywords, method names, etc.
|
||||
Check to ensure that JavaScript reserved words aren't being used as
|
||||
identifiers. Because CoffeeScript reserves a handful of keywords that are
|
||||
allowed in JavaScript, we're careful not to tag them as keywords when
|
||||
referenced as property names here, so you can still do <code>jQuery.is()</code> even
|
||||
though <code>is</code> means <code>===</code> otherwise.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">identifier_token: </span><span class="o">-></span>
|
||||
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">id: </span><span class="err">@</span><span class="nx">match</span> <span class="nx">IDENTIFIER</span><span class="p">,</span> <span class="mi">1</span>
|
||||
<span class="err">@</span><span class="nx">name_access_type</span><span class="p">()</span>
|
||||
<span class="nv">tag: </span><span class="s1">'IDENTIFIER'</span>
|
||||
|
|
@ -102,60 +116,55 @@ short-circuiting if any of them succeed.</p> </td> <td c
|
|||
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">number: </span><span class="err">@</span><span class="nx">match</span> <span class="nx">NUMBER</span><span class="p">,</span> <span class="mi">1</span>
|
||||
<span class="err">@</span><span class="nx">token</span> <span class="s1">'NUMBER'</span><span class="p">,</span> <span class="nx">number</span>
|
||||
<span class="err">@</span><span class="nx">i</span> <span class="o">+=</span> <span class="nx">number</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-22"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-22">#</a> </div> <p>Matches strings, including multi-line strings.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">string_token: </span><span class="o">-></span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-22"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-22">#</a> </div> <p>Matches strings, including multi-line strings. Ensures that quotation marks
|
||||
are balanced within the string's contents, and within nested interpolations.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">string_token: </span><span class="o">-></span>
|
||||
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nx">starts</span><span class="p">(</span><span class="err">@</span><span class="nx">chunk</span><span class="p">,</span> <span class="s1">'"'</span><span class="p">)</span> <span class="o">or</span> <span class="nx">starts</span><span class="p">(</span><span class="err">@</span><span class="nx">chunk</span><span class="p">,</span> <span class="s2">"'"</span><span class="p">)</span>
|
||||
<span class="nv">string: </span><span class="err">@</span><span class="nx">balanced_token</span> <span class="p">[</span><span class="s1">'"'</span><span class="p">,</span> <span class="s1">'"'</span><span class="p">],</span> <span class="p">[</span><span class="s1">'${'</span><span class="p">,</span> <span class="s1">'}'</span><span class="p">]</span>
|
||||
<span class="nv">string: </span><span class="err">@</span><span class="nx">balanced_token</span> <span class="p">[</span><span class="s2">"'"</span><span class="p">,</span> <span class="s2">"'"</span><span class="p">]</span> <span class="k">if</span> <span class="nx">string</span> <span class="o">is</span> <span class="kc">false</span>
|
||||
<span class="nv">string: </span><span class="err">@</span><span class="nx">balanced_token</span> <span class="p">[</span><span class="s2">"'"</span><span class="p">,</span> <span class="s2">"'"</span><span class="p">]</span> <span class="nx">unless</span> <span class="nx">string</span>
|
||||
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nx">string</span>
|
||||
<span class="err">@</span><span class="nx">interpolate_string</span> <span class="nx">string</span><span class="p">.</span><span class="nx">replace</span> <span class="nx">STRING_NEWLINES</span><span class="p">,</span> <span class="s2">" \\\n"</span>
|
||||
<span class="err">@</span><span class="nx">line</span> <span class="o">+=</span> <span class="nx">count</span> <span class="nx">string</span><span class="p">,</span> <span class="s2">"\n"</span>
|
||||
<span class="err">@</span><span class="nx">i</span> <span class="o">+=</span> <span class="nx">string</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-23"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-23">#</a> </div> <p>Matches heredocs, adjusting indentation to the correct level.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">heredoc_token: </span><span class="o">-></span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-23"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-23">#</a> </div> <p>Matches heredocs, adjusting indentation to the correct level, as heredocs
|
||||
preserve whitespace, but ignore indentation to the left.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">heredoc_token: </span><span class="o">-></span>
|
||||
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nx">match</span> <span class="o">=</span> <span class="err">@</span><span class="nx">chunk</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">HEREDOC</span><span class="p">)</span>
|
||||
<span class="nv">doc: </span><span class="err">@</span><span class="nx">sanitize_heredoc</span> <span class="nx">match</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">or</span> <span class="nx">match</span><span class="p">[</span><span class="mi">4</span><span class="p">]</span>
|
||||
<span class="err">@</span><span class="nx">token</span> <span class="s1">'STRING'</span><span class="p">,</span> <span class="s2">"\"$doc\""</span>
|
||||
<span class="err">@</span><span class="nx">line</span> <span class="o">+=</span> <span class="nx">count</span> <span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="s2">"\n"</span>
|
||||
<span class="err">@</span><span class="nx">i</span> <span class="o">+=</span> <span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">].</span><span class="nx">length</span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-24"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-24">#</a> </div> <p>Matches interpolated JavaScript.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">js_token: </span><span class="o">-></span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-24"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-24">#</a> </div> <p>Matches JavaScript interpolated directly into the source via backticks.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">js_token: </span><span class="o">-></span>
|
||||
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nx">starts</span> <span class="err">@</span><span class="nx">chunk</span><span class="p">,</span> <span class="s1">'`'</span>
|
||||
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">script: </span><span class="err">@</span><span class="nx">balanced_token</span> <span class="p">[</span><span class="s1">'`'</span><span class="p">,</span> <span class="s1">'`'</span><span class="p">]</span>
|
||||
<span class="err">@</span><span class="nx">token</span> <span class="s1">'JS'</span><span class="p">,</span> <span class="nx">script</span><span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="nx">JS_CLEANER</span><span class="p">,</span> <span class="s1">''</span><span class="p">)</span>
|
||||
<span class="err">@</span><span class="nx">i</span> <span class="o">+=</span> <span class="nx">script</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-25"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-25">#</a> </div> <p>Matches regular expression literals.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">regex_token: </span><span class="o">-></span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-25"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-25">#</a> </div> <p>Matches regular expression literals. Lexing regular expressions is difficult
|
||||
to distinguish from division, so we borrow some basic heuristics from
|
||||
JavaScript and Ruby.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">regex_token: </span><span class="o">-></span>
|
||||
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">regex: </span><span class="err">@</span><span class="nx">match</span> <span class="nx">REGEX</span><span class="p">,</span> <span class="mi">1</span>
|
||||
<span class="k">return</span> <span class="kc">false</span> <span class="k">if</span> <span class="nx">include</span> <span class="nx">NOT_REGEX</span><span class="p">,</span> <span class="err">@</span><span class="nx">tag</span><span class="p">()</span>
|
||||
<span class="err">@</span><span class="nx">token</span> <span class="s1">'REGEX'</span><span class="p">,</span> <span class="nx">regex</span>
|
||||
<span class="err">@</span><span class="nx">i</span> <span class="o">+=</span> <span class="nx">regex</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-26"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-26">#</a> </div> <p>Matches a balanced group such as a single or double-quoted string. Pass in
|
||||
a series of delimiters, all of which must be balanced correctly within the
|
||||
token's contents.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">balanced_token: </span><span class="p">(</span><span class="nx">delimited</span><span class="p">...)</span> <span class="o">-></span>
|
||||
<span class="nv">levels: </span><span class="p">[]</span>
|
||||
<span class="nv">i: </span><span class="mi">0</span>
|
||||
<span class="k">while</span> <span class="nx">i</span> <span class="o"><</span> <span class="err">@</span><span class="nx">chunk</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="k">for</span> <span class="nx">pair</span> <span class="k">in</span> <span class="nx">delimited</span>
|
||||
<span class="p">[</span><span class="nx">open</span><span class="p">,</span> <span class="nx">close</span><span class="p">]</span><span class="o">:</span> <span class="nx">pair</span>
|
||||
<span class="k">if</span> <span class="nx">levels</span><span class="p">.</span><span class="nx">length</span> <span class="o">and</span> <span class="nx">starts</span> <span class="err">@</span><span class="nx">chunk</span><span class="p">,</span> <span class="s1">'\\'</span><span class="p">,</span> <span class="nx">i</span>
|
||||
<span class="nx">i</span> <span class="o">+=</span> <span class="mi">1</span>
|
||||
<span class="k">break</span>
|
||||
<span class="k">else</span> <span class="k">if</span> <span class="nx">levels</span><span class="p">.</span><span class="nx">length</span> <span class="o">and</span> <span class="nx">starts</span><span class="p">(</span><span class="err">@</span><span class="nx">chunk</span><span class="p">,</span> <span class="nx">close</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">and</span> <span class="nx">levels</span><span class="p">[</span><span class="nx">levels</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span> <span class="o">is</span> <span class="nx">pair</span>
|
||||
<span class="nx">levels</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
|
||||
<span class="nx">i</span> <span class="o">+=</span> <span class="nx">close</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
<span class="nx">i</span> <span class="o">+=</span> <span class="mi">1</span> <span class="nx">unless</span> <span class="nx">levels</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="k">break</span>
|
||||
<span class="k">else</span> <span class="k">if</span> <span class="nx">starts</span> <span class="err">@</span><span class="nx">chunk</span><span class="p">,</span> <span class="nx">open</span><span class="p">,</span> <span class="nx">i</span>
|
||||
<span class="nx">levels</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">pair</span><span class="p">)</span>
|
||||
<span class="nx">i</span> <span class="o">+=</span> <span class="nx">open</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
<span class="k">break</span>
|
||||
<span class="k">break</span> <span class="nx">unless</span> <span class="nx">levels</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="nx">i</span> <span class="o">+=</span> <span class="mi">1</span>
|
||||
<span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span> <span class="s2">"SyntaxError: Unterminated ${levels.pop()[0]} starting on line ${@line + 1}"</span> <span class="k">if</span> <span class="nx">levels</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="k">return</span> <span class="kc">false</span> <span class="k">if</span> <span class="nx">i</span> <span class="o">is</span> <span class="mi">0</span>
|
||||
<span class="k">return</span> <span class="err">@</span><span class="nx">chunk</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-27"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-27">#</a> </div> <p>Matches and conumes comments.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">comment_token: </span><span class="o">-></span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-26"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-26">#</a> </div> <p>Matches a token in which which the passed delimiter pairs must be correctly
|
||||
balanced (ie. strings, JS literals).</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">balanced_token: </span><span class="p">(</span><span class="nx">delimited</span><span class="p">...)</span> <span class="o">-></span>
|
||||
<span class="err">@</span><span class="nx">balanced_string</span> <span class="err">@</span><span class="nx">chunk</span><span class="p">,</span> <span class="nx">delimited</span><span class="p">...</span></pre></div> </td> </tr> <tr id="section-27"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-27">#</a> </div> <p>Matches and conumes comments. We pass through comments into JavaScript,
|
||||
so they're treated as real tokens, like any other part of the language.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">comment_token: </span><span class="o">-></span>
|
||||
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">comment: </span><span class="err">@</span><span class="nx">match</span> <span class="nx">COMMENT</span><span class="p">,</span> <span class="mi">1</span>
|
||||
<span class="err">@</span><span class="nx">line</span> <span class="o">+=</span> <span class="p">(</span><span class="nx">comment</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">MULTILINER</span><span class="p">)</span> <span class="o">or</span> <span class="p">[]).</span><span class="nx">length</span>
|
||||
<span class="nv">lines: </span><span class="nx">comment</span><span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="nx">COMMENT_CLEANER</span><span class="p">,</span> <span class="s1">''</span><span class="p">).</span><span class="nx">split</span><span class="p">(</span><span class="nx">MULTILINER</span><span class="p">)</span>
|
||||
<span class="err">@</span><span class="nx">token</span> <span class="s1">'COMMENT'</span><span class="p">,</span> <span class="nx">compact</span> <span class="nx">lines</span>
|
||||
<span class="err">@</span><span class="nx">token</span> <span class="s1">'TERMINATOR'</span><span class="p">,</span> <span class="s2">"\n"</span>
|
||||
<span class="err">@</span><span class="nx">i</span> <span class="o">+=</span> <span class="nx">comment</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-28"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-28">#</a> </div> <p>Matches newlines, indents, and outdents, and determines which is which.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">line_token: </span><span class="o">-></span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-28"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-28">#</a> </div> <p>Matches newlines, indents, and outdents, and determines which is which.
|
||||
If we can detect that the current line is continued onto the the next line,
|
||||
then the newline is suppressed:</p>
|
||||
|
||||
<pre><code>elements
|
||||
.each( ... )
|
||||
.map( ... )
|
||||
</code></pre>
|
||||
|
||||
<p>Keeps track of the level of indentation, because a single outdent token
|
||||
can close multiple indents, so we need to know how far in we happen to be.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">line_token: </span><span class="o">-></span>
|
||||
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">indent: </span><span class="err">@</span><span class="nx">match</span> <span class="nx">MULTI_DENT</span><span class="p">,</span> <span class="mi">1</span>
|
||||
<span class="err">@</span><span class="nx">line</span> <span class="o">+=</span> <span class="nx">indent</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">MULTILINER</span><span class="p">).</span><span class="nx">length</span>
|
||||
<span class="err">@</span><span class="nx">i</span> <span class="o">+=</span> <span class="nx">indent</span><span class="p">.</span><span class="nx">length</span>
|
||||
|
|
@ -165,18 +174,18 @@ token's contents.</p> </td> <td class="code">
|
|||
<span class="nv">no_newlines: </span><span class="nx">next_character</span> <span class="o">is</span> <span class="s1">'.'</span> <span class="o">or</span> <span class="p">(</span><span class="err">@</span><span class="nx">value</span><span class="p">()</span> <span class="o">and</span> <span class="err">@</span><span class="nx">value</span><span class="p">().</span><span class="nx">match</span><span class="p">(</span><span class="nx">NO_NEWLINE</span><span class="p">)</span> <span class="o">and</span>
|
||||
<span class="nx">prev</span> <span class="o">and</span> <span class="p">(</span><span class="nx">prev</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">isnt</span> <span class="s1">'.'</span><span class="p">)</span> <span class="o">and</span> <span class="o">not</span> <span class="err">@</span><span class="nx">value</span><span class="p">().</span><span class="nx">match</span><span class="p">(</span><span class="nx">CODE</span><span class="p">))</span>
|
||||
<span class="k">if</span> <span class="nx">size</span> <span class="o">is</span> <span class="err">@</span><span class="nx">indent</span>
|
||||
<span class="k">return</span> <span class="err">@</span><span class="nx">suppress_newlines</span><span class="p">(</span><span class="nx">indent</span><span class="p">)</span> <span class="k">if</span> <span class="nx">no_newlines</span>
|
||||
<span class="k">return</span> <span class="err">@</span><span class="nx">suppress_newlines</span><span class="p">()</span> <span class="k">if</span> <span class="nx">no_newlines</span>
|
||||
<span class="k">return</span> <span class="err">@</span><span class="nx">newline_token</span><span class="p">(</span><span class="nx">indent</span><span class="p">)</span>
|
||||
<span class="k">else</span> <span class="k">if</span> <span class="nx">size</span> <span class="o">></span> <span class="err">@</span><span class="nx">indent</span>
|
||||
<span class="k">return</span> <span class="err">@</span><span class="nx">suppress_newlines</span><span class="p">(</span><span class="nx">indent</span><span class="p">)</span> <span class="k">if</span> <span class="nx">no_newlines</span>
|
||||
<span class="k">return</span> <span class="err">@</span><span class="nx">suppress_newlines</span><span class="p">()</span> <span class="k">if</span> <span class="nx">no_newlines</span>
|
||||
<span class="nv">diff: </span><span class="nx">size</span> <span class="o">-</span> <span class="err">@</span><span class="nx">indent</span>
|
||||
<span class="err">@</span><span class="nx">token</span> <span class="s1">'INDENT'</span><span class="p">,</span> <span class="nx">diff</span>
|
||||
<span class="err">@</span><span class="nx">indents</span><span class="p">.</span><span class="nx">push</span> <span class="nx">diff</span>
|
||||
<span class="k">else</span>
|
||||
<span class="err">@</span><span class="nx">outdent_token</span> <span class="err">@</span><span class="nx">indent</span> <span class="o">-</span> <span class="nx">size</span><span class="p">,</span> <span class="nx">no_newlines</span>
|
||||
<span class="err">@</span><span class="nv">indent: </span><span class="nx">size</span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-29"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-29">#</a> </div> <p>Record an outdent token or tokens, if we happen to be moving back inwards
|
||||
past multiple recorded indents.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">outdent_token: </span><span class="p">(</span><span class="nx">move_out</span><span class="p">,</span> <span class="nx">no_newlines</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-29"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-29">#</a> </div> <p>Record an outdent token or multiple tokens, if we happen to be moving back
|
||||
inwards past several recorded indents.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">outdent_token: </span><span class="p">(</span><span class="nx">move_out</span><span class="p">,</span> <span class="nx">no_newlines</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="k">while</span> <span class="nx">move_out</span> <span class="o">></span> <span class="mi">0</span> <span class="o">and</span> <span class="err">@</span><span class="nx">indents</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="nv">last_indent: </span><span class="err">@</span><span class="nx">indents</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
|
||||
<span class="err">@</span><span class="nx">token</span> <span class="s1">'OUTDENT'</span><span class="p">,</span> <span class="nx">last_indent</span>
|
||||
|
|
@ -188,14 +197,16 @@ as being "spaced", because there are some cases where it makes a difference.</p>
|
|||
<span class="nv">prev: </span><span class="err">@</span><span class="nx">prev</span><span class="p">()</span>
|
||||
<span class="nv">prev.spaced: </span><span class="kc">true</span> <span class="k">if</span> <span class="nx">prev</span>
|
||||
<span class="err">@</span><span class="nx">i</span> <span class="o">+=</span> <span class="nx">space</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-31"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-31">#</a> </div> <p>Generate a newline token. Multiple newlines get merged together.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">newline_token: </span><span class="p">(</span><span class="nx">newlines</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-31"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-31">#</a> </div> <p>Generate a newline token. Consecutive newlines get merged together.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">newline_token: </span><span class="p">(</span><span class="nx">newlines</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="err">@</span><span class="nx">token</span> <span class="s1">'TERMINATOR'</span><span class="p">,</span> <span class="s2">"\n"</span> <span class="nx">unless</span> <span class="err">@</span><span class="nx">tag</span><span class="p">()</span> <span class="o">is</span> <span class="s1">'TERMINATOR'</span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-32"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-32">#</a> </div> <p>Use a <code>\</code> at a line-ending to suppress the newline.
|
||||
The slash is removed here once its job is done.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">suppress_newlines: </span><span class="p">(</span><span class="nx">newlines</span><span class="p">)</span> <span class="o">-></span>
|
||||
The slash is removed here once its job is done.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">suppress_newlines: </span><span class="o">-></span>
|
||||
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span> <span class="k">if</span> <span class="err">@</span><span class="nx">value</span><span class="p">()</span> <span class="o">is</span> <span class="s2">"\\"</span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-33"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-33">#</a> </div> <p>We treat all other single characters as a token. Eg.: <code>( ) , . !</code>
|
||||
Multi-character operators are also literal tokens, so that Jison can assign
|
||||
the proper order of operations.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">literal_token: </span><span class="o">-></span>
|
||||
the proper order of operations. There are some symbols that we tag specially
|
||||
here. <code>;</code> and newlines are both treated as a <code>TERMINATOR</code>, we distinguish
|
||||
parentheses that indicate a method call from regular parentheses, and so on.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">literal_token: </span><span class="o">-></span>
|
||||
<span class="nv">match: </span><span class="err">@</span><span class="nx">chunk</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">OPERATOR</span><span class="p">)</span>
|
||||
<span class="nv">value: </span><span class="nx">match</span> <span class="o">and</span> <span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
|
||||
<span class="err">@</span><span class="nx">tag_parameters</span><span class="p">()</span> <span class="k">if</span> <span class="nx">value</span> <span class="o">and</span> <span class="nx">value</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">CODE</span><span class="p">)</span>
|
||||
|
|
@ -227,15 +238,14 @@ if it's a special kind of accessor.</p> </td> <td class=
|
|||
<span class="err">@</span><span class="nx">tag</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="s1">'SOAK_ACCESS'</span><span class="p">)</span>
|
||||
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span><span class="p">(</span><span class="o">-</span><span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
|
||||
<span class="k">else</span>
|
||||
<span class="err">@</span><span class="nx">tag</span> <span class="mi">1</span><span class="p">,</span> <span class="s1">'PROPERTY_ACCESS'</span></pre></div> </td> </tr> <tr id="section-36"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-36">#</a> </div> <p>Sanitize a heredoc by escaping double quotes and erasing all external
|
||||
indentation on the left-hand side.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">sanitize_heredoc: </span><span class="p">(</span><span class="nx">doc</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="err">@</span><span class="nx">tag</span> <span class="mi">1</span><span class="p">,</span> <span class="s1">'PROPERTY_ACCESS'</span></pre></div> </td> </tr> <tr id="section-36"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-36">#</a> </div> <p>Sanitize a heredoc by escaping internal double quotes and erasing all
|
||||
external indentation on the left-hand side.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">sanitize_heredoc: </span><span class="p">(</span><span class="nx">doc</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="nv">indent: </span><span class="p">(</span><span class="nx">doc</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">HEREDOC_INDENT</span><span class="p">)</span> <span class="o">or</span> <span class="p">[</span><span class="s1">''</span><span class="p">]).</span><span class="nx">sort</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span>
|
||||
<span class="nx">doc</span><span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="k">new</span> <span class="nb">RegExp</span><span class="p">(</span><span class="s2">"^"</span> <span class="o">+</span><span class="nx">indent</span><span class="p">,</span> <span class="s1">'gm'</span><span class="p">),</span> <span class="s1">''</span><span class="p">)</span>
|
||||
<span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="nx">MULTILINER</span><span class="p">,</span> <span class="s2">"\\n"</span><span class="p">)</span>
|
||||
<span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="sr">/"/g</span><span class="p">,</span> <span class="s1">'\\"'</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-37"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-37">#</a> </div> <p>A source of ambiguity in our grammar was parameter lists in function
|
||||
definitions (as opposed to argument lists in function calls). Tag
|
||||
parameter identifiers in order to avoid this. Also, parameter lists can
|
||||
make use of splats.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tag_parameters: </span><span class="o">-></span>
|
||||
<span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="sr">/"/g</span><span class="p">,</span> <span class="s1">'\\"'</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-37"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-37">#</a> </div> <p>A source of ambiguity in our grammar used to be parameter lists in function
|
||||
definitions versus argument lists in function calls. Walk backwards, tagging
|
||||
parameters specially in order to make things easier for the parser.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tag_parameters: </span><span class="o">-></span>
|
||||
<span class="k">return</span> <span class="k">if</span> <span class="err">@</span><span class="nx">tag</span><span class="p">()</span> <span class="o">isnt</span> <span class="s1">')'</span>
|
||||
<span class="nv">i: </span><span class="mi">0</span>
|
||||
<span class="k">while</span> <span class="kc">true</span>
|
||||
|
|
@ -247,69 +257,92 @@ make use of splats.</p> </td> <td class="code">
|
|||
<span class="k">when</span> <span class="s1">')'</span> <span class="k">then</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">:</span> <span class="s1">'PARAM_END'</span>
|
||||
<span class="k">when</span> <span class="s1">'('</span> <span class="k">then</span> <span class="k">return</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">:</span> <span class="s1">'PARAM_START'</span>
|
||||
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-38"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-38">#</a> </div> <p>Close up all remaining open blocks at the end of the file.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">close_indentation: </span><span class="o">-></span>
|
||||
<span class="err">@</span><span class="nx">outdent_token</span><span class="p">(</span><span class="err">@</span><span class="nx">indent</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-39"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-39">#</a> </div> <p>Error for when you try to use a forbidden word in JavaScript as
|
||||
<span class="err">@</span><span class="nx">outdent_token</span><span class="p">(</span><span class="err">@</span><span class="nx">indent</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-39"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-39">#</a> </div> <p>The error for when you try to use a forbidden word in JavaScript as
|
||||
an identifier.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">identifier_error: </span><span class="p">(</span><span class="nx">word</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span> <span class="s2">"SyntaxError: Reserved word \"$word\" on line ${@line + 1}"</span></pre></div> </td> </tr> <tr id="section-40"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-40">#</a> </div> <p>Error for when you try to assign to a reserved word in JavaScript,
|
||||
<span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span> <span class="s2">"SyntaxError: Reserved word \"$word\" on line ${@line + 1}"</span></pre></div> </td> </tr> <tr id="section-40"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-40">#</a> </div> <p>The error for when you try to assign to a reserved word in JavaScript,
|
||||
like "function" or "default".</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">assignment_error: </span><span class="o">-></span>
|
||||
<span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span> <span class="s2">"SyntaxError: Reserved word \"${@value()}\" on line ${@line + 1} can't be assigned"</span></pre></div> </td> </tr> <tr id="section-41"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-41">#</a> </div> <p>Expand variables and expressions inside double-quoted strings using
|
||||
<a href="http://wiki.ecmascript.org/doku.php?id=strawman:string_interpolation">ECMA Harmony's interpolation syntax</a>.</p>
|
||||
<span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span> <span class="s2">"SyntaxError: Reserved word \"${@value()}\" on line ${@line + 1} can't be assigned"</span></pre></div> </td> </tr> <tr id="section-41"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-41">#</a> </div> <p>Matches a balanced group such as a single or double-quoted string. Pass in
|
||||
a series of delimiters, all of which must be nested correctly within the
|
||||
contents of the string. This method allows us to have strings within
|
||||
interpolations within strings etc...</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">balanced_string: </span><span class="p">(</span><span class="nx">str</span><span class="p">,</span> <span class="nx">delimited</span><span class="p">...)</span> <span class="o">-></span>
|
||||
<span class="nv">levels: </span><span class="p">[]</span>
|
||||
<span class="nv">i: </span><span class="mi">0</span>
|
||||
<span class="k">while</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">str</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="k">for</span> <span class="nx">pair</span> <span class="k">in</span> <span class="nx">delimited</span>
|
||||
<span class="p">[</span><span class="nx">open</span><span class="p">,</span> <span class="nx">close</span><span class="p">]</span><span class="o">:</span> <span class="nx">pair</span>
|
||||
<span class="k">if</span> <span class="nx">levels</span><span class="p">.</span><span class="nx">length</span> <span class="o">and</span> <span class="nx">starts</span> <span class="nx">str</span><span class="p">,</span> <span class="s1">'\\'</span><span class="p">,</span> <span class="nx">i</span>
|
||||
<span class="nx">i</span> <span class="o">+=</span> <span class="mi">1</span>
|
||||
<span class="k">break</span>
|
||||
<span class="k">else</span> <span class="k">if</span> <span class="nx">levels</span><span class="p">.</span><span class="nx">length</span> <span class="o">and</span> <span class="nx">starts</span><span class="p">(</span><span class="nx">str</span><span class="p">,</span> <span class="nx">close</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">and</span> <span class="nx">levels</span><span class="p">[</span><span class="nx">levels</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span> <span class="o">is</span> <span class="nx">pair</span>
|
||||
<span class="nx">levels</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
|
||||
<span class="nx">i</span> <span class="o">+=</span> <span class="nx">close</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
<span class="nx">i</span> <span class="o">+=</span> <span class="mi">1</span> <span class="nx">unless</span> <span class="nx">levels</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="k">break</span>
|
||||
<span class="k">else</span> <span class="k">if</span> <span class="nx">starts</span> <span class="nx">str</span><span class="p">,</span> <span class="nx">open</span><span class="p">,</span> <span class="nx">i</span>
|
||||
<span class="nx">levels</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">pair</span><span class="p">)</span>
|
||||
<span class="nx">i</span> <span class="o">+=</span> <span class="nx">open</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
<span class="k">break</span>
|
||||
<span class="k">break</span> <span class="nx">unless</span> <span class="nx">levels</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="nx">i</span> <span class="o">+=</span> <span class="mi">1</span>
|
||||
<span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span> <span class="s2">"SyntaxError: Unterminated ${levels.pop()[0]} starting on line ${@line + 1}"</span> <span class="k">if</span> <span class="nx">levels</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="k">return</span> <span class="kc">false</span> <span class="k">if</span> <span class="nx">i</span> <span class="o">is</span> <span class="mi">0</span>
|
||||
<span class="k">return</span> <span class="nx">str</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-42"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-42">#</a> </div> <p>Expand variables and expressions inside double-quoted strings using
|
||||
<a href="http://wiki.ecmascript.org/doku.php?id=strawman:string_interpolation">ECMA Harmony's interpolation syntax</a>
|
||||
for substitution of bare variables as well as arbitrary expressions.</p>
|
||||
|
||||
<pre><code>"Hello $name."
|
||||
"Hello ${name.capitalize()}."
|
||||
</code></pre> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">interpolate_string: </span><span class="p">(</span><span class="nx">str</span><span class="p">)</span> <span class="o">-></span>
|
||||
</code></pre>
|
||||
|
||||
<p>If it encounters an interpolation, this method will recursively create a
|
||||
new Lexer, tokenize the interpolated contents, and merge them into the
|
||||
token stream.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">interpolate_string: </span><span class="p">(</span><span class="nx">str</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="k">if</span> <span class="nx">str</span><span class="p">.</span><span class="nx">length</span> <span class="o"><</span> <span class="mi">3</span> <span class="o">or</span> <span class="o">not</span> <span class="nx">starts</span> <span class="nx">str</span><span class="p">,</span> <span class="s1">'"'</span>
|
||||
<span class="err">@</span><span class="nx">token</span> <span class="s1">'STRING'</span><span class="p">,</span> <span class="nx">str</span>
|
||||
<span class="k">else</span>
|
||||
<span class="nv">lexer: </span> <span class="k">new</span> <span class="nx">Lexer</span><span class="p">()</span>
|
||||
<span class="nv">tokens: </span><span class="p">[]</span>
|
||||
<span class="nv">quote: </span> <span class="nx">str</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
|
||||
<span class="nv">str: </span> <span class="nx">str</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nx">str</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span>
|
||||
<span class="k">while</span> <span class="nx">str</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="nv">match: </span><span class="nx">str</span><span class="p">.</span><span class="nx">match</span> <span class="nx">INTERPOLATION</span>
|
||||
<span class="k">if</span> <span class="nx">match</span>
|
||||
<span class="p">[</span><span class="nx">group</span><span class="p">,</span> <span class="nx">before</span><span class="p">,</span> <span class="nx">interp</span><span class="p">]</span><span class="o">:</span> <span class="nx">match</span>
|
||||
<span class="k">if</span> <span class="nx">starts</span> <span class="nx">before</span><span class="p">,</span> <span class="s1">'\\'</span><span class="p">,</span> <span class="nx">before</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
<span class="nv">prev: </span><span class="nx">before</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nx">before</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span>
|
||||
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'STRING'</span><span class="p">,</span> <span class="s2">"$quote$prev$$interp$quote"</span><span class="p">]</span> <span class="k">if</span> <span class="nx">before</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="k">else</span>
|
||||
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'STRING'</span><span class="p">,</span> <span class="s2">"$quote$before$quote"</span><span class="p">]</span> <span class="k">if</span> <span class="nx">before</span><span class="p">.</span><span class="nx">length</span>
|
||||
<span class="k">if</span> <span class="nx">starts</span> <span class="nx">interp</span><span class="p">,</span> <span class="s1">'{'</span>
|
||||
<span class="nv">inner: </span><span class="nx">interp</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nx">interp</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span>
|
||||
<span class="nv">nested: </span><span class="nx">lexer</span><span class="p">.</span><span class="nx">tokenize</span> <span class="s2">"($inner)"</span><span class="p">,</span> <span class="p">{</span><span class="nv">rewrite: </span><span class="kc">no</span><span class="p">,</span> <span class="nv">line: </span><span class="err">@</span><span class="nx">line</span><span class="p">}</span>
|
||||
<span class="nx">nested</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
|
||||
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'TOKENS'</span><span class="p">,</span> <span class="nx">nested</span><span class="p">]</span>
|
||||
<span class="k">else</span>
|
||||
<span class="nv">interp: </span><span class="s2">"this.${ interp.substring(1) }"</span> <span class="k">if</span> <span class="nx">starts</span> <span class="nx">interp</span><span class="p">,</span> <span class="s1">'@'</span>
|
||||
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'IDENTIFIER'</span><span class="p">,</span> <span class="nx">interp</span><span class="p">]</span>
|
||||
<span class="nv">str: </span><span class="nx">str</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="nx">group</span><span class="p">.</span><span class="nx">length</span><span class="p">)</span>
|
||||
<span class="k">else</span>
|
||||
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'STRING'</span><span class="p">,</span> <span class="s2">"$quote$str$quote"</span><span class="p">]</span>
|
||||
<span class="nv">str: </span><span class="s1">''</span>
|
||||
<span class="k">if</span> <span class="nx">tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">></span> <span class="mi">1</span>
|
||||
<span class="k">for</span> <span class="nx">i</span> <span class="k">in</span> <span class="p">[</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">..</span><span class="mi">1</span><span class="p">]</span>
|
||||
<span class="p">[</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">tok</span><span class="p">]</span><span class="o">:</span> <span class="p">[</span><span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">],</span> <span class="nx">tokens</span><span class="p">[</span><span class="nx">i</span><span class="p">]]</span>
|
||||
<span class="k">if</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">'STRING'</span> <span class="o">and</span> <span class="nx">prev</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">'STRING'</span>
|
||||
<span class="p">[</span><span class="nx">prev</span><span class="p">,</span> <span class="nx">tok</span><span class="p">]</span><span class="o">:</span> <span class="p">[</span><span class="nx">prev</span><span class="p">[</span><span class="mi">1</span><span class="p">].</span><span class="nx">substring</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nx">prev</span><span class="p">[</span><span class="mi">1</span><span class="p">].</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">),</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">1</span><span class="p">].</span><span class="nx">substring</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">1</span><span class="p">].</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)]</span>
|
||||
<span class="nx">tokens</span><span class="p">.</span><span class="nx">splice</span> <span class="nx">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="p">[</span><span class="s1">'STRING'</span><span class="p">,</span> <span class="s2">"$quote$prev$tok$quote"</span><span class="p">]</span>
|
||||
<span class="nv">lexer: </span> <span class="k">new</span> <span class="nx">Lexer</span><span class="p">()</span>
|
||||
<span class="nv">tokens: </span> <span class="p">[]</span>
|
||||
<span class="nv">quote: </span> <span class="nx">str</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
|
||||
<span class="p">[</span><span class="nx">i</span><span class="p">,</span> <span class="nx">pi</span><span class="p">]</span><span class="o">:</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">]</span>
|
||||
<span class="k">while</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">str</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
<span class="k">if</span> <span class="nx">starts</span> <span class="nx">str</span><span class="p">,</span> <span class="s1">'\\'</span><span class="p">,</span> <span class="nx">i</span>
|
||||
<span class="nx">i</span> <span class="o">+=</span> <span class="mi">1</span>
|
||||
<span class="k">else</span> <span class="k">if</span> <span class="nv">match: </span><span class="nx">str</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="nx">i</span><span class="p">).</span><span class="nx">match</span> <span class="nx">INTERPOLATION</span>
|
||||
<span class="p">[</span><span class="nx">group</span><span class="p">,</span> <span class="nx">interp</span><span class="p">]</span><span class="o">:</span> <span class="nx">match</span>
|
||||
<span class="nv">interp: </span><span class="s2">"this.${ interp.substring(1) }"</span> <span class="k">if</span> <span class="nx">starts</span> <span class="nx">interp</span><span class="p">,</span> <span class="s1">'@'</span>
|
||||
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'STRING'</span><span class="p">,</span> <span class="s2">"$quote${ str.substring(pi, i) }$quote"</span><span class="p">]</span> <span class="k">if</span> <span class="nx">pi</span> <span class="o"><</span> <span class="nx">i</span>
|
||||
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'IDENTIFIER'</span><span class="p">,</span> <span class="nx">interp</span><span class="p">]</span>
|
||||
<span class="nx">i</span> <span class="o">+=</span> <span class="nx">group</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
<span class="nv">pi: </span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span>
|
||||
<span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nv">expr: </span><span class="err">@</span><span class="nx">balanced_string</span> <span class="nx">str</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="nx">i</span><span class="p">),</span> <span class="p">[</span><span class="s1">'${'</span><span class="p">,</span> <span class="s1">'}'</span><span class="p">])</span> <span class="o">and</span> <span class="nx">expr</span><span class="p">.</span><span class="nx">length</span> <span class="o">></span> <span class="mi">3</span>
|
||||
<span class="nv">inner: </span><span class="nx">expr</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="nx">expr</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span>
|
||||
<span class="nv">nested: </span><span class="nx">lexer</span><span class="p">.</span><span class="nx">tokenize</span> <span class="s2">"($inner)"</span><span class="p">,</span> <span class="p">{</span><span class="nv">rewrite: </span><span class="kc">no</span><span class="p">,</span> <span class="nv">line: </span><span class="err">@</span><span class="nx">line</span><span class="p">}</span>
|
||||
<span class="nx">nested</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
|
||||
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'STRING'</span><span class="p">,</span> <span class="s2">"$quote${ str.substring(pi, i) }$quote"</span><span class="p">]</span> <span class="k">if</span> <span class="nx">pi</span> <span class="o"><</span> <span class="nx">i</span>
|
||||
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'TOKENS'</span><span class="p">,</span> <span class="nx">nested</span><span class="p">]</span>
|
||||
<span class="nx">i</span> <span class="o">+=</span> <span class="nx">expr</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
<span class="nv">pi: </span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span>
|
||||
<span class="nx">i</span> <span class="o">+=</span> <span class="mi">1</span>
|
||||
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'STRING'</span><span class="p">,</span> <span class="s2">"$quote${ str.substring(pi, i) }$quote"</span><span class="p">]</span> <span class="k">if</span> <span class="nx">pi</span> <span class="o"><</span> <span class="nx">i</span> <span class="o">and</span> <span class="nx">pi</span> <span class="o"><</span> <span class="nx">str</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
<span class="k">for</span> <span class="nx">each</span><span class="p">,</span> <span class="nx">i</span> <span class="k">in</span> <span class="nx">tokens</span>
|
||||
<span class="k">if</span> <span class="nx">each</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">'TOKENS'</span>
|
||||
<span class="err">@</span><span class="nx">token</span> <span class="nx">nested</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="nx">nested</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="k">for</span> <span class="nx">nested</span> <span class="k">in</span> <span class="nx">each</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
|
||||
<span class="err">@</span><span class="nv">tokens: </span><span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">concat</span> <span class="nx">each</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
|
||||
<span class="k">else</span>
|
||||
<span class="err">@</span><span class="nx">token</span> <span class="nx">each</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="nx">each</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
|
||||
<span class="err">@</span><span class="nx">token</span> <span class="s1">'+'</span><span class="p">,</span> <span class="s1">'+'</span> <span class="k">if</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span></pre></div> </td> </tr> <tr id="section-42"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-42">#</a> </div> <h2>Helpers</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-43"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-43">#</a> </div> <p>Add a token to the results, taking note of the line number.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">token: </span><span class="p">(</span><span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span><span class="p">([</span><span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span><span class="p">,</span> <span class="err">@</span><span class="nx">line</span><span class="p">])</span></pre></div> </td> </tr> <tr id="section-44"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-44">#</a> </div> <p>Peek at a tag in the current token stream.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tag: </span><span class="p">(</span><span class="nx">index</span><span class="p">,</span> <span class="nx">tag</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="err">@</span><span class="nx">token</span> <span class="s1">'+'</span><span class="p">,</span> <span class="s1">'+'</span> <span class="k">if</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span></pre></div> </td> </tr> <tr id="section-43"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-43">#</a> </div> <h2>Helpers</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-44"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-44">#</a> </div> <p>Add a token to the results, taking note of the line number.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">token: </span><span class="p">(</span><span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span><span class="p">([</span><span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span><span class="p">,</span> <span class="err">@</span><span class="nx">line</span><span class="p">])</span></pre></div> </td> </tr> <tr id="section-45"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-45">#</a> </div> <p>Peek at a tag in the current token stream.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tag: </span><span class="p">(</span><span class="nx">index</span><span class="p">,</span> <span class="nx">tag</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="k">return</span> <span class="nx">unless</span> <span class="nv">tok: </span><span class="err">@</span><span class="nx">prev</span><span class="p">(</span><span class="nx">index</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">:</span> <span class="nx">tag</span> <span class="k">if</span> <span class="nx">tag</span><span class="o">?</span>
|
||||
<span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-45"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-45">#</a> </div> <p>Peek at a value in the current token stream.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">value: </span><span class="p">(</span><span class="nx">index</span><span class="p">,</span> <span class="nx">val</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-46"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-46">#</a> </div> <p>Peek at a value in the current token stream.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">value: </span><span class="p">(</span><span class="nx">index</span><span class="p">,</span> <span class="nx">val</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="k">return</span> <span class="nx">unless</span> <span class="nv">tok: </span><span class="err">@</span><span class="nx">prev</span><span class="p">(</span><span class="nx">index</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">:</span> <span class="nx">val</span> <span class="k">if</span> <span class="nx">val</span><span class="o">?</span>
|
||||
<span class="nx">tok</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-46"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-46">#</a> </div> <p>Peek at a previous token, entire.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">prev: </span><span class="p">(</span><span class="nx">index</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="err">@</span><span class="nx">tokens</span><span class="p">[</span><span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="p">(</span><span class="nx">index</span> <span class="o">or</span> <span class="mi">1</span><span class="p">)]</span></pre></div> </td> </tr> <tr id="section-47"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-47">#</a> </div> <p>Attempt to match a string against the current chunk, returning the indexed
|
||||
<span class="nx">tok</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-47"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-47">#</a> </div> <p>Peek at a previous token, entire.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">prev: </span><span class="p">(</span><span class="nx">index</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="err">@</span><span class="nx">tokens</span><span class="p">[</span><span class="err">@</span><span class="nx">tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="p">(</span><span class="nx">index</span> <span class="o">or</span> <span class="mi">1</span><span class="p">)]</span></pre></div> </td> </tr> <tr id="section-48"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-48">#</a> </div> <p>Attempt to match a string against the current chunk, returning the indexed
|
||||
match if successful, and <code>false</code> otherwise.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">match: </span><span class="p">(</span><span class="nx">regex</span><span class="p">,</span> <span class="nx">index</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">m: </span><span class="err">@</span><span class="nx">chunk</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">regex</span><span class="p">)</span>
|
||||
<span class="k">if</span> <span class="nx">m</span> <span class="k">then</span> <span class="nx">m</span><span class="p">[</span><span class="nx">index</span><span class="p">]</span> <span class="k">else</span> <span class="kc">false</span></pre></div> </td> </tr> <tr id="section-48"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-48">#</a> </div> <h2>Utility Functions</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-49"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-49">#</a> </div> <p>Does a list include a value?</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">include: </span><span class="p">(</span><span class="nx">list</span><span class="p">,</span> <span class="nx">value</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="nx">list</span><span class="p">.</span><span class="nx">indexOf</span><span class="p">(</span><span class="nx">value</span><span class="p">)</span> <span class="o">>=</span> <span class="mi">0</span></pre></div> </td> </tr> <tr id="section-50"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-50">#</a> </div> <p>Peek at the beginning of a given string to see if it matches a sequence.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">starts: </span><span class="p">(</span><span class="nx">string</span><span class="p">,</span> <span class="nx">literal</span><span class="p">,</span> <span class="nx">start</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="nx">string</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="nx">start</span><span class="p">,</span> <span class="p">(</span><span class="nx">start</span> <span class="o">or</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="nx">literal</span><span class="p">.</span><span class="nx">length</span><span class="p">)</span> <span class="o">is</span> <span class="nx">literal</span></pre></div> </td> </tr> <tr id="section-51"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-51">#</a> </div> <p>Trim out all falsy values from an array.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">compact: </span><span class="p">(</span><span class="nx">array</span><span class="p">)</span> <span class="o">-></span> <span class="nx">item</span> <span class="k">for</span> <span class="nx">item</span> <span class="k">in</span> <span class="nx">array</span> <span class="k">when</span> <span class="nx">item</span></pre></div> </td> </tr> <tr id="section-52"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-52">#</a> </div> <p>Count the number of occurences of a character in a string.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">count: </span><span class="p">(</span><span class="nx">string</span><span class="p">,</span> <span class="nx">letter</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="k">if</span> <span class="nx">m</span> <span class="k">then</span> <span class="nx">m</span><span class="p">[</span><span class="nx">index</span><span class="p">]</span> <span class="k">else</span> <span class="kc">false</span></pre></div> </td> </tr> <tr id="section-49"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-49">#</a> </div> <h2>Utility Functions</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-50"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-50">#</a> </div> <p>Does a list include a value?</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">include: </span><span class="p">(</span><span class="nx">list</span><span class="p">,</span> <span class="nx">value</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="nx">list</span><span class="p">.</span><span class="nx">indexOf</span><span class="p">(</span><span class="nx">value</span><span class="p">)</span> <span class="o">>=</span> <span class="mi">0</span></pre></div> </td> </tr> <tr id="section-51"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-51">#</a> </div> <p>Peek at the beginning of a given string to see if it matches a sequence.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">starts: </span><span class="p">(</span><span class="nx">string</span><span class="p">,</span> <span class="nx">literal</span><span class="p">,</span> <span class="nx">start</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="nx">string</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="nx">start</span><span class="p">,</span> <span class="p">(</span><span class="nx">start</span> <span class="o">or</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="nx">literal</span><span class="p">.</span><span class="nx">length</span><span class="p">)</span> <span class="o">is</span> <span class="nx">literal</span></pre></div> </td> </tr> <tr id="section-52"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-52">#</a> </div> <p>Trim out all falsy values from an array.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">compact: </span><span class="p">(</span><span class="nx">array</span><span class="p">)</span> <span class="o">-></span> <span class="nx">item</span> <span class="k">for</span> <span class="nx">item</span> <span class="k">in</span> <span class="nx">array</span> <span class="k">when</span> <span class="nx">item</span></pre></div> </td> </tr> <tr id="section-53"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-53">#</a> </div> <p>Count the number of occurences of a character in a string.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">count: </span><span class="p">(</span><span class="nx">string</span><span class="p">,</span> <span class="nx">letter</span><span class="p">)</span> <span class="o">-></span>
|
||||
<span class="nv">num: </span><span class="mi">0</span>
|
||||
<span class="nv">pos: </span><span class="nx">string</span><span class="p">.</span><span class="nx">indexOf</span><span class="p">(</span><span class="nx">letter</span><span class="p">)</span>
|
||||
<span class="k">while</span> <span class="nx">pos</span> <span class="o">isnt</span> <span class="o">-</span><span class="mi">1</span>
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue