mirror of
https://github.com/jashkenas/coffeescript.git
synced 2022-11-09 12:23:24 -05:00
422 lines
No EOL
96 KiB
HTML
422 lines
No EOL
96 KiB
HTML
<!DOCTYPE html> <html> <head> <title>lexer.coffee</title> <meta http-equiv="content-type" content="text/html; charset=UTF-8"> <link rel="stylesheet" media="all" href="docco.css" /> </head> <body> <div id="container"> <div id="background"></div> <div id="jump_to"> Jump To … <div id="jump_wrapper"> <div id="jump_page"> <a class="source" href="cake.html"> cake.coffee </a> <a class="source" href="coffee-script.html"> coffee-script.coffee </a> <a class="source" href="command.html"> command.coffee </a> <a class="source" href="grammar.html"> grammar.coffee </a> <a class="source" href="helpers.html"> helpers.coffee </a> <a class="source" href="index.html"> index.coffee </a> <a class="source" href="lexer.html"> lexer.coffee </a> <a class="source" href="nodes.html"> nodes.coffee </a> <a class="source" href="optparse.html"> optparse.coffee </a> <a class="source" href="repl.html"> repl.coffee </a> <a class="source" href="rewriter.html"> rewriter.coffee </a> <a class="source" href="scope.html"> scope.coffee </a> </div> </div> </div> <table cellpadding="0" cellspacing="0"> <thead> <tr> <th class="docs"> <h1> lexer.coffee </h1> </th> <th class="code"> </th> </tr> </thead> <tbody> <tr id="section-1"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-1">#</a> </div> <p>The CoffeeScript Lexer. Uses a series of token-matching regexes to attempt
|
|
matches against the beginning of the source code. When a match is found,
|
|
a token is produced, we consume the match, and start again. Tokens are in the
|
|
form:</p>
|
|
|
|
<pre><code>[tag, value, lineNumber]
|
|
</code></pre>
|
|
|
|
<p>Which is a format that can be fed directly into <a href="http://github.com/zaach/jison">Jison</a>.</p> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-2"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-2">#</a> </div> <p>Set up the Lexer for both Node.js and the browser, depending on where we are.</p> </td> <td class="code"> <div class="highlight"><pre><span class="k">if</span> <span class="nx">process</span><span class="o">?</span>
|
|
<span class="p">{</span><span class="nx">Rewriter</span><span class="p">}</span><span class="o">:</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'./rewriter'</span><span class="p">)</span>
|
|
<span class="p">{</span><span class="nx">helpers</span><span class="p">}</span><span class="o">:</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'./helpers'</span><span class="p">)</span>
|
|
<span class="k">else</span>
|
|
<span class="k">this</span><span class="p">.</span><span class="nv">exports: </span><span class="k">this</span>
|
|
<span class="nv">Rewriter: </span> <span class="k">this</span><span class="p">.</span><span class="nx">Rewriter</span>
|
|
<span class="nv">helpers: </span> <span class="k">this</span><span class="p">.</span><span class="nx">helpers</span></pre></div> </td> </tr> <tr id="section-3"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-3">#</a> </div> <p>Import the helpers we need.</p> </td> <td class="code"> <div class="highlight"><pre><span class="p">{</span><span class="nx">include</span><span class="p">,</span> <span class="nx">count</span><span class="p">,</span> <span class="nx">starts</span><span class="p">,</span> <span class="nx">compact</span><span class="p">}</span><span class="o">:</span> <span class="nx">helpers</span></pre></div> </td> </tr> <tr id="section-4"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-4">#</a> </div> <h2>The Lexer Class</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-5"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-5">#</a> </div> <p>The Lexer class reads a stream of CoffeeScript and divvys it up into tagged
|
|
tokens. Some potential ambiguity in the grammar has been avoided by
|
|
pushing some extra smarts into the Lexer.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">exports.Lexer: </span><span class="nx">class</span> <span class="nx">Lexer</span></pre></div> </td> </tr> <tr id="section-6"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-6">#</a> </div> <p><strong>tokenize</strong> is the Lexer's main method. Scan by attempting to match tokens
|
|
one at a time, using a regular expression anchored at the start of the
|
|
remaining code, or a custom recursive token-matching method
|
|
(for interpolations). When the next token has been recorded, we move forward
|
|
within the code past the token, and begin again.</p>
|
|
|
|
<p>Each tokenizing method is responsible for incrementing <code>@i</code> by the number of
|
|
characters it has consumed. <code>@i</code> can be thought of as our finger on the page
|
|
of source.</p>
|
|
|
|
<p>Before returning the token stream, run it through the <a href="rewriter.html">Rewriter</a>
|
|
unless explicitly asked not to.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tokenize: </span><span class="p">(</span><span class="nx">code</span><span class="p">,</span> <span class="nx">options</span><span class="p">)</span> <span class="o">-></span>
|
|
<span class="nv">code : </span><span class="nx">code</span><span class="p">.</span><span class="nx">replace</span> <span class="sr">/(\r|\s+$)/g</span><span class="p">,</span> <span class="s1">''</span>
|
|
<span class="nv">o : </span><span class="nx">options</span> <span class="o">or</span> <span class="p">{}</span>
|
|
<span class="vi">@code : </span><span class="nx">code</span> <span class="c1"># The remainder of the source code.</span>
|
|
<span class="vi">@i : </span><span class="mi">0</span> <span class="c1"># Current character position we're parsing.</span>
|
|
<span class="vi">@line : </span><span class="nx">o</span><span class="p">.</span><span class="nx">line</span> <span class="o">or</span> <span class="mi">0</span> <span class="c1"># The current line.</span>
|
|
<span class="vi">@indent : </span><span class="mi">0</span> <span class="c1"># The current indentation level.</span>
|
|
<span class="vi">@outdebt : </span><span class="mi">0</span> <span class="c1"># The under-outdentation of the last outdent.</span>
|
|
<span class="vi">@indents : </span><span class="p">[]</span> <span class="c1"># The stack of all current indentation levels.</span>
|
|
<span class="vi">@tokens : </span><span class="p">[]</span> <span class="c1"># Stream of parsed tokens in the form ['TYPE', value, line]</span>
|
|
<span class="k">while</span> <span class="nx">@i</span> <span class="o"><</span> <span class="nx">@code</span><span class="p">.</span><span class="nx">length</span>
|
|
<span class="vi">@chunk: </span><span class="nx">@code</span><span class="p">.</span><span class="nx">slice</span> <span class="nx">@i</span>
|
|
<span class="nx">@extractNextToken</span><span class="p">()</span>
|
|
<span class="nx">@closeIndentation</span><span class="p">()</span>
|
|
<span class="k">return</span> <span class="nx">@tokens</span> <span class="k">if</span> <span class="nx">o</span><span class="p">.</span><span class="nx">rewrite</span> <span class="o">is</span> <span class="kc">off</span>
|
|
<span class="p">(</span><span class="k">new</span> <span class="nx">Rewriter</span><span class="p">).</span><span class="nx">rewrite</span> <span class="nx">@tokens</span></pre></div> </td> </tr> <tr id="section-7"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-7">#</a> </div> <p>At every position, run through this list of attempted matches,
|
|
short-circuiting if any of them succeed. Their order determines precedence:
|
|
<code>@literalToken</code> is the fallback catch-all.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">extractNextToken: </span><span class="o">-></span>
|
|
<span class="k">return</span> <span class="k">if</span> <span class="nx">@identifierToken</span><span class="p">()</span>
|
|
<span class="k">return</span> <span class="k">if</span> <span class="nx">@commentToken</span><span class="p">()</span>
|
|
<span class="k">return</span> <span class="k">if</span> <span class="nx">@whitespaceToken</span><span class="p">()</span>
|
|
<span class="k">return</span> <span class="k">if</span> <span class="nx">@lineToken</span><span class="p">()</span>
|
|
<span class="k">return</span> <span class="k">if</span> <span class="nx">@heredocToken</span><span class="p">()</span>
|
|
<span class="k">return</span> <span class="k">if</span> <span class="nx">@stringToken</span><span class="p">()</span>
|
|
<span class="k">return</span> <span class="k">if</span> <span class="nx">@numberToken</span><span class="p">()</span>
|
|
<span class="k">return</span> <span class="k">if</span> <span class="nx">@regexToken</span><span class="p">()</span>
|
|
<span class="k">return</span> <span class="k">if</span> <span class="nx">@jsToken</span><span class="p">()</span>
|
|
<span class="k">return</span> <span class="nx">@literalToken</span><span class="p">()</span></pre></div> </td> </tr> <tr id="section-8"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-8">#</a> </div> <h2>Tokenizers</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-9"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-9">#</a> </div> <p>Matches identifying literals: variables, keywords, method names, etc.
|
|
Check to ensure that JavaScript reserved words aren't being used as
|
|
identifiers. Because CoffeeScript reserves a handful of keywords that are
|
|
allowed in JavaScript, we're careful not to tag them as keywords when
|
|
referenced as property names here, so you can still do <code>jQuery.is()</code> even
|
|
though <code>is</code> means <code>===</code> otherwise.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">identifierToken: </span><span class="o">-></span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">id: </span><span class="nx">@match</span> <span class="nx">IDENTIFIER</span><span class="p">,</span> <span class="mi">1</span>
|
|
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">id</span><span class="p">.</span><span class="nx">length</span>
|
|
<span class="nv">forcedIdentifier: </span><span class="nx">@tagAccessor</span><span class="p">()</span> <span class="o">or</span> <span class="nx">@match</span> <span class="nx">ASSIGNED</span><span class="p">,</span> <span class="mi">1</span>
|
|
<span class="nv">tag: </span><span class="s1">'IDENTIFIER'</span>
|
|
<span class="nv">tag: </span><span class="nx">id</span><span class="p">.</span><span class="nx">toUpperCase</span><span class="p">()</span> <span class="k">if</span> <span class="nx">include</span><span class="p">(</span><span class="nx">JS_KEYWORDS</span><span class="p">,</span> <span class="nx">id</span><span class="p">)</span> <span class="o">or</span> <span class="p">(</span><span class="o">not</span> <span class="nx">forcedIdentifier</span> <span class="o">and</span> <span class="nx">include</span><span class="p">(</span><span class="nx">COFFEE_KEYWORDS</span><span class="p">,</span> <span class="nx">id</span><span class="p">))</span>
|
|
<span class="nv">tag: </span><span class="s1">'LEADING_WHEN'</span> <span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s1">'WHEN'</span> <span class="o">and</span> <span class="nx">include</span> <span class="nx">LINE_BREAK</span><span class="p">,</span> <span class="nx">@tag</span><span class="p">()</span>
|
|
<span class="k">if</span> <span class="nx">include</span><span class="p">(</span><span class="nx">JS_FORBIDDEN</span><span class="p">,</span> <span class="nx">id</span><span class="p">)</span>
|
|
<span class="k">if</span> <span class="nx">forcedIdentifier</span>
|
|
<span class="nv">tag: </span><span class="s1">'STRING'</span>
|
|
<span class="nv">id: </span> <span class="s2">"'$id'"</span>
|
|
<span class="k">if</span> <span class="nx">forcedIdentifier</span> <span class="o">is</span> <span class="s1">'accessor'</span>
|
|
<span class="nv">close_index: </span><span class="kc">true</span>
|
|
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span> <span class="k">if</span> <span class="nx">@tag</span><span class="p">()</span> <span class="o">isnt</span> <span class="s1">'@'</span>
|
|
<span class="nx">@token</span> <span class="s1">'INDEX_START'</span><span class="p">,</span> <span class="s1">'['</span>
|
|
<span class="k">else</span> <span class="k">if</span> <span class="nx">include</span><span class="p">(</span><span class="nx">RESERVED</span><span class="p">,</span> <span class="nx">id</span><span class="p">)</span>
|
|
<span class="nx">@identifierError</span> <span class="nx">id</span>
|
|
<span class="nx">unless</span> <span class="nx">forcedIdentifier</span>
|
|
<span class="nv">tag: id: </span><span class="nx">CONVERSIONS</span><span class="p">[</span><span class="nx">id</span><span class="p">]</span> <span class="k">if</span> <span class="nx">include</span> <span class="nx">COFFEE_ALIASES</span><span class="p">,</span> <span class="nx">id</span>
|
|
<span class="k">return</span> <span class="nx">@tagHalfAssignment</span> <span class="nx">tag</span> <span class="k">if</span> <span class="nx">@prev</span><span class="p">()</span> <span class="o">and</span> <span class="nx">@prev</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">'ASSIGN'</span> <span class="o">and</span> <span class="nx">include</span> <span class="nx">HALF_ASSIGNMENTS</span><span class="p">,</span> <span class="nx">tag</span>
|
|
<span class="nx">@token</span> <span class="nx">tag</span><span class="p">,</span> <span class="nx">id</span>
|
|
<span class="nx">@token</span> <span class="s1">']'</span><span class="p">,</span> <span class="s1">']'</span> <span class="k">if</span> <span class="nx">close_index</span>
|
|
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-10"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-10">#</a> </div> <p>Matches numbers, including decimals, hex, and exponential notation.
|
|
Be careful not to interfere with ranges-in-progress.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">numberToken: </span><span class="o">-></span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">number: </span><span class="nx">@match</span> <span class="nx">NUMBER</span><span class="p">,</span> <span class="mi">1</span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="k">if</span> <span class="nx">@tag</span><span class="p">()</span> <span class="o">is</span> <span class="s1">'.'</span> <span class="o">and</span> <span class="nx">starts</span> <span class="nx">number</span><span class="p">,</span> <span class="s1">'.'</span>
|
|
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">number</span><span class="p">.</span><span class="nx">length</span>
|
|
<span class="nx">@token</span> <span class="s1">'NUMBER'</span><span class="p">,</span> <span class="nx">number</span>
|
|
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-11"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-11">#</a> </div> <p>Matches strings, including multi-line strings. Ensures that quotation marks
|
|
are balanced within the string's contents, and within nested interpolations.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">stringToken: </span><span class="o">-></span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nx">starts</span><span class="p">(</span><span class="nx">@chunk</span><span class="p">,</span> <span class="s1">'"'</span><span class="p">)</span> <span class="o">or</span> <span class="nx">starts</span><span class="p">(</span><span class="nx">@chunk</span><span class="p">,</span> <span class="s2">"'"</span><span class="p">)</span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">string:</span>
|
|
<span class="nx">@balancedToken</span><span class="p">([</span><span class="s1">'"'</span><span class="p">,</span> <span class="s1">'"'</span><span class="p">],</span> <span class="p">[</span><span class="s1">'${'</span><span class="p">,</span> <span class="s1">'}'</span><span class="p">])</span> <span class="o">or</span>
|
|
<span class="nx">@balancedToken</span> <span class="p">[</span><span class="s2">"'"</span><span class="p">,</span> <span class="s2">"'"</span><span class="p">]</span>
|
|
<span class="nx">@interpolateString</span> <span class="nx">string</span><span class="p">.</span><span class="nx">replace</span> <span class="nx">STRING_NEWLINES</span><span class="p">,</span> <span class="s2">" \\\n"</span>
|
|
<span class="vi">@line: </span><span class="o">+</span> <span class="nx">count</span> <span class="nx">string</span><span class="p">,</span> <span class="s2">"\n"</span>
|
|
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">string</span><span class="p">.</span><span class="nx">length</span>
|
|
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-12"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-12">#</a> </div> <p>Matches heredocs, adjusting indentation to the correct level, as heredocs
|
|
preserve whitespace, but ignore indentation to the left.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">heredocToken: </span><span class="o">-></span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">match: </span><span class="nx">@chunk</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">HEREDOC</span><span class="p">)</span>
|
|
<span class="nv">quote: </span><span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">].</span><span class="nx">substr</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span>
|
|
<span class="nv">doc: </span><span class="nx">@sanitizeHeredoc</span> <span class="nx">match</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">or</span> <span class="nx">match</span><span class="p">[</span><span class="mi">4</span><span class="p">],</span> <span class="p">{</span><span class="nx">quote</span><span class="p">}</span>
|
|
<span class="nx">@interpolateString</span> <span class="s2">"$quote$doc$quote"</span>
|
|
<span class="vi">@line: </span><span class="o">+</span> <span class="nx">count</span> <span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="s2">"\n"</span>
|
|
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">].</span><span class="nx">length</span>
|
|
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-13"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-13">#</a> </div> <p>Matches and conumes comments.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">commentToken: </span><span class="o">-></span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">match: </span><span class="nx">@chunk</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">COMMENT</span><span class="p">)</span>
|
|
<span class="vi">@line: </span><span class="o">+</span> <span class="nx">count</span> <span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="s2">"\n"</span>
|
|
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">].</span><span class="nx">length</span>
|
|
<span class="k">if</span> <span class="nx">match</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
|
|
<span class="nv">comment: </span><span class="nx">@sanitizeHeredoc</span> <span class="nx">match</span><span class="p">[</span><span class="mi">2</span><span class="p">],</span> <span class="p">{</span><span class="nv">herecomment: </span><span class="kc">true</span><span class="p">}</span>
|
|
<span class="nx">@token</span> <span class="s1">'HERECOMMENT'</span><span class="p">,</span> <span class="nx">comment</span><span class="p">.</span><span class="nx">split</span> <span class="nx">MULTILINER</span>
|
|
<span class="nx">@token</span> <span class="s1">'TERMINATOR'</span><span class="p">,</span> <span class="s1">'\n'</span>
|
|
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-14"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-14">#</a> </div> <p>Matches JavaScript interpolated directly into the source via backticks.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">jsToken: </span><span class="o">-></span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nx">starts</span> <span class="nx">@chunk</span><span class="p">,</span> <span class="s1">'`'</span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">script: </span><span class="nx">@balancedToken</span> <span class="p">[</span><span class="s1">'`'</span><span class="p">,</span> <span class="s1">'`'</span><span class="p">]</span>
|
|
<span class="nx">@token</span> <span class="s1">'JS'</span><span class="p">,</span> <span class="nx">script</span><span class="p">.</span><span class="nx">replace</span> <span class="nx">JS_CLEANER</span><span class="p">,</span> <span class="s1">''</span>
|
|
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">script</span><span class="p">.</span><span class="nx">length</span>
|
|
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-15"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-15">#</a> </div> <p>Matches regular expression literals. Lexing regular expressions is difficult
|
|
to distinguish from division, so we borrow some basic heuristics from
|
|
JavaScript and Ruby, borrow slash balancing from <code>@balancedToken</code>, and
|
|
borrow interpolation from <code>@interpolateString</code>.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">regexToken: </span><span class="o">-></span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nx">@chunk</span><span class="p">.</span><span class="nx">match</span> <span class="nx">REGEX_START</span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="k">if</span> <span class="nx">include</span> <span class="nx">NOT_REGEX</span><span class="p">,</span> <span class="nx">@tag</span><span class="p">()</span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">regex: </span><span class="nx">@balancedToken</span> <span class="p">[</span><span class="s1">'/'</span><span class="p">,</span> <span class="s1">'/'</span><span class="p">]</span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">end: </span><span class="nx">@chunk</span><span class="p">.</span><span class="nx">substr</span><span class="p">(</span><span class="nx">regex</span><span class="p">.</span><span class="nx">length</span><span class="p">).</span><span class="nx">match</span> <span class="nx">REGEX_END</span>
|
|
<span class="nv">regex: </span><span class="o">+</span> <span class="nv">flags: </span><span class="nx">end</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="k">if</span> <span class="nx">end</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
|
|
<span class="k">if</span> <span class="nx">regex</span><span class="p">.</span><span class="nx">match</span> <span class="nx">REGEX_INTERPOLATION</span>
|
|
<span class="nv">str: </span><span class="nx">regex</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">1</span><span class="p">).</span><span class="nx">split</span><span class="p">(</span><span class="s1">'/'</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
|
|
<span class="nv">str: </span><span class="nx">str</span><span class="p">.</span><span class="nx">replace</span> <span class="nx">REGEX_ESCAPE</span><span class="p">,</span> <span class="p">(</span><span class="nx">escaped</span><span class="p">)</span> <span class="o">-></span> <span class="s1">'\\'</span> <span class="o">+</span> <span class="nx">escaped</span>
|
|
<span class="vi">@tokens: </span><span class="nx">@tokens</span><span class="p">.</span><span class="nx">concat</span> <span class="p">[[</span><span class="s1">'('</span><span class="p">,</span> <span class="s1">'('</span><span class="p">],</span> <span class="p">[</span><span class="s1">'NEW'</span><span class="p">,</span> <span class="s1">'new'</span><span class="p">],</span> <span class="p">[</span><span class="s1">'IDENTIFIER'</span><span class="p">,</span> <span class="s1">'RegExp'</span><span class="p">],</span> <span class="p">[</span><span class="s1">'CALL_START'</span><span class="p">,</span> <span class="s1">'('</span><span class="p">]]</span>
|
|
<span class="nx">@interpolateString</span> <span class="s2">"\"$str\""</span><span class="p">,</span> <span class="kc">yes</span>
|
|
<span class="vi">@tokens: </span><span class="nx">@tokens</span><span class="p">.</span><span class="nx">concat</span> <span class="p">[[</span><span class="s1">','</span><span class="p">,</span> <span class="s1">','</span><span class="p">],</span> <span class="p">[</span><span class="s1">'STRING'</span><span class="p">,</span> <span class="s2">"\"$flags\""</span><span class="p">],</span> <span class="p">[</span><span class="s1">')'</span><span class="p">,</span> <span class="s1">')'</span><span class="p">],</span> <span class="p">[</span><span class="s1">')'</span><span class="p">,</span> <span class="s1">')'</span><span class="p">]]</span>
|
|
<span class="k">else</span>
|
|
<span class="nx">@token</span> <span class="s1">'REGEX'</span><span class="p">,</span> <span class="nx">regex</span>
|
|
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">regex</span><span class="p">.</span><span class="nx">length</span>
|
|
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-16"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-16">#</a> </div> <p>Matches a token in which which the passed delimiter pairs must be correctly
|
|
balanced (ie. strings, JS literals).</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">balancedToken: </span><span class="p">(</span><span class="nx">delimited</span><span class="p">...)</span> <span class="o">-></span>
|
|
<span class="nx">@balancedString</span> <span class="nx">@chunk</span><span class="p">,</span> <span class="nx">delimited</span></pre></div> </td> </tr> <tr id="section-17"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-17">#</a> </div> <p>Matches newlines, indents, and outdents, and determines which is which.
|
|
If we can detect that the current line is continued onto the the next line,
|
|
then the newline is suppressed:</p>
|
|
|
|
<pre><code>elements
|
|
.each( ... )
|
|
.map( ... )
|
|
</code></pre>
|
|
|
|
<p>Keeps track of the level of indentation, because a single outdent token
|
|
can close multiple indents, so we need to know how far in we happen to be.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">lineToken: </span><span class="o">-></span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">indent: </span><span class="nx">@match</span> <span class="nx">MULTI_DENT</span><span class="p">,</span> <span class="mi">1</span>
|
|
<span class="vi">@line: </span><span class="o">+</span> <span class="nx">count</span> <span class="nx">indent</span><span class="p">,</span> <span class="s2">"\n"</span>
|
|
<span class="vi">@i : </span><span class="o">+</span> <span class="nx">indent</span><span class="p">.</span><span class="nx">length</span>
|
|
<span class="nv">prev: </span><span class="nx">@prev</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
|
|
<span class="nv">size: </span><span class="nx">indent</span><span class="p">.</span><span class="nx">match</span><span class="p">(</span><span class="nx">LAST_DENTS</span><span class="p">).</span><span class="nx">reverse</span><span class="p">()[</span><span class="mi">0</span><span class="p">].</span><span class="nx">match</span><span class="p">(</span><span class="nx">LAST_DENT</span><span class="p">)[</span><span class="mi">1</span><span class="p">].</span><span class="nx">length</span>
|
|
<span class="nv">nextCharacter: </span><span class="nx">@match</span> <span class="nx">NEXT_CHARACTER</span><span class="p">,</span> <span class="mi">1</span>
|
|
<span class="nv">noNewlines: </span><span class="nx">nextCharacter</span> <span class="o">is</span> <span class="s1">'.'</span> <span class="o">or</span> <span class="nx">nextCharacter</span> <span class="o">is</span> <span class="s1">','</span> <span class="o">or</span> <span class="nx">@unfinished</span><span class="p">()</span>
|
|
<span class="k">if</span> <span class="nx">size</span> <span class="o">is</span> <span class="nx">@indent</span>
|
|
<span class="k">return</span> <span class="nx">@suppressNewlines</span><span class="p">()</span> <span class="k">if</span> <span class="nx">noNewlines</span>
|
|
<span class="k">return</span> <span class="nx">@newlineToken</span> <span class="nx">indent</span>
|
|
<span class="k">else</span> <span class="k">if</span> <span class="nx">size</span> <span class="o">></span> <span class="nx">@indent</span>
|
|
<span class="k">return</span> <span class="nx">@suppressNewlines</span><span class="p">()</span> <span class="k">if</span> <span class="nx">noNewlines</span>
|
|
<span class="nv">diff: </span><span class="nx">size</span> <span class="o">-</span> <span class="nx">@indent</span>
|
|
<span class="nx">@token</span> <span class="s1">'INDENT'</span><span class="p">,</span> <span class="nx">diff</span>
|
|
<span class="nx">@indents</span><span class="p">.</span><span class="nx">push</span> <span class="nx">diff</span>
|
|
<span class="k">else</span>
|
|
<span class="nx">@outdentToken</span> <span class="nx">@indent</span> <span class="o">-</span> <span class="nx">size</span><span class="p">,</span> <span class="nx">noNewlines</span>
|
|
<span class="vi">@indent: </span><span class="nx">size</span>
|
|
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-18"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-18">#</a> </div> <p>Record an outdent token or multiple tokens, if we happen to be moving back
|
|
inwards past several recorded indents.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">outdentToken: </span><span class="p">(</span><span class="nx">moveOut</span><span class="p">,</span> <span class="nx">noNewlines</span><span class="p">)</span> <span class="o">-></span>
|
|
<span class="k">if</span> <span class="nx">moveOut</span> <span class="o">></span> <span class="o">-</span><span class="nx">@outdebt</span>
|
|
<span class="k">while</span> <span class="nx">moveOut</span> <span class="o">></span> <span class="mi">0</span> <span class="o">and</span> <span class="nx">@indents</span><span class="p">.</span><span class="nx">length</span>
|
|
<span class="nv">lastIndent: </span><span class="nx">@indents</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
|
|
<span class="nx">@token</span> <span class="s1">'OUTDENT'</span><span class="p">,</span> <span class="nx">lastIndent</span>
|
|
<span class="nv">moveOut: </span><span class="o">-</span> <span class="nx">lastIndent</span>
|
|
<span class="k">else</span>
|
|
<span class="vi">@outdebt: </span><span class="o">+</span> <span class="nx">moveOut</span>
|
|
<span class="vi">@outdebt: </span><span class="nx">moveOut</span> <span class="nx">unless</span> <span class="nx">noNewlines</span>
|
|
<span class="nx">@token</span> <span class="s1">'TERMINATOR'</span><span class="p">,</span> <span class="s2">"\n"</span> <span class="nx">unless</span> <span class="nx">@tag</span><span class="p">()</span> <span class="o">is</span> <span class="s1">'TERMINATOR'</span> <span class="o">or</span> <span class="nx">noNewlines</span>
|
|
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-19"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-19">#</a> </div> <p>Matches and consumes non-meaningful whitespace. Tag the previous token
|
|
as being "spaced", because there are some cases where it makes a difference.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">whitespaceToken: </span><span class="o">-></span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">space: </span><span class="nx">@match</span> <span class="nx">WHITESPACE</span><span class="p">,</span> <span class="mi">1</span>
|
|
<span class="nv">prev: </span><span class="nx">@prev</span><span class="p">()</span>
|
|
<span class="nv">prev.spaced: </span><span class="kc">true</span> <span class="k">if</span> <span class="nx">prev</span>
|
|
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">space</span><span class="p">.</span><span class="nx">length</span>
|
|
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-20"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-20">#</a> </div> <p>Generate a newline token. Consecutive newlines get merged together.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">newlineToken: </span><span class="p">(</span><span class="nx">newlines</span><span class="p">)</span> <span class="o">-></span>
|
|
<span class="nx">@token</span> <span class="s1">'TERMINATOR'</span><span class="p">,</span> <span class="s2">"\n"</span> <span class="nx">unless</span> <span class="nx">@tag</span><span class="p">()</span> <span class="o">is</span> <span class="s1">'TERMINATOR'</span>
|
|
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-21"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-21">#</a> </div> <p>Use a <code>\</code> at a line-ending to suppress the newline.
|
|
The slash is removed here once its job is done.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">suppressNewlines: </span><span class="o">-></span>
|
|
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span> <span class="k">if</span> <span class="nx">@value</span><span class="p">()</span> <span class="o">is</span> <span class="s2">"\\"</span>
|
|
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-22"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-22">#</a> </div> <p>We treat all other single characters as a token. Eg.: <code>( ) , . !</code>
|
|
Multi-character operators are also literal tokens, so that Jison can assign
|
|
the proper order of operations. There are some symbols that we tag specially
|
|
here. <code>;</code> and newlines are both treated as a <code>TERMINATOR</code>, we distinguish
|
|
parentheses that indicate a method call from regular parentheses, and so on.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">literalToken: </span><span class="o">-></span>
|
|
<span class="nv">match: </span><span class="nx">@chunk</span><span class="p">.</span><span class="nx">match</span> <span class="nx">OPERATOR</span>
|
|
<span class="nv">value: </span><span class="nx">match</span> <span class="o">and</span> <span class="nx">match</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
|
|
<span class="nv">space: </span><span class="nx">match</span> <span class="o">and</span> <span class="nx">match</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
|
|
<span class="nx">@tagParameters</span><span class="p">()</span> <span class="k">if</span> <span class="nx">value</span> <span class="o">and</span> <span class="nx">value</span><span class="p">.</span><span class="nx">match</span> <span class="nx">CODE</span>
|
|
<span class="nv">value: </span><span class="o">or</span> <span class="nx">@chunk</span><span class="p">.</span><span class="nx">substr</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span>
|
|
<span class="nv">prevSpaced: </span><span class="nx">@prev</span><span class="p">()</span> <span class="o">and</span> <span class="nx">@prev</span><span class="p">().</span><span class="nx">spaced</span>
|
|
<span class="nv">tag: </span><span class="nx">value</span>
|
|
<span class="k">if</span> <span class="nx">value</span><span class="p">.</span><span class="nx">match</span> <span class="nx">ASSIGNMENT</span>
|
|
<span class="nv">tag: </span><span class="s1">'ASSIGN'</span>
|
|
<span class="nx">@assignmentError</span><span class="p">()</span> <span class="k">if</span> <span class="nx">include</span> <span class="nx">JS_FORBIDDEN</span><span class="p">,</span> <span class="nx">@value</span>
|
|
<span class="k">else</span> <span class="k">if</span> <span class="nx">value</span> <span class="o">is</span> <span class="s1">';'</span>
|
|
<span class="nv">tag: </span><span class="s1">'TERMINATOR'</span>
|
|
<span class="k">else</span> <span class="k">if</span> <span class="nx">value</span> <span class="o">is</span> <span class="s1">'?'</span> <span class="o">and</span> <span class="nx">prevSpaced</span>
|
|
<span class="nv">tag: </span><span class="s1">'OP?'</span>
|
|
<span class="k">else</span> <span class="k">if</span> <span class="nx">include</span><span class="p">(</span><span class="nx">CALLABLE</span><span class="p">,</span> <span class="nx">@tag</span><span class="p">())</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">prevSpaced</span>
|
|
<span class="k">if</span> <span class="nx">value</span> <span class="o">is</span> <span class="s1">'('</span>
|
|
<span class="nv">tag: </span><span class="s1">'CALL_START'</span>
|
|
<span class="k">else</span> <span class="k">if</span> <span class="nx">value</span> <span class="o">is</span> <span class="s1">'['</span>
|
|
<span class="nv">tag: </span><span class="s1">'INDEX_START'</span>
|
|
<span class="nx">@tag</span> <span class="mi">1</span><span class="p">,</span> <span class="s1">'INDEX_SOAK'</span> <span class="k">if</span> <span class="nx">@tag</span><span class="p">()</span> <span class="o">is</span> <span class="s1">'?'</span>
|
|
<span class="nx">@tag</span> <span class="mi">1</span><span class="p">,</span> <span class="s1">'INDEX_PROTO'</span> <span class="k">if</span> <span class="nx">@tag</span><span class="p">()</span> <span class="o">is</span> <span class="s1">'::'</span>
|
|
<span class="vi">@i: </span><span class="o">+</span> <span class="nx">value</span><span class="p">.</span><span class="nx">length</span>
|
|
<span class="k">return</span> <span class="nx">@tagHalfAssignment</span> <span class="nx">tag</span> <span class="k">if</span> <span class="nx">space</span> <span class="o">and</span> <span class="nx">prevSpaced</span> <span class="o">and</span> <span class="nx">@prev</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">'ASSIGN'</span> <span class="o">and</span> <span class="nx">include</span> <span class="nx">HALF_ASSIGNMENTS</span><span class="p">,</span> <span class="nx">tag</span>
|
|
<span class="nx">@token</span> <span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span>
|
|
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-23"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-23">#</a> </div> <h2>Token Manipulators</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-24"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-24">#</a> </div> <p>As we consume a new <code>IDENTIFIER</code>, look at the previous token to determine
|
|
if it's a special kind of accessor. Return <code>true</code> if any type of accessor
|
|
is the previous token.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tagAccessor: </span><span class="o">-></span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="k">if</span> <span class="p">(</span><span class="o">not</span> <span class="nv">prev: </span><span class="nx">@prev</span><span class="p">())</span> <span class="o">or</span> <span class="p">(</span><span class="nx">prev</span> <span class="o">and</span> <span class="nx">prev</span><span class="p">.</span><span class="nx">spaced</span><span class="p">)</span>
|
|
<span class="nv">accessor: </span><span class="k">if</span> <span class="nx">prev</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">is</span> <span class="s1">'::'</span>
|
|
<span class="nx">@tag</span> <span class="mi">1</span><span class="p">,</span> <span class="s1">'PROTOTYPE_ACCESS'</span>
|
|
<span class="k">else</span> <span class="k">if</span> <span class="nx">prev</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">is</span> <span class="s1">'.'</span> <span class="o">and</span> <span class="o">not</span> <span class="p">(</span><span class="nx">@value</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="o">is</span> <span class="s1">'.'</span><span class="p">)</span>
|
|
<span class="k">if</span> <span class="nx">@tag</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="o">is</span> <span class="s1">'?'</span>
|
|
<span class="nx">@tag</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="s1">'SOAK_ACCESS'</span><span class="p">)</span>
|
|
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">splice</span><span class="p">(</span><span class="o">-</span><span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
|
|
<span class="k">else</span>
|
|
<span class="nx">@tag</span> <span class="mi">1</span><span class="p">,</span> <span class="s1">'PROPERTY_ACCESS'</span>
|
|
<span class="k">else</span>
|
|
<span class="nx">prev</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">'@'</span>
|
|
<span class="k">if</span> <span class="nx">accessor</span> <span class="k">then</span> <span class="s1">'accessor'</span> <span class="k">else</span> <span class="kc">false</span></pre></div> </td> </tr> <tr id="section-25"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-25">#</a> </div> <p>Sanitize a heredoc or herecomment by escaping internal double quotes and
|
|
erasing all external indentation on the left-hand side.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">sanitizeHeredoc: </span><span class="p">(</span><span class="nx">doc</span><span class="p">,</span> <span class="nx">options</span><span class="p">)</span> <span class="o">-></span>
|
|
<span class="k">while</span> <span class="nv">match: </span><span class="nx">HEREDOC_INDENT</span><span class="p">.</span><span class="nx">exec</span> <span class="nx">doc</span>
|
|
<span class="nv">attempt: </span><span class="k">if</span> <span class="nx">match</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span><span class="o">?</span> <span class="k">then</span> <span class="nx">match</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="k">else</span> <span class="nx">match</span><span class="p">[</span><span class="mi">3</span><span class="p">]</span>
|
|
<span class="nv">indent: </span><span class="nx">attempt</span> <span class="k">if</span> <span class="o">not</span> <span class="nx">indent</span> <span class="o">or</span> <span class="nx">attempt</span><span class="p">.</span><span class="nx">length</span> <span class="o"><</span> <span class="nx">indent</span><span class="p">.</span><span class="nx">length</span>
|
|
<span class="nv">doc: </span><span class="nx">doc</span><span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="k">new</span> <span class="nb">RegExp</span><span class="p">(</span><span class="s2">"^"</span> <span class="o">+</span><span class="nx">indent</span><span class="p">,</span> <span class="s1">'gm'</span><span class="p">),</span> <span class="s1">''</span><span class="p">)</span>
|
|
<span class="k">return</span> <span class="nx">doc</span> <span class="k">if</span> <span class="nx">options</span><span class="p">.</span><span class="nx">herecomment</span>
|
|
<span class="nx">doc</span><span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="nx">MULTILINER</span><span class="p">,</span> <span class="s2">"\\n"</span><span class="p">)</span>
|
|
<span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="k">new</span> <span class="nb">RegExp</span><span class="p">(</span><span class="nx">options</span><span class="p">.</span><span class="nx">quote</span><span class="p">,</span> <span class="s1">'g'</span><span class="p">),</span> <span class="s2">"\\$options.quote"</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-26"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-26">#</a> </div> <p>Tag a half assignment.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tagHalfAssignment: </span><span class="p">(</span><span class="nx">tag</span><span class="p">)</span> <span class="o">-></span>
|
|
<span class="nv">tag: </span> <span class="s1">'?'</span> <span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s1">'OP?'</span>
|
|
<span class="nv">last: </span><span class="nx">@tokens</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
|
|
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s2">"$tag="</span><span class="p">,</span> <span class="s2">"$tag="</span><span class="p">,</span> <span class="nx">last</span><span class="p">[</span><span class="mi">2</span><span class="p">]]</span>
|
|
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-27"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-27">#</a> </div> <p>A source of ambiguity in our grammar used to be parameter lists in function
|
|
definitions versus argument lists in function calls. Walk backwards, tagging
|
|
parameters specially in order to make things easier for the parser.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tagParameters: </span><span class="o">-></span>
|
|
<span class="k">return</span> <span class="k">if</span> <span class="nx">@tag</span><span class="p">()</span> <span class="o">isnt</span> <span class="s1">')'</span>
|
|
<span class="nv">i: </span><span class="mi">0</span>
|
|
<span class="nx">loop</span>
|
|
<span class="nv">i: </span><span class="o">+</span> <span class="mi">1</span>
|
|
<span class="nv">tok: </span><span class="nx">@prev</span> <span class="nx">i</span>
|
|
<span class="k">return</span> <span class="k">if</span> <span class="o">not</span> <span class="nx">tok</span>
|
|
<span class="k">switch</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
|
<span class="k">when</span> <span class="s1">'IDENTIFIER'</span> <span class="k">then</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">:</span> <span class="s1">'PARAM'</span>
|
|
<span class="k">when</span> <span class="s1">')'</span> <span class="k">then</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">:</span> <span class="s1">'PARAM_END'</span>
|
|
<span class="k">when</span> <span class="s1">'('</span><span class="p">,</span> <span class="s1">'CALL_START'</span> <span class="k">then</span> <span class="k">return</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">:</span> <span class="s1">'PARAM_START'</span>
|
|
<span class="kc">true</span></pre></div> </td> </tr> <tr id="section-28"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-28">#</a> </div> <p>Close up all remaining open blocks at the end of the file.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">closeIndentation: </span><span class="o">-></span>
|
|
<span class="nx">@outdentToken</span> <span class="nx">@indent</span></pre></div> </td> </tr> <tr id="section-29"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-29">#</a> </div> <p>The error for when you try to use a forbidden word in JavaScript as
|
|
an identifier.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">identifierError: </span><span class="p">(</span><span class="nx">word</span><span class="p">)</span> <span class="o">-></span>
|
|
<span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span> <span class="s2">"SyntaxError: Reserved word \"$word\" on line ${@line + 1}"</span></pre></div> </td> </tr> <tr id="section-30"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-30">#</a> </div> <p>The error for when you try to assign to a reserved word in JavaScript,
|
|
like "function" or "default".</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">assignmentError: </span><span class="o">-></span>
|
|
<span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span> <span class="s2">"SyntaxError: Reserved word \"${@value()}\" on line ${@line + 1} can't be assigned"</span></pre></div> </td> </tr> <tr id="section-31"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-31">#</a> </div> <p>Matches a balanced group such as a single or double-quoted string. Pass in
|
|
a series of delimiters, all of which must be nested correctly within the
|
|
contents of the string. This method allows us to have strings within
|
|
interpolations within strings, ad infinitum.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">balancedString: </span><span class="p">(</span><span class="nx">str</span><span class="p">,</span> <span class="nx">delimited</span><span class="p">,</span> <span class="nx">options</span><span class="p">)</span> <span class="o">-></span>
|
|
<span class="nv">options: </span><span class="o">or</span> <span class="p">{}</span>
|
|
<span class="nv">slash: </span><span class="nx">delimited</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">'/'</span>
|
|
<span class="nv">levels: </span><span class="p">[]</span>
|
|
<span class="nv">i: </span><span class="mi">0</span>
|
|
<span class="k">while</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">str</span><span class="p">.</span><span class="nx">length</span>
|
|
<span class="k">if</span> <span class="nx">levels</span><span class="p">.</span><span class="nx">length</span> <span class="o">and</span> <span class="nx">starts</span> <span class="nx">str</span><span class="p">,</span> <span class="s1">'\\'</span><span class="p">,</span> <span class="nx">i</span>
|
|
<span class="nv">i: </span><span class="o">+</span> <span class="mi">1</span>
|
|
<span class="k">else</span>
|
|
<span class="k">for</span> <span class="nx">pair</span> <span class="k">in</span> <span class="nx">delimited</span>
|
|
<span class="p">[</span><span class="nx">open</span><span class="p">,</span> <span class="nx">close</span><span class="p">]</span><span class="o">:</span> <span class="nx">pair</span>
|
|
<span class="k">if</span> <span class="nx">levels</span><span class="p">.</span><span class="nx">length</span> <span class="o">and</span> <span class="nx">starts</span><span class="p">(</span><span class="nx">str</span><span class="p">,</span> <span class="nx">close</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span> <span class="o">and</span> <span class="nx">levels</span><span class="p">[</span><span class="nx">levels</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span> <span class="o">is</span> <span class="nx">pair</span>
|
|
<span class="nx">levels</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
|
|
<span class="nv">i: </span><span class="o">+</span> <span class="nx">close</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
|
<span class="nv">i: </span><span class="o">+</span> <span class="mi">1</span> <span class="nx">unless</span> <span class="nx">levels</span><span class="p">.</span><span class="nx">length</span>
|
|
<span class="k">break</span>
|
|
<span class="k">else</span> <span class="k">if</span> <span class="nx">starts</span> <span class="nx">str</span><span class="p">,</span> <span class="nx">open</span><span class="p">,</span> <span class="nx">i</span>
|
|
<span class="nx">levels</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">pair</span><span class="p">)</span>
|
|
<span class="nv">i: </span><span class="o">+</span> <span class="nx">open</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
|
<span class="k">break</span>
|
|
<span class="k">break</span> <span class="k">if</span> <span class="o">not</span> <span class="nx">levels</span><span class="p">.</span><span class="nx">length</span> <span class="o">or</span> <span class="nx">slash</span> <span class="o">and</span> <span class="nx">starts</span> <span class="nx">str</span><span class="p">,</span> <span class="s1">'\n'</span><span class="p">,</span> <span class="nx">i</span>
|
|
<span class="nv">i: </span><span class="o">+</span> <span class="mi">1</span>
|
|
<span class="k">if</span> <span class="nx">levels</span><span class="p">.</span><span class="nx">length</span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="k">if</span> <span class="nx">slash</span>
|
|
<span class="k">throw</span> <span class="k">new</span> <span class="nb">Error</span> <span class="s2">"SyntaxError: Unterminated ${levels.pop()[0]} starting on line ${@line + 1}"</span>
|
|
<span class="k">if</span> <span class="o">not</span> <span class="nx">i</span> <span class="k">then</span> <span class="kc">false</span> <span class="k">else</span> <span class="nx">str</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nx">i</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-32"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-32">#</a> </div> <p>Expand variables and expressions inside double-quoted strings using
|
|
<a href="http://wiki.ecmascript.org/doku.php?id=strawman:string_interpolation">ECMA Harmony's interpolation syntax</a>
|
|
for substitution of bare variables as well as arbitrary expressions.</p>
|
|
|
|
<pre><code>"Hello $name."
|
|
"Hello ${name.capitalize()}."
|
|
</code></pre>
|
|
|
|
<p>If it encounters an interpolation, this method will recursively create a
|
|
new Lexer, tokenize the interpolated contents, and merge them into the
|
|
token stream.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">interpolateString: </span><span class="p">(</span><span class="nx">str</span><span class="p">,</span> <span class="nx">escapeQuotes</span><span class="p">)</span> <span class="o">-></span>
|
|
<span class="k">if</span> <span class="nx">str</span><span class="p">.</span><span class="nx">length</span> <span class="o"><</span> <span class="mi">3</span> <span class="o">or</span> <span class="o">not</span> <span class="nx">starts</span> <span class="nx">str</span><span class="p">,</span> <span class="s1">'"'</span>
|
|
<span class="nx">@token</span> <span class="s1">'STRING'</span><span class="p">,</span> <span class="nx">str</span>
|
|
<span class="k">else</span>
|
|
<span class="nv">lexer: </span> <span class="k">new</span> <span class="nx">Lexer</span>
|
|
<span class="nv">tokens: </span> <span class="p">[]</span>
|
|
<span class="nv">quote: </span> <span class="nx">str</span><span class="p">.</span><span class="nx">substring</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span>
|
|
<span class="p">[</span><span class="nx">i</span><span class="p">,</span> <span class="nx">pi</span><span class="p">]</span><span class="o">:</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">]</span>
|
|
<span class="k">while</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">str</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
|
<span class="k">if</span> <span class="nx">starts</span> <span class="nx">str</span><span class="p">,</span> <span class="s1">'\\'</span><span class="p">,</span> <span class="nx">i</span>
|
|
<span class="nv">i: </span><span class="o">+</span> <span class="mi">1</span>
|
|
<span class="k">else</span> <span class="k">if</span> <span class="nv">match: </span><span class="nx">str</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="nx">i</span><span class="p">).</span><span class="nx">match</span> <span class="nx">INTERPOLATION</span>
|
|
<span class="p">[</span><span class="nx">group</span><span class="p">,</span> <span class="nx">interp</span><span class="p">]</span><span class="o">:</span> <span class="nx">match</span>
|
|
<span class="nv">interp: </span><span class="s2">"this.${ interp.substring(1) }"</span> <span class="k">if</span> <span class="nx">starts</span> <span class="nx">interp</span><span class="p">,</span> <span class="s1">'@'</span>
|
|
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'STRING'</span><span class="p">,</span> <span class="s2">"$quote${ str.substring(pi, i) }$quote"</span><span class="p">]</span> <span class="k">if</span> <span class="nx">pi</span> <span class="o"><</span> <span class="nx">i</span>
|
|
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'IDENTIFIER'</span><span class="p">,</span> <span class="nx">interp</span><span class="p">]</span>
|
|
<span class="nv">i: </span><span class="o">+</span> <span class="nx">group</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
|
<span class="nv">pi: </span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span>
|
|
<span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nv">expr: </span><span class="nx">@balancedString</span> <span class="nx">str</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="nx">i</span><span class="p">),</span> <span class="p">[[</span><span class="s1">'${'</span><span class="p">,</span> <span class="s1">'}'</span><span class="p">]])</span>
|
|
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'STRING'</span><span class="p">,</span> <span class="s2">"$quote${ str.substring(pi, i) }$quote"</span><span class="p">]</span> <span class="k">if</span> <span class="nx">pi</span> <span class="o"><</span> <span class="nx">i</span>
|
|
<span class="nv">inner: </span><span class="nx">expr</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="nx">expr</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span>
|
|
<span class="k">if</span> <span class="nx">inner</span><span class="p">.</span><span class="nx">length</span>
|
|
<span class="nv">inner: </span><span class="nx">inner</span><span class="p">.</span><span class="nx">replace</span> <span class="k">new</span> <span class="nb">RegExp</span><span class="p">(</span><span class="s1">'\\\\'</span> <span class="o">+</span> <span class="nx">quote</span><span class="p">,</span> <span class="s1">'g'</span><span class="p">),</span> <span class="nx">quote</span>
|
|
<span class="nv">nested: </span><span class="nx">lexer</span><span class="p">.</span><span class="nx">tokenize</span> <span class="s2">"($inner)"</span><span class="p">,</span> <span class="p">{</span><span class="nv">line: </span><span class="nx">@line</span><span class="p">}</span>
|
|
<span class="p">(</span><span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">:</span> <span class="s1">')'</span><span class="p">)</span> <span class="k">for</span> <span class="nx">tok</span><span class="p">,</span> <span class="nx">idx</span> <span class="k">in</span> <span class="nx">nested</span> <span class="k">when</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">'CALL_END'</span>
|
|
<span class="nx">nested</span><span class="p">.</span><span class="nx">pop</span><span class="p">()</span>
|
|
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'TOKENS'</span><span class="p">,</span> <span class="nx">nested</span><span class="p">]</span>
|
|
<span class="k">else</span>
|
|
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'STRING'</span><span class="p">,</span> <span class="s2">"$quote$quote"</span><span class="p">]</span>
|
|
<span class="nv">i: </span><span class="o">+</span> <span class="nx">expr</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
|
<span class="nv">pi: </span><span class="nx">i</span> <span class="o">+</span> <span class="mi">1</span>
|
|
<span class="nv">i: </span><span class="o">+</span> <span class="mi">1</span>
|
|
<span class="nx">tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="s1">'STRING'</span><span class="p">,</span> <span class="s2">"$quote${ str.substring(pi, i) }$quote"</span><span class="p">]</span> <span class="k">if</span> <span class="nx">pi</span> <span class="o"><</span> <span class="nx">i</span> <span class="o">and</span> <span class="nx">pi</span> <span class="o"><</span> <span class="nx">str</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
|
<span class="nx">tokens</span><span class="p">.</span><span class="nx">unshift</span> <span class="p">[</span><span class="s1">'STRING'</span><span class="p">,</span> <span class="s1">'""'</span><span class="p">]</span> <span class="nx">unless</span> <span class="nx">tokens</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span> <span class="o">is</span> <span class="s1">'STRING'</span>
|
|
<span class="nv">interpolated: </span><span class="nx">tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">></span> <span class="mi">1</span>
|
|
<span class="nx">@token</span> <span class="s1">'('</span><span class="p">,</span> <span class="s1">'('</span> <span class="k">if</span> <span class="nx">interpolated</span>
|
|
<span class="k">for</span> <span class="nx">token</span><span class="p">,</span> <span class="nx">i</span> <span class="k">in</span> <span class="nx">tokens</span>
|
|
<span class="p">[</span><span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span><span class="p">]</span><span class="o">:</span> <span class="nx">token</span>
|
|
<span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s1">'TOKENS'</span>
|
|
<span class="vi">@tokens: </span><span class="nx">@tokens</span><span class="p">.</span><span class="nx">concat</span> <span class="nx">value</span>
|
|
<span class="k">else</span> <span class="k">if</span> <span class="nx">tag</span> <span class="o">is</span> <span class="s1">'STRING'</span> <span class="o">and</span> <span class="nx">escapeQuotes</span>
|
|
<span class="nv">escaped: </span><span class="nx">value</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nx">value</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span><span class="p">).</span><span class="nx">replace</span><span class="p">(</span><span class="sr">/"/g</span><span class="p">,</span> <span class="s1">'\\"'</span><span class="p">)</span>
|
|
<span class="nx">@token</span> <span class="nx">tag</span><span class="p">,</span> <span class="s2">"\"$escaped\""</span>
|
|
<span class="k">else</span>
|
|
<span class="nx">@token</span> <span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span>
|
|
<span class="nx">@token</span> <span class="s1">'+'</span><span class="p">,</span> <span class="s1">'+'</span> <span class="k">if</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mi">1</span>
|
|
<span class="nx">@token</span> <span class="s1">')'</span><span class="p">,</span> <span class="s1">')'</span> <span class="k">if</span> <span class="nx">interpolated</span>
|
|
<span class="nx">tokens</span></pre></div> </td> </tr> <tr id="section-33"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-33">#</a> </div> <h2>Helpers</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-34"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-34">#</a> </div> <p>Add a token to the results, taking note of the line number.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">token: </span><span class="p">(</span><span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span><span class="p">)</span> <span class="o">-></span>
|
|
<span class="nx">@tokens</span><span class="p">.</span><span class="nx">push</span> <span class="p">[</span><span class="nx">tag</span><span class="p">,</span> <span class="nx">value</span><span class="p">,</span> <span class="nx">@line</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-35"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-35">#</a> </div> <p>Peek at a tag in the current token stream.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">tag: </span><span class="p">(</span><span class="nx">index</span><span class="p">,</span> <span class="nx">newTag</span><span class="p">)</span> <span class="o">-></span>
|
|
<span class="k">return</span> <span class="nx">unless</span> <span class="nv">tok: </span><span class="nx">@prev</span> <span class="nx">index</span>
|
|
<span class="k">return</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">:</span> <span class="nx">newTag</span> <span class="k">if</span> <span class="nx">newTag</span><span class="o">?</span>
|
|
<span class="nx">tok</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-36"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-36">#</a> </div> <p>Peek at a value in the current token stream.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">value: </span><span class="p">(</span><span class="nx">index</span><span class="p">,</span> <span class="nx">val</span><span class="p">)</span> <span class="o">-></span>
|
|
<span class="k">return</span> <span class="nx">unless</span> <span class="nv">tok: </span><span class="nx">@prev</span> <span class="nx">index</span>
|
|
<span class="k">return</span> <span class="nx">tok</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">:</span> <span class="nx">val</span> <span class="k">if</span> <span class="nx">val</span><span class="o">?</span>
|
|
<span class="nx">tok</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-37"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-37">#</a> </div> <p>Peek at a previous token, entire.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">prev: </span><span class="p">(</span><span class="nx">index</span><span class="p">)</span> <span class="o">-></span>
|
|
<span class="nx">@tokens</span><span class="p">[</span><span class="nx">@tokens</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="p">(</span><span class="nx">index</span> <span class="o">or</span> <span class="mi">1</span><span class="p">)]</span></pre></div> </td> </tr> <tr id="section-38"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-38">#</a> </div> <p>Attempt to match a string against the current chunk, returning the indexed
|
|
match if successful, and <code>false</code> otherwise.</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">match: </span><span class="p">(</span><span class="nx">regex</span><span class="p">,</span> <span class="nx">index</span><span class="p">)</span> <span class="o">-></span>
|
|
<span class="k">return</span> <span class="kc">false</span> <span class="nx">unless</span> <span class="nv">m: </span><span class="nx">@chunk</span><span class="p">.</span><span class="nx">match</span> <span class="nx">regex</span>
|
|
<span class="k">if</span> <span class="nx">m</span> <span class="k">then</span> <span class="nx">m</span><span class="p">[</span><span class="nx">index</span><span class="p">]</span> <span class="k">else</span> <span class="kc">false</span></pre></div> </td> </tr> <tr id="section-39"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-39">#</a> </div> <p>Are we in the midst of an unfinished expression?</p> </td> <td class="code"> <div class="highlight"><pre> <span class="nv">unfinished: </span><span class="o">-></span>
|
|
<span class="nv">prev: </span><span class="nx">@prev</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
|
|
<span class="nx">@value</span><span class="p">()</span> <span class="o">and</span> <span class="nx">@value</span><span class="p">().</span><span class="nx">match</span> <span class="o">and</span> <span class="nx">@value</span><span class="p">().</span><span class="nx">match</span><span class="p">(</span><span class="nx">NO_NEWLINE</span><span class="p">)</span> <span class="o">and</span>
|
|
<span class="nx">prev</span> <span class="o">and</span> <span class="p">(</span><span class="nx">prev</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">isnt</span> <span class="s1">'.'</span><span class="p">)</span> <span class="o">and</span> <span class="o">not</span> <span class="nx">@value</span><span class="p">().</span><span class="nx">match</span><span class="p">(</span><span class="nx">CODE</span><span class="p">)</span></pre></div> </td> </tr> <tr id="section-40"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-40">#</a> </div> <h2>Constants</h2> </td> <td class="code"> <div class="highlight"><pre></pre></div> </td> </tr> <tr id="section-41"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-41">#</a> </div> <p>Keywords that CoffeeScript shares in common with JavaScript.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">JS_KEYWORDS: </span><span class="p">[</span>
|
|
<span class="s2">"if"</span><span class="p">,</span> <span class="s2">"else"</span><span class="p">,</span>
|
|
<span class="s2">"true"</span><span class="p">,</span> <span class="s2">"false"</span><span class="p">,</span>
|
|
<span class="s2">"new"</span><span class="p">,</span> <span class="s2">"return"</span><span class="p">,</span>
|
|
<span class="s2">"try"</span><span class="p">,</span> <span class="s2">"catch"</span><span class="p">,</span> <span class="s2">"finally"</span><span class="p">,</span> <span class="s2">"throw"</span><span class="p">,</span>
|
|
<span class="s2">"break"</span><span class="p">,</span> <span class="s2">"continue"</span><span class="p">,</span>
|
|
<span class="s2">"for"</span><span class="p">,</span> <span class="s2">"in"</span><span class="p">,</span> <span class="s2">"while"</span><span class="p">,</span>
|
|
<span class="s2">"delete"</span><span class="p">,</span> <span class="s2">"instanceof"</span><span class="p">,</span> <span class="s2">"typeof"</span><span class="p">,</span>
|
|
<span class="s2">"switch"</span><span class="p">,</span> <span class="s2">"super"</span><span class="p">,</span> <span class="s2">"extends"</span><span class="p">,</span> <span class="s2">"class"</span><span class="p">,</span>
|
|
<span class="s2">"this"</span><span class="p">,</span> <span class="s2">"null"</span>
|
|
<span class="p">]</span></pre></div> </td> </tr> <tr id="section-42"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-42">#</a> </div> <p>CoffeeScript-only keywords, which we're more relaxed about allowing. They can't
|
|
be used standalone, but you can reference them as an attached property.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">COFFEE_ALIASES: </span> <span class="p">[</span><span class="s2">"and"</span><span class="p">,</span> <span class="s2">"or"</span><span class="p">,</span> <span class="s2">"is"</span><span class="p">,</span> <span class="s2">"isnt"</span><span class="p">,</span> <span class="s2">"not"</span><span class="p">]</span>
|
|
<span class="nv">COFFEE_KEYWORDS: </span><span class="nx">COFFEE_ALIASES</span><span class="p">.</span><span class="nx">concat</span> <span class="p">[</span>
|
|
<span class="s2">"then"</span><span class="p">,</span> <span class="s2">"unless"</span><span class="p">,</span> <span class="s2">"until"</span><span class="p">,</span> <span class="s2">"loop"</span><span class="p">,</span>
|
|
<span class="s2">"yes"</span><span class="p">,</span> <span class="s2">"no"</span><span class="p">,</span> <span class="s2">"on"</span><span class="p">,</span> <span class="s2">"off"</span><span class="p">,</span>
|
|
<span class="s2">"of"</span><span class="p">,</span> <span class="s2">"by"</span><span class="p">,</span> <span class="s2">"where"</span><span class="p">,</span> <span class="s2">"when"</span>
|
|
<span class="p">]</span></pre></div> </td> </tr> <tr id="section-43"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-43">#</a> </div> <p>The list of keywords that are reserved by JavaScript, but not used, or are
|
|
used by CoffeeScript internally. We throw an error when these are encountered,
|
|
to avoid having a JavaScript error at runtime.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">RESERVED: </span><span class="p">[</span>
|
|
<span class="s2">"case"</span><span class="p">,</span> <span class="s2">"default"</span><span class="p">,</span> <span class="s2">"do"</span><span class="p">,</span> <span class="s2">"function"</span><span class="p">,</span> <span class="s2">"var"</span><span class="p">,</span> <span class="s2">"void"</span><span class="p">,</span> <span class="s2">"with"</span>
|
|
<span class="s2">"const"</span><span class="p">,</span> <span class="s2">"let"</span><span class="p">,</span> <span class="s2">"enum"</span><span class="p">,</span> <span class="s2">"export"</span><span class="p">,</span> <span class="s2">"import"</span><span class="p">,</span> <span class="s2">"native"</span>
|
|
<span class="p">]</span></pre></div> </td> </tr> <tr id="section-44"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-44">#</a> </div> <p>The superset of both JavaScript keywords and reserved words, none of which may
|
|
be used as identifiers or properties.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">JS_FORBIDDEN: </span><span class="nx">JS_KEYWORDS</span><span class="p">.</span><span class="nx">concat</span> <span class="nx">RESERVED</span></pre></div> </td> </tr> <tr id="section-45"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-45">#</a> </div> <p>Token matching regexes.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">IDENTIFIER : </span><span class="sr">/^([a-zA-Z\$_](\w|\$)*)/</span>
|
|
<span class="nv">NUMBER : </span><span class="sr">/^(((\b0(x|X)[0-9a-fA-F]+)|((\b[0-9]+(\.[0-9]+)?|\.[0-9]+)(e[+\-]?[0-9]+)?)))\b/i</span>
|
|
<span class="nv">HEREDOC : </span><span class="sr">/^("{6}|'{6}|"{3}\n?([\s\S]*?)\n?([ \t]*)"{3}|'{3}\n?([\s\S]*?)\n?([ \t]*)'{3})/</span>
|
|
<span class="nv">INTERPOLATION : </span><span class="sr">/^\$([a-zA-Z_@]\w*(\.\w+)*)/</span>
|
|
<span class="nv">OPERATOR : </span><span class="sr">/^(-[\-=>]?|\+[+=]?|[*&|\/%=<>:!?]+)([ \t]*)/</span>
|
|
<span class="nv">WHITESPACE : </span><span class="sr">/^([ \t]+)/</span>
|
|
<span class="nv">COMMENT : </span><span class="sr">/^(\s*#{3}(?!#)[ \t]*\n+([\s\S]*?)[ \t]*\n+[ \t]*#{3}|(\s*#(?!##[^#])[^\n]*)+)/</span>
|
|
<span class="nv">CODE : </span><span class="sr">/^((-|=)>)/</span>
|
|
<span class="nv">MULTI_DENT : </span><span class="sr">/^((\n([ \t]*))+)(\.)?/</span>
|
|
<span class="nv">LAST_DENTS : </span><span class="sr">/\n([ \t]*)/g</span>
|
|
<span class="nv">LAST_DENT : </span><span class="sr">/\n([ \t]*)/</span>
|
|
<span class="nv">ASSIGNMENT : </span><span class="sr">/^[:=]$/</span></pre></div> </td> </tr> <tr id="section-46"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-46">#</a> </div> <p>Regex-matching-regexes.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">REGEX_START : </span><span class="sr">/^\/[^\/ ]/</span>
|
|
<span class="nv">REGEX_INTERPOLATION: </span><span class="sr">/([^\\]\$[a-zA-Z_@]|[^\\]\$\{.*[^\\]\})/</span>
|
|
<span class="nv">REGEX_END : </span><span class="sr">/^(([imgy]{1,4})\b|\W|$)/</span>
|
|
<span class="nv">REGEX_ESCAPE : </span><span class="sr">/\\[^\$]/g</span></pre></div> </td> </tr> <tr id="section-47"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-47">#</a> </div> <p>Token cleaning regexes.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">JS_CLEANER : </span><span class="sr">/(^`|`$)/g</span>
|
|
<span class="nv">MULTILINER : </span><span class="sr">/\n/g</span>
|
|
<span class="nv">STRING_NEWLINES : </span><span class="sr">/\n[ \t]*/g</span>
|
|
<span class="nv">NO_NEWLINE : </span><span class="sr">/^([+\*&|\/\-%=<>:!.\\][<>=&|]*|and|or|is|isnt|not|delete|typeof|instanceof)$/</span>
|
|
<span class="nv">HEREDOC_INDENT : </span><span class="sr">/(\n+([ \t]*)|^([ \t]+))/g</span>
|
|
<span class="nv">ASSIGNED : </span><span class="sr">/^([a-zA-Z\$_]\w*[ \t]*?[:=][^=])/</span>
|
|
<span class="nv">NEXT_CHARACTER : </span><span class="sr">/^\s*(\S)/</span></pre></div> </td> </tr> <tr id="section-48"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-48">#</a> </div> <p>Tokens which a regular expression will never immediately follow, but which
|
|
a division operator might.</p>
|
|
|
|
<p>See: http://www.mozilla.org/js/language/js20-2002-04/rationale/syntax.html#regular-expressions</p>
|
|
|
|
<p>Our list is shorter, due to sans-parentheses method calls.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">NOT_REGEX: </span><span class="p">[</span>
|
|
<span class="s1">'NUMBER'</span><span class="p">,</span> <span class="s1">'REGEX'</span><span class="p">,</span> <span class="s1">'++'</span><span class="p">,</span> <span class="s1">'--'</span><span class="p">,</span> <span class="s1">'FALSE'</span><span class="p">,</span> <span class="s1">'NULL'</span><span class="p">,</span> <span class="s1">'TRUE'</span><span class="p">,</span> <span class="s1">']'</span>
|
|
<span class="p">]</span></pre></div> </td> </tr> <tr id="section-49"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-49">#</a> </div> <p>Tokens which could legitimately be invoked or indexed. A opening
|
|
parentheses or bracket following these tokens will be recorded as the start
|
|
of a function invocation or indexing operation.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">CALLABLE: </span><span class="p">[</span><span class="s1">'IDENTIFIER'</span><span class="p">,</span> <span class="s1">'SUPER'</span><span class="p">,</span> <span class="s1">')'</span><span class="p">,</span> <span class="s1">']'</span><span class="p">,</span> <span class="s1">'}'</span><span class="p">,</span> <span class="s1">'STRING'</span><span class="p">,</span> <span class="s1">'@'</span><span class="p">,</span> <span class="s1">'THIS'</span><span class="p">,</span> <span class="s1">'?'</span><span class="p">,</span> <span class="s1">'::'</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-50"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-50">#</a> </div> <p>Tokens that, when immediately preceding a <code>WHEN</code>, indicate that the <code>WHEN</code>
|
|
occurs at the start of a line. We disambiguate these from trailing whens to
|
|
avoid an ambiguity in the grammar.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">LINE_BREAK: </span><span class="p">[</span><span class="s1">'INDENT'</span><span class="p">,</span> <span class="s1">'OUTDENT'</span><span class="p">,</span> <span class="s1">'TERMINATOR'</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-51"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-51">#</a> </div> <p>Half-assignments...</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">HALF_ASSIGNMENTS: </span><span class="p">[</span><span class="s1">'-'</span><span class="p">,</span> <span class="s1">'+'</span><span class="p">,</span> <span class="s1">'/'</span><span class="p">,</span> <span class="s1">'*'</span><span class="p">,</span> <span class="s1">'%'</span><span class="p">,</span> <span class="s1">'||'</span><span class="p">,</span> <span class="s1">'&&'</span><span class="p">,</span> <span class="s1">'?'</span><span class="p">,</span> <span class="s1">'OP?'</span><span class="p">]</span></pre></div> </td> </tr> <tr id="section-52"> <td class="docs"> <div class="octowrap"> <a class="octothorpe" href="#section-52">#</a> </div> <p>Conversions from CoffeeScript operators into JavaScript ones.</p> </td> <td class="code"> <div class="highlight"><pre><span class="nv">CONVERSIONS: </span><span class="p">{</span>
|
|
<span class="s1">'and'</span><span class="o">:</span> <span class="s1">'&&'</span>
|
|
<span class="s1">'or'</span><span class="o">:</span> <span class="s1">'||'</span>
|
|
<span class="s1">'is'</span><span class="o">:</span> <span class="s1">'=='</span>
|
|
<span class="s1">'isnt'</span><span class="o">:</span> <span class="s1">'!='</span>
|
|
<span class="s1">'not'</span><span class="o">:</span> <span class="s1">'!'</span>
|
|
<span class="p">}</span>
|
|
|
|
</pre></div> </td> </tr> </tbody> </table> </div> </body> </html> |