<!DOCTYPE html><html><head><title>rewriter.coffee</title><metahttp-equiv="content-type"content="text/html; charset=UTF-8"><linkrel="stylesheet"media="all"href="docco.css"/></head><body><divid="container"><divid="background"></div><divid="jump_to"> Jump To …<divid="jump_wrapper"><divid="jump_page"><aclass="source"href="cake.html"> cake.coffee </a><aclass="source"href="coffee-script.html"> coffee-script.coffee </a><aclass="source"href="command.html"> command.coffee </a><aclass="source"href="grammar.html"> grammar.coffee </a><aclass="source"href="lexer.html"> lexer.coffee </a><aclass="source"href="nodes.html"> nodes.coffee </a><aclass="source"href="optparse.html"> optparse.coffee </a><aclass="source"href="repl.html"> repl.coffee </a><aclass="source"href="rewriter.html"> rewriter.coffee </a><aclass="source"href="scope.html"> scope.coffee </a></div></div></div><tablecellpadding="0"cellspacing="0"><thead><tr><thclass="docs"><h1> rewriter.coffee </h1></th><thclass="code"></th></tr></thead><tbody><trid="section-1"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-1">#</a></div><p>The CoffeeScript language has a decent amount of optional syntax,
implicit syntax, and shorthand syntax. These things can greatly complicate a
grammar and bloat the resulting parse table. Instead of making the parser
handle it all, we take a series of passes over the token stream,
using this <strong>Rewriter</strong> to convert shorthand into the unambiguous long form,
add implicit indentation and parentheses, balance incorrect nestings, and
generally clean things up.</p></td><tdclass="code"><divclass="highlight"><pre></pre></div></td></tr><trid="section-2"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-2">#</a></div><p>Set up exported variables for both Node.js and the browser.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="k">this</span><spanclass="p">.</span><spanclass="nv">exports: </span><spanclass="k">this</span><spanclass="nx">unless</span><spanclass="nx">process</span><spanclass="o">?</span></pre></div></td></tr><trid="section-3"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-3">#</a></div><p>The <strong>Rewriter</strong> class is used by the <ahref="lexer.html">Lexer</a>, directly against
its internal array of tokens.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">exports.Rewriter: </span><spanclass="nx">class</span><spanclass="nx">Rewriter</span></pre></div></td></tr><trid="section-4"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-4">#</a></div><p>Rewrite the token stream in multiple passes, one logical filter at
stream, with a big ol' efficient switch, but it's much nicer to work with
like this. The order of these passes matters -- indentation must be
corrected before implicit parentheses can be wrapped around blocks of code.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">rewrite: </span><spanclass="p">(</span><spanclass="nx">tokens</span><spanclass="p">)</span><spanclass="o">-></span>
<spanclass="err">@</span><spanclass="nx">tokens</span></pre></div></td></tr><trid="section-5"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-5">#</a></div><p>Rewrite the token stream, looking one token ahead and behind.
<spanclass="kc">true</span></pre></div></td></tr><trid="section-6"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-6">#</a></div><p>Massage newlines and indentations so that comments don't have to be
correctly indented, or appear on a line of their own.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">adjust_comments: </span><spanclass="o">-></span>
<spanclass="k">return</span><spanclass="mi">1</span></pre></div></td></tr><trid="section-7"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-7">#</a></div><p>Leading newlines would introduce an ambiguity in the grammar, so we
<spanclass="err">@</span><spanclass="nx">tokens</span><spanclass="p">.</span><spanclass="nx">shift</span><spanclass="p">()</span><spanclass="k">while</span><spanclass="err">@</span><spanclass="nx">tokens</span><spanclass="p">[</span><spanclass="mi">0</span><spanclass="p">][</span><spanclass="mi">0</span><spanclass="p">]</span><spanclass="o">is</span><spanclass="s1">'TERMINATOR'</span></pre></div></td></tr><trid="section-8"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-8">#</a></div><p>Some blocks occur in the middle of expressions -- when we're expecting
<spanclass="k">return</span><spanclass="mi">0</span></pre></div></td></tr><trid="section-9"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-9">#</a></div><p>The lexer has tagged the opening parenthesis of a method call, and the
opening bracket of an indexing operation. Match them with their paired
<spanclass="k">return</span><spanclass="mi">1</span></pre></div></td></tr><trid="section-10"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-10">#</a></div><p>Methods may be optionally called without parentheses, for simple cases.
<spanclass="k">return</span><spanclass="mi">2</span></pre></div></td></tr><trid="section-11"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-11">#</a></div><p>Because our grammar is LALR(1), it can't handle some single-line
expressions that lack ending delimiters. The <strong>Rewriter</strong> adds the implicit
blocks, so it doesn't need to. ')' can close a single-line block,
but we need to make sure it's balanced.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">add_implicit_indentation: </span><spanclass="o">-></span>
<spanclass="k">return</span><spanclass="mi">0</span></pre></div></td></tr><trid="section-12"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-12">#</a></div><p>Ensure that all listed pairs of tokens are correctly balanced throughout
the course of the token stream.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">ensure_balance: </span><spanclass="p">(</span><spanclass="nx">pairs</span><spanclass="p">)</span><spanclass="o">-></span>
<spanclass="k">throw</span><spanclass="k">new</span><spanclass="nb">Error</span><spanclass="p">(</span><spanclass="s2">"too many ${token[1]} on line ${token[2] + 1}"</span><spanclass="p">)</span><spanclass="k">if</span><spanclass="nx">levels</span><spanclass="p">[</span><spanclass="nx">open</span><spanclass="p">]</span><spanclass="o"><</span><spanclass="mi">0</span>
<spanclass="k">throw</span><spanclass="k">new</span><spanclass="nb">Error</span><spanclass="p">(</span><spanclass="s2">"unclosed ${unclosed[0]}"</span><spanclass="p">)</span><spanclass="k">if</span><spanclass="nx">unclosed</span><spanclass="p">.</span><spanclass="nx">length</span></pre></div></td></tr><trid="section-13"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-13">#</a></div><p>We'd like to support syntax like this:</p>
<pre><code>el.click((event) ->
el.hide())
</code></pre>
<p>In order to accomplish this, move outdents that follow closing parens
<spanclass="k">return</span><spanclass="mi">1</span></pre></div></td></tr><trid="section-14"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-14">#</a></div><h2>Constants</h2></td><tdclass="code"><divclass="highlight"><pre></pre></div></td></tr><trid="section-15"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-15">#</a></div><p>List of the token pairs that must be balanced.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">BALANCED_PAIRS: </span><spanclass="p">[[</span><spanclass="s1">'('</span><spanclass="p">,</span><spanclass="s1">')'</span><spanclass="p">],</span><spanclass="p">[</span><spanclass="s1">'['</span><spanclass="p">,</span><spanclass="s1">']'</span><spanclass="p">],</span><spanclass="p">[</span><spanclass="s1">'{'</span><spanclass="p">,</span><spanclass="s1">'}'</span><spanclass="p">],</span><spanclass="p">[</span><spanclass="s1">'INDENT'</span><spanclass="p">,</span><spanclass="s1">'OUTDENT'</span><spanclass="p">],</span>
<spanclass="p">[</span><spanclass="s1">'INDEX_START'</span><spanclass="p">,</span><spanclass="s1">'INDEX_END'</span><spanclass="p">],</span><spanclass="p">[</span><spanclass="s1">'SOAKED_INDEX_START'</span><spanclass="p">,</span><spanclass="s1">'SOAKED_INDEX_END'</span><spanclass="p">]]</span></pre></div></td></tr><trid="section-16"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-16">#</a></div><p>The inverse mappings of <code>BALANCED_PAIRS</code> we're trying to fix up, so we can
look things up from either end.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">INVERSES: </span><spanclass="p">{}</span>
<spanclass="nx">INVERSES</span><spanclass="p">[</span><spanclass="nx">pair</span><spanclass="p">[</span><spanclass="mi">1</span><spanclass="p">]]</span><spanclass="o">:</span><spanclass="nx">pair</span><spanclass="p">[</span><spanclass="mi">0</span><spanclass="p">]</span></pre></div></td></tr><trid="section-17"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-17">#</a></div><p>The tokens that signal the start of a balanced pair.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">EXPRESSION_START: </span><spanclass="nx">pair</span><spanclass="p">[</span><spanclass="mi">0</span><spanclass="p">]</span><spanclass="k">for</span><spanclass="nx">pair</span><spanclass="k">in</span><spanclass="nx">BALANCED_PAIRS</span></pre></div></td></tr><trid="section-18"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-18">#</a></div><p>The tokens that signal the end of a balanced pair.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">EXPRESSION_END: </span><spanclass="nx">pair</span><spanclass="p">[</span><spanclass="mi">1</span><spanclass="p">]</span><spanclass="k">for</span><spanclass="nx">pair</span><spanclass="k">in</span><spanclass="nx">BALANCED_PAIRS</span></pre></div></td></tr><trid="section-19"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-19">#</a></div><p>Tokens that indicate the close of a clause of an expression.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">EXPRESSION_CLOSE: </span><spanclass="p">[</span><spanclass="s1">'CATCH'</span><spanclass="p">,</span><spanclass="s1">'WHEN'</span><spanclass="p">,</span><spanclass="s1">'ELSE'</span><spanclass="p">,</span><spanclass="s1">'FINALLY'</span><spanclass="p">].</span><spanclass="nx">concat</span><spanclass="nx">EXPRESSION_END</span></pre></div></td></tr><trid="section-20"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-20">#</a></div><p>Tokens that, if followed by an <code>IMPLICIT_CALL</code>, indicate a function invocation.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">IMPLICIT_FUNC: </span><spanclass="p">[</span><spanclass="s1">'IDENTIFIER'</span><spanclass="p">,</span><spanclass="s1">'SUPER'</span><spanclass="p">,</span><spanclass="s1">')'</span><spanclass="p">,</span><spanclass="s1">'CALL_END'</span><spanclass="p">,</span><spanclass="s1">']'</span><spanclass="p">,</span><spanclass="s1">'INDEX_END'</span><spanclass="p">]</span></pre></div></td></tr><trid="section-21"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-21">#</a></div><p>If preceded by an <code>IMPLICIT_FUNC</code>, indicates a function invocation.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">IMPLICIT_CALL: </span><spanclass="p">[</span><spanclass="s1">'IDENTIFIER'</span><spanclass="p">,</span><spanclass="s1">'NUMBER'</span><spanclass="p">,</span><spanclass="s1">'STRING'</span><spanclass="p">,</span><spanclass="s1">'JS'</span><span
<spanclass="s1">'@'</span><spanclass="p">,</span><spanclass="s1">'->'</span><spanclass="p">,</span><spanclass="s1">'=>'</span><spanclass="p">,</span><spanclass="s1">'['</span><spanclass="p">,</span><spanclass="s1">'('</span><spanclass="p">,</span><spanclass="s1">'{'</span><spanclass="p">]</span></pre></div></td></tr><trid="section-22"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-22">#</a></div><p>Tokens indicating that the implicit call must enclose a block of expressions.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">IMPLICIT_BLOCK: </span><spanclass="p">[</span><spanclass="s1">'->'</span><spanclass="p">,</span><spanclass="s1">'=>'</span><spanclass="p">,</span><spanclass="s1">'{'</span><spanclass="p">,</span><spanclass="s1">'['</span><spanclass="p">,</span><spanclass="s1">','</span><spanclass="p">]</span></pre></div></td></tr><trid="section-23"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-23">#</a></div><p>Tokens that always mark the end of an implicit call for single-liners.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">IMPLICIT_END: </span><spanclass="p">[</span><spanclass="s1">'IF'</span><spanclass="p">,</span><spanclass="s1">'UNLESS'</span><spanclass="p">,</span><spanclass="s1">'FOR'</span><spanclass="p">,</span><spanclass="s1">'WHILE'</span><spanclass="p">,</span><spanclass="s1">'TERMINATOR'</span><spanclass="p">,</span><spanclass="s1">'INDENT'</span><spanclass="p">,</span><spanclass="s1">'OUTDENT'</span><spanclass="p">]</span></pre></div></td></tr><trid="section-24"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-24">#</a></div><p>Single-line flavors of block expressions that have unclosed endings.
The grammar can't disambiguate them, so we insert the implicit indentation.</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">SINGLE_LINERS: </span><spanclass="p">[</span><spanclass="s1">'ELSE'</span><spanclass="p">,</span><spanclass="s2">"->"</span><spanclass="p">,</span><spanclass="s2">"=>"</span><spanclass="p">,</span><spanclass="s1">'TRY'</span><spanclass="p">,</span><spanclass="s1">'FINALLY'</span><spanclass="p">,</span><spanclass="s1">'THEN'</span><spanclass="p">]</span>
<spanclass="nv">SINGLE_CLOSERS: </span><spanclass="p">[</span><spanclass="s1">'TERMINATOR'</span><spanclass="p">,</span><spanclass="s1">'CATCH'</span><spanclass="p">,</span><spanclass="s1">'FINALLY'</span><spanclass="p">,</span><spanclass="s1">'ELSE'</span><spanclass="p">,</span><spanclass="s1">'OUTDENT'</span><spanclass="p">,</span><spanclass="s1">'LEADING_WHEN'</span><spanclass="p">]</span></pre></div></td></tr><trid="section-25"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-25">#</a></div><h2>Utility Functions</h2></td><tdclass="code"><divclass="highlight"><pre></pre></div></td></tr><trid="section-26"><tdclass="docs"><divclass="octowrap"><aclass="octothorpe"href="#section-26">#</a></div><p>Does a list include a value?</p></td><tdclass="code"><divclass="highlight"><pre><spanclass="nv">include: </span><spanclass="p">(</span><spanclass="nx">list</span><spanclass="p">,</span><spanclass="nx">value</span><spanclass="p">)</span><spanclass="o">-></span>
