2010-05-04 23:31:28 -04:00
<!DOCTYPE html> < html > < head > < title > rewriter.coffee< / title > < meta http-equiv = "content-type" content = "text/html; charset=UTF-8" > < link rel = "stylesheet" media = "all" href = "docco.css" / > < / head > < body > < div id = "container" > < div id = "background" > < / div > < div id = "jump_to" > Jump To … < div id = "jump_wrapper" > < div id = "jump_page" > < a class = "source" href = "cake.html" > cake.coffee < / a > < a class = "source" href = "coffee-script.html" > coffee-script.coffee < / a > < a class = "source" href = "command.html" > command.coffee < / a > < a class = "source" href = "grammar.html" > grammar.coffee < / a > < a class = "source" href = "helpers.html" > helpers.coffee < / a > < a class = "source" href = "index.html" > index.coffee < / a > < a class = "source" href = "lexer.html" > lexer.coffee < / a > < a class = "source" href = "nodes.html" > nodes.coffee < / a > < a class = "source" href = "optparse.html" > optparse.coffee < / a > < a class = "source" href = "repl.html" > repl.coffee < / a > < a class = "source" href = "rewriter.html" > rewriter.coffee < / a > < a class = "source" href = "scope.html" > scope.coffee < / a > < / div > < / div > < / div > < table cellpadding = "0" cellspacing = "0" > < thead > < tr > < th class = "docs" > < h1 > rewriter.coffee < / h1 > < / th > < th class = "code" > < / th > < / tr > < / thead > < tbody > < tr id = "section-1" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-1" > #< / a > < / div > < p > The CoffeeScript language has a good deal of optional syntax, implicit syntax,
2010-03-16 23:18:54 -04:00
and shorthand syntax. This can greatly complicate a grammar and bloat
the resulting parse table. Instead of making the parser handle it all, we take
a series of passes over the token stream, using this < strong > Rewriter< / strong > to convert
shorthand into the unambiguous long form, add implicit indentation and
parentheses, balance incorrect nestings, and generally clean things up.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < / pre > < / div > < / td > < / tr > < tr id = "section-2" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-2" > #< / a > < / div > < p > Set up exported variables for both Node.js and the browser.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "k" > if< / span > < span class = "nx" > process< / span > < span class = "o" > ?< / span >
2010-05-04 23:31:28 -04:00
< span class = "p" > {< / span > < span class = "nx" > helpers< / span > < span class = "p" > }< / span > < span class = "o" > :< / span > < span class = "nx" > require< / span > < span class = "p" > (< / span > < span class = "s1" > ' ./helpers' < / span > < span class = "p" > )< / span >
2010-03-09 23:53:21 -05:00
< span class = "k" > else< / span >
2010-03-16 23:18:54 -04:00
< span class = "k" > this< / span > < span class = "p" > .< / span > < span class = "nv" > exports: < / span > < span class = "k" > this< / span >
2010-05-04 23:31:28 -04:00
< span class = "nv" > helpers: < / span > < span class = "k" > this< / span > < span class = "p" > .< / span > < span class = "nx" > helpers< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-3" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-3" > #< / a > < / div > < p > Import the helpers we need.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "p" > {< / span > < span class = "nx" > include< / span > < span class = "p" > }< / span > < span class = "o" > :< / span > < span class = "nx" > helpers< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-4" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-4" > #< / a > < / div > < p > The < strong > Rewriter< / strong > class is used by the < a href = "lexer.html" > Lexer< / a > , directly against
2010-03-16 23:18:54 -04:00
its internal array of tokens.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > exports.Rewriter: < / span > < span class = "nx" > class< / span > < span class = "nx" > Rewriter< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-5" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-5" > #< / a > < / div > < p > Rewrite the token stream in multiple passes, one logical filter at
2010-03-06 23:48:06 -05:00
a time. This could certainly be changed into a single pass through the
2010-03-07 14:41:52 -05:00
stream, with a big ol' efficient switch, but it's much nicer to work with
like this. The order of these passes matters -- indentation must be
corrected before implicit parentheses can be wrapped around blocks of code.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > rewrite: < / span > < span class = "p" > (< / span > < span class = "nx" > tokens< / span > < span class = "p" > )< / span > < span class = "o" > -> < / span >
2010-04-03 12:01:46 -04:00
< span class = "vi" > @tokens: < / span > < span class = "nx" > tokens< / span >
< span class = "nx" > @adjust_comments< / span > < span class = "p" > ()< / span >
< span class = "nx" > @remove_leading_newlines< / span > < span class = "p" > ()< / span >
< span class = "nx" > @remove_mid_expression_newlines< / span > < span class = "p" > ()< / span >
< span class = "nx" > @close_open_calls_and_indexes< / span > < span class = "p" > ()< / span >
< span class = "nx" > @add_implicit_indentation< / span > < span class = "p" > ()< / span >
< span class = "nx" > @add_implicit_parentheses< / span > < span class = "p" > ()< / span >
< span class = "nx" > @ensure_balance< / span > < span class = "nx" > BALANCED_PAIRS< / span >
< span class = "nx" > @rewrite_closing_parens< / span > < span class = "p" > ()< / span >
< span class = "nx" > @tokens< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-6" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-6" > #< / a > < / div > < p > Rewrite the token stream, looking one token ahead and behind.
2010-03-06 23:48:06 -05:00
Allow the return value of the block to tell us how many tokens to move
forwards (or backwards) in the stream, to make sure we don't miss anything
2010-03-07 14:41:52 -05:00
as tokens are inserted and removed, and the stream changes length under
our feet.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > scan_tokens: < / span > < span class = "p" > (< / span > < span class = "nx" > block< / span > < span class = "p" > )< / span > < span class = "o" > -> < / span >
2010-03-06 23:48:06 -05:00
< span class = "nv" > i: < / span > < span class = "mi" > 0< / span >
< span class = "k" > while< / span > < span class = "kc" > true< / span >
2010-04-03 12:01:46 -04:00
< span class = "k" > break< / span > < span class = "nx" > unless< / span > < span class = "nx" > @tokens< / span > < span class = "p" > [< / span > < span class = "nx" > i< / span > < span class = "p" > ]< / span >
< span class = "nv" > move: < / span > < span class = "nx" > block< / span > < span class = "nx" > @tokens< / span > < span class = "p" > [< / span > < span class = "nx" > i< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "p" > ],< / span > < span class = "nx" > @tokens< / span > < span class = "p" > [< / span > < span class = "nx" > i< / span > < span class = "p" > ],< / span > < span class = "nx" > @tokens< / span > < span class = "p" > [< / span > < span class = "nx" > i< / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span > < span class = "p" > ],< / span > < span class = "nx" > i< / span >
2010-03-23 00:18:50 -04:00
< span class = "nv" > i: < / span > < span class = "o" > +< / span > < span class = "nx" > move< / span >
2010-03-16 23:18:54 -04:00
< span class = "kc" > true< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-7" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-7" > #< / a > < / div > < p > Massage newlines and indentations so that comments don't have to be
2010-03-07 14:41:52 -05:00
correctly indented, or appear on a line of their own.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > adjust_comments: < / span > < span class = "o" > -> < / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @scan_tokens< / span > < span class = "p" > (< / span > < span class = "nx" > prev< / span > < span class = "p" > ,< / span > < span class = "nx" > token< / span > < span class = "p" > ,< / span > < span class = "nx" > post< / span > < span class = "p" > ,< / span > < span class = "nx" > i< / span > < span class = "p" > )< / span > < span class = "o" > => < / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > return< / span > < span class = "mi" > 1< / span > < span class = "nx" > unless< / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' COMMENT' < / span >
2010-05-04 23:31:28 -04:00
< span class = "p" > [< / span > < span class = "nx" > before< / span > < span class = "p" > ,< / span > < span class = "nx" > after< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "p" > [< / span > < span class = "nx" > @tokens< / span > < span class = "p" > [< / span > < span class = "nx" > i< / span > < span class = "o" > -< / span > < span class = "mi" > 2< / span > < span class = "p" > ],< / span > < span class = "nx" > @tokens< / span > < span class = "p" > [< / span > < span class = "nx" > i< / span > < span class = "o" > +< / span > < span class = "mi" > 2< / span > < span class = "p" > ]]< / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > if< / span > < span class = "nx" > after< / span > < span class = "o" > and< / span > < span class = "nx" > after< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' INDENT' < / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > splice< / span > < span class = "nx" > i< / span > < span class = "o" > +< / span > < span class = "mi" > 2< / span > < span class = "p" > ,< / span > < span class = "mi" > 1< / span >
2010-05-04 23:31:28 -04:00
< span class = "k" > if< / span > < span class = "nx" > before< / span > < span class = "o" > and< / span > < span class = "nx" > before< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' OUTDENT' < / span > < span class = "o" > and< / span > < span class = "nx" > post< / span > < span class = "o" > and< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "nx" > post< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' TERMINATOR' < / span >
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > splice< / span > < span class = "nx" > i< / span > < span class = "o" > -< / span > < span class = "mi" > 2< / span > < span class = "p" > ,< / span > < span class = "mi" > 1< / span >
< span class = "k" > else< / span >
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > splice< / span > < span class = "nx" > i< / span > < span class = "p" > ,< / span > < span class = "mi" > 0< / span > < span class = "p" > ,< / span > < span class = "nx" > after< / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > return< / span > < span class = "mi" > 1< / span >
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > prev< / span > < span class = "o" > and< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > isnt< / span > < span class = "s1" > ' TERMINATOR' < / span > < span class = "o" > and< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > isnt< / span > < span class = "s1" > ' INDENT' < / span > < span class = "o" > and< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > isnt< / span > < span class = "s1" > ' OUTDENT' < / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > splice< / span > < span class = "nx" > i< / span > < span class = "p" > ,< / span > < span class = "mi" > 0< / span > < span class = "p" > ,< / span > < span class = "p" > [< / span > < span class = "s1" > ' TERMINATOR' < / span > < span class = "p" > ,< / span > < span class = "s2" > " \n" < / span > < span class = "p" > ,< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 2< / span > < span class = "p" > ]]< / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > return< / span > < span class = "mi" > 2< / span >
< span class = "k" > else< / span >
2010-03-16 23:18:54 -04:00
< span class = "k" > return< / span > < span class = "mi" > 1< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-8" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-8" > #< / a > < / div > < p > Leading newlines would introduce an ambiguity in the grammar, so we
2010-03-06 23:48:06 -05:00
dispatch them here.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > remove_leading_newlines: < / span > < span class = "o" > -> < / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > shift< / span > < span class = "p" > ()< / span > < span class = "k" > while< / span > < span class = "nx" > @tokens< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > and< / span > < span class = "nx" > @tokens< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ][< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' TERMINATOR' < / span > < / pre > < / div > < / td > < / tr > < tr id = "section-9" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-9" > #< / a > < / div > < p > Some blocks occur in the middle of expressions -- when we're expecting
2010-03-06 23:48:06 -05:00
this, remove their trailing newlines.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > remove_mid_expression_newlines: < / span > < span class = "o" > -> < / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @scan_tokens< / span > < span class = "p" > (< / span > < span class = "nx" > prev< / span > < span class = "p" > ,< / span > < span class = "nx" > token< / span > < span class = "p" > ,< / span > < span class = "nx" > post< / span > < span class = "p" > ,< / span > < span class = "nx" > i< / span > < span class = "p" > )< / span > < span class = "o" > => < / span >
2010-03-07 14:41:52 -05:00
< span class = "k" > return< / span > < span class = "mi" > 1< / span > < span class = "nx" > unless< / span > < span class = "nx" > post< / span > < span class = "o" > and< / span > < span class = "nx" > include< / span > < span class = "p" > (< / span > < span class = "nx" > EXPRESSION_CLOSE< / span > < span class = "p" > ,< / span > < span class = "nx" > post< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ])< / span > < span class = "o" > and< / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' TERMINATOR' < / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > splice< / span > < span class = "nx" > i< / span > < span class = "p" > ,< / span > < span class = "mi" > 1< / span >
2010-03-16 23:18:54 -04:00
< span class = "k" > return< / span > < span class = "mi" > 0< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-10" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-10" > #< / a > < / div > < p > The lexer has tagged the opening parenthesis of a method call, and the
2010-03-07 14:41:52 -05:00
opening bracket of an indexing operation. Match them with their paired
close.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > close_open_calls_and_indexes: < / span > < span class = "o" > -> < / span >
2010-03-06 23:48:06 -05:00
< span class = "nv" > parens: < / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
< span class = "nv" > brackets: < / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @scan_tokens< / span > < span class = "p" > (< / span > < span class = "nx" > prev< / span > < span class = "p" > ,< / span > < span class = "nx" > token< / span > < span class = "p" > ,< / span > < span class = "nx" > post< / span > < span class = "p" > ,< / span > < span class = "nx" > i< / span > < span class = "p" > )< / span > < span class = "o" > => < / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > switch< / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
2010-03-31 00:17:49 -04:00
< span class = "k" > when< / span > < span class = "s1" > ' CALL_START' < / span > < span class = "k" > then< / span > < span class = "nx" > parens< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "mi" > 0< / span >
< span class = "k" > when< / span > < span class = "s1" > ' INDEX_START' < / span > < span class = "k" > then< / span > < span class = "nx" > brackets< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "mi" > 0< / span >
2010-03-23 00:18:50 -04:00
< span class = "k" > when< / span > < span class = "s1" > ' (' < / span > < span class = "k" > then< / span > < span class = "nx" > parens< / span > < span class = "p" > [< / span > < span class = "nx" > parens< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span >
< span class = "k" > when< / span > < span class = "s1" > ' [' < / span > < span class = "k" > then< / span > < span class = "nx" > brackets< / span > < span class = "p" > [< / span > < span class = "nx" > brackets< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > when< / span > < span class = "s1" > ' )' < / span >
< span class = "k" > if< / span > < span class = "nx" > parens< / span > < span class = "p" > [< / span > < span class = "nx" > parens< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "mi" > 0< / span >
< span class = "nx" > parens< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span >
< span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "s1" > ' CALL_END' < / span >
< span class = "k" > else< / span >
2010-03-23 00:18:50 -04:00
< span class = "nx" > parens< / span > < span class = "p" > [< / span > < span class = "nx" > parens< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > when< / span > < span class = "s1" > ' ]' < / span >
< span class = "k" > if< / span > < span class = "nx" > brackets< / span > < span class = "p" > [< / span > < span class = "nx" > brackets< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "o" > ==< / span > < span class = "mi" > 0< / span >
< span class = "nx" > brackets< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span >
< span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "s1" > ' INDEX_END' < / span >
< span class = "k" > else< / span >
2010-03-23 00:18:50 -04:00
< span class = "nx" > brackets< / span > < span class = "p" > [< / span > < span class = "nx" > brackets< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span >
2010-03-16 23:18:54 -04:00
< span class = "k" > return< / span > < span class = "mi" > 1< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-11" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-11" > #< / a > < / div > < p > Methods may be optionally called without parentheses, for simple cases.
2010-03-06 23:48:06 -05:00
Insert the implicit parentheses here, so that the parser doesn't have to
deal with them.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > add_implicit_parentheses: < / span > < span class = "o" > -> < / span >
< span class = "nv" > stack: < / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
2010-05-04 23:31:28 -04:00
< span class = "nv" > close_calls: < / span > < span class = "p" > (< / span > < span class = "nx" > i< / span > < span class = "p" > )< / span > < span class = "o" > => < / span >
< span class = "k" > for< / span > < span class = "nx" > tmp< / span > < span class = "k" > in< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ...< / span > < span class = "nx" > stack< / span > < span class = "p" > [< / span > < span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "p" > ]]< / span >
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > splice< / span > < span class = "p" > (< / span > < span class = "nx" > i< / span > < span class = "p" > ,< / span > < span class = "mi" > 0< / span > < span class = "p" > ,< / span > < span class = "p" > [< / span > < span class = "s1" > ' CALL_END' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' )' < / span > < span class = "p" > ,< / span > < span class = "nx" > @tokens< / span > < span class = "p" > [< / span > < span class = "nx" > i< / span > < span class = "p" > ][< / span > < span class = "mi" > 2< / span > < span class = "p" > ]])< / span >
< span class = "nv" > size: < / span > < span class = "nx" > stack< / span > < span class = "p" > [< / span > < span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span >
< span class = "nx" > stack< / span > < span class = "p" > [< / span > < span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "mi" > 0< / span >
< span class = "nx" > size< / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @scan_tokens< / span > < span class = "p" > (< / span > < span class = "nx" > prev< / span > < span class = "p" > ,< / span > < span class = "nx" > token< / span > < span class = "p" > ,< / span > < span class = "nx" > post< / span > < span class = "p" > ,< / span > < span class = "nx" > i< / span > < span class = "p" > )< / span > < span class = "o" > => < / span >
2010-03-06 23:48:06 -05:00
< span class = "nv" > tag: < / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
2010-05-04 23:31:28 -04:00
< span class = "nx" > stack< / span > < span class = "p" > [< / span > < span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 2< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "o" > +< / span > < span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span > < span class = "k" > if< / span > < span class = "nx" > tag< / span > < span class = "o" > is< / span > < span class = "s1" > ' OUTDENT' < / span >
2010-03-16 23:18:54 -04:00
< span class = "nv" > open: < / span > < span class = "nx" > stack< / span > < span class = "p" > [< / span > < span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "o" > > < / span > < span class = "mi" > 0< / span >
2010-05-15 01:18:05 -04:00
< span class = "k" > if< / span > < span class = "nx" > prev< / span > < span class = "o" > and< / span > < span class = "nx" > prev< / span > < span class = "p" > .< / span > < span class = "nx" > spaced< / span > < span class = "o" > and< / span > < span class = "nx" > include< / span > < span class = "p" > (< / span > < span class = "nx" > IMPLICIT_FUNC< / span > < span class = "p" > ,< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ])< / span > < span class = "o" > and< / span > < span class = "nx" > include< / span > < span class = "p" > (< / span > < span class = "nx" > IMPLICIT_CALL< / span > < span class = "p" > ,< / span > < span class = "nx" > tag< / span > < span class = "p" > )< / span >
2010-05-04 23:31:28 -04:00
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > splice< / span > < span class = "nx" > i< / span > < span class = "p" > ,< / span > < span class = "mi" > 0< / span > < span class = "p" > ,< / span > < span class = "p" > [< / span > < span class = "s1" > ' CALL_START' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' (' < / span > < span class = "p" > ,< / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 2< / span > < span class = "p" > ]]< / span >
< span class = "nx" > stack< / span > < span class = "p" > [< / span > < span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span >
< span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "mi" > 0< / span > < span class = "k" > if< / span > < span class = "nx" > include< / span > < span class = "p" > (< / span > < span class = "nx" > EXPRESSION_START< / span > < span class = "p" > ,< / span > < span class = "nx" > tag< / span > < span class = "p" > )< / span >
< span class = "k" > return< / span > < span class = "mi" > 2< / span >
< span class = "k" > if< / span > < span class = "nx" > include< / span > < span class = "p" > (< / span > < span class = "nx" > EXPRESSION_START< / span > < span class = "p" > ,< / span > < span class = "nx" > tag< / span > < span class = "p" > )< / span >
< span class = "k" > if< / span > < span class = "nx" > tag< / span > < span class = "o" > is< / span > < span class = "s1" > ' INDENT' < / span > < span class = "o" > and< / span > < span class = "o" > !< / span > < span class = "nx" > token< / span > < span class = "p" > .< / span > < span class = "nx" > generated< / span > < span class = "o" > and< / span > < span class = "nx" > open< / span > < span class = "o" > and< / span > < span class = "o" > not< / span > < span class = "p" > (< / span > < span class = "nx" > prev< / span > < span class = "o" > and< / span > < span class = "nx" > include< / span > < span class = "p" > (< / span > < span class = "nx" > IMPLICIT_BLOCK< / span > < span class = "p" > ,< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]))< / span >
< span class = "nv" > size: < / span > < span class = "nx" > close_calls< / span > < span class = "p" > (< / span > < span class = "nx" > i< / span > < span class = "p" > )< / span >
< span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "mi" > 0< / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > return< / span > < span class = "nx" > size< / span >
2010-05-04 23:31:28 -04:00
< span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "mi" > 0< / span >
< span class = "k" > return< / span > < span class = "mi" > 1< / span >
< span class = "k" > if< / span > < span class = "nx" > open< / span > < span class = "o" > and< / span > < span class = "o" > !< / span > < span class = "nx" > token< / span > < span class = "p" > .< / span > < span class = "nx" > generated< / span > < span class = "o" > and< / span > < span class = "p" > (< / span > < span class = "o" > !< / span > < span class = "nx" > post< / span > < span class = "o" > or< / span > < span class = "nx" > include< / span > < span class = "p" > (< / span > < span class = "nx" > IMPLICIT_END< / span > < span class = "p" > ,< / span > < span class = "nx" > tag< / span > < span class = "p" > ))< / span >
< span class = "nv" > j: < / span > < span class = "mi" > 1< / span > < span class = "p" > ;< / span > < span class = "nx" > j< / span > < span class = "o" > ++< / span > < span class = "k" > while< / span > < span class = "p" > (< / span > < span class = "nv" > nx: < / span > < span class = "nx" > @tokens< / span > < span class = "p" > [< / span > < span class = "nx" > i< / span > < span class = "o" > +< / span > < span class = "nx" > j< / span > < span class = "p" > ])< / span > < span class = "o" > ?< / span > < span class = "o" > and< / span > < span class = "nx" > include< / span > < span class = "p" > (< / span > < span class = "nx" > IMPLICIT_END< / span > < span class = "p" > ,< / span > < span class = "nx" > nx< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ])< / span >
< span class = "k" > if< / span > < span class = "nx" > nx< / span > < span class = "o" > ?< / span > < span class = "o" > and< / span > < span class = "nx" > nx< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' ,' < / span >
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > splice< / span > < span class = "p" > (< / span > < span class = "nx" > i< / span > < span class = "p" > ,< / span > < span class = "mi" > 1< / span > < span class = "p" > )< / span > < span class = "k" > if< / span > < span class = "nx" > tag< / span > < span class = "o" > is< / span > < span class = "s1" > ' TERMINATOR' < / span >
< span class = "k" > else< / span >
< span class = "nv" > size: < / span > < span class = "nx" > close_calls< / span > < span class = "p" > (< / span > < span class = "nx" > i< / span > < span class = "p" > )< / span >
< span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span > < span class = "k" > if< / span > < span class = "nx" > tag< / span > < span class = "o" > isnt< / span > < span class = "s1" > ' OUTDENT' < / span > < span class = "o" > and< / span > < span class = "nx" > include< / span > < span class = "nx" > EXPRESSION_END< / span > < span class = "p" > ,< / span > < span class = "nx" > tag< / span >
< span class = "k" > return< / span > < span class = "nx" > size< / span >
< span class = "k" > if< / span > < span class = "nx" > tag< / span > < span class = "o" > isnt< / span > < span class = "s1" > ' OUTDENT' < / span > < span class = "o" > and< / span > < span class = "nx" > include< / span > < span class = "nx" > EXPRESSION_END< / span > < span class = "p" > ,< / span > < span class = "nx" > tag< / span >
< span class = "nx" > stack< / span > < span class = "p" > [< / span > < span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 2< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "o" > +< / span > < span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span >
< span class = "k" > return< / span > < span class = "mi" > 1< / span >
< span class = "k" > return< / span > < span class = "mi" > 1< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-12" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-12" > #< / a > < / div > < p > Because our grammar is LALR(1), it can't handle some single-line
2010-03-07 14:41:52 -05:00
expressions that lack ending delimiters. The < strong > Rewriter< / strong > adds the implicit
blocks, so it doesn't need to. ')' can close a single-line block,
but we need to make sure it's balanced.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > add_implicit_indentation: < / span > < span class = "o" > -> < / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @scan_tokens< / span > < span class = "p" > (< / span > < span class = "nx" > prev< / span > < span class = "p" > ,< / span > < span class = "nx" > token< / span > < span class = "p" > ,< / span > < span class = "nx" > post< / span > < span class = "p" > ,< / span > < span class = "nx" > i< / span > < span class = "p" > )< / span > < span class = "o" > => < / span >
2010-03-07 14:41:52 -05:00
< span class = "k" > return< / span > < span class = "mi" > 1< / span > < span class = "nx" > unless< / span > < span class = "nx" > include< / span > < span class = "p" > (< / span > < span class = "nx" > SINGLE_LINERS< / span > < span class = "p" > ,< / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ])< / span > < span class = "o" > and< / span >
< span class = "nx" > post< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > isnt< / span > < span class = "s1" > ' INDENT' < / span > < span class = "o" > and< / span >
2010-03-06 23:48:06 -05:00
< span class = "o" > not< / span > < span class = "p" > (< / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' ELSE' < / span > < span class = "o" > and< / span > < span class = "nx" > post< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' IF' < / span > < span class = "p" > )< / span >
< span class = "nv" > starter: < / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
2010-05-04 23:31:28 -04:00
< span class = "nv" > indent: < / span > < span class = "p" > [< / span > < span class = "s1" > ' INDENT' < / span > < span class = "p" > ,< / span > < span class = "mi" > 2< / span > < span class = "p" > ,< / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 2< / span > < span class = "p" > ]]< / span >
< span class = "nv" > indent.generated: < / span > < span class = "kc" > true< / span >
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > splice< / span > < span class = "nx" > i< / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "mi" > 0< / span > < span class = "p" > ,< / span > < span class = "nx" > indent< / span >
2010-03-06 23:48:06 -05:00
< span class = "nv" > idx: < / span > < span class = "nx" > i< / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span >
< span class = "nv" > parens: < / span > < span class = "mi" > 0< / span >
< span class = "k" > while< / span > < span class = "kc" > true< / span >
2010-03-23 00:18:50 -04:00
< span class = "nv" > idx: < / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span >
2010-04-03 12:01:46 -04:00
< span class = "nv" > tok: < / span > < span class = "nx" > @tokens< / span > < span class = "p" > [< / span > < span class = "nx" > idx< / span > < span class = "p" > ]< / span >
< span class = "nv" > pre: < / span > < span class = "nx" > @tokens< / span > < span class = "p" > [< / span > < span class = "nx" > idx< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > if< / span > < span class = "p" > (< / span > < span class = "o" > not< / span > < span class = "nx" > tok< / span > < span class = "o" > or< / span >
2010-03-07 14:41:52 -05:00
< span class = "p" > (< / span > < span class = "nx" > include< / span > < span class = "p" > (< / span > < span class = "nx" > SINGLE_CLOSERS< / span > < span class = "p" > ,< / span > < span class = "nx" > tok< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ])< / span > < span class = "o" > and< / span > < span class = "nx" > tok< / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "o" > isnt< / span > < span class = "s1" > ' ;' < / span > < span class = "p" > )< / span > < span class = "o" > or< / span >
2010-03-29 20:48:41 -04:00
< span class = "p" > (< / span > < span class = "nx" > tok< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' )' < / span > < span class = "o" > and< / span > < span class = "nx" > parens< / span > < span class = "o" > is< / span > < span class = "mi" > 0< / span > < span class = "p" > ))< / span > < span class = "o" > and< / span >
2010-03-06 23:48:06 -05:00
< span class = "o" > not< / span > < span class = "p" > (< / span > < span class = "nx" > starter< / span > < span class = "o" > is< / span > < span class = "s1" > ' ELSE' < / span > < span class = "o" > and< / span > < span class = "nx" > tok< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' ELSE' < / span > < span class = "p" > )< / span >
< span class = "nv" > insertion: < / span > < span class = "k" > if< / span > < span class = "nx" > pre< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s2" > " ," < / span > < span class = "k" > then< / span > < span class = "nx" > idx< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "k" > else< / span > < span class = "nx" > idx< / span >
2010-03-29 20:48:41 -04:00
< span class = "nv" > outdent: < / span > < span class = "p" > [< / span > < span class = "s1" > ' OUTDENT' < / span > < span class = "p" > ,< / span > < span class = "mi" > 2< / span > < span class = "p" > ,< / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 2< / span > < span class = "p" > ]]< / span >
< span class = "nv" > outdent.generated: < / span > < span class = "kc" > true< / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > splice< / span > < span class = "nx" > insertion< / span > < span class = "p" > ,< / span > < span class = "mi" > 0< / span > < span class = "p" > ,< / span > < span class = "nx" > outdent< / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > break< / span >
2010-03-23 00:18:50 -04:00
< span class = "nv" > parens: < / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span > < span class = "k" > if< / span > < span class = "nx" > tok< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' (' < / span >
< span class = "nv" > parens: < / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "k" > if< / span > < span class = "nx" > tok< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' )' < / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > return< / span > < span class = "mi" > 1< / span > < span class = "nx" > unless< / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' THEN' < / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > splice< / span > < span class = "nx" > i< / span > < span class = "p" > ,< / span > < span class = "mi" > 1< / span >
2010-03-16 23:18:54 -04:00
< span class = "k" > return< / span > < span class = "mi" > 0< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-13" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-13" > #< / a > < / div > < p > Ensure that all listed pairs of tokens are correctly balanced throughout
2010-03-06 23:48:06 -05:00
the course of the token stream.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > ensure_balance: < / span > < span class = "p" > (< / span > < span class = "nx" > pairs< / span > < span class = "p" > )< / span > < span class = "o" > -> < / span >
< span class = "nv" > levels: < / span > < span class = "p" > {}< / span >
2010-03-16 23:18:54 -04:00
< span class = "nv" > open_line: < / span > < span class = "p" > {}< / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @scan_tokens< / span > < span class = "p" > (< / span > < span class = "nx" > prev< / span > < span class = "p" > ,< / span > < span class = "nx" > token< / span > < span class = "p" > ,< / span > < span class = "nx" > post< / span > < span class = "p" > ,< / span > < span class = "nx" > i< / span > < span class = "p" > )< / span > < span class = "o" > => < / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > for< / span > < span class = "nx" > pair< / span > < span class = "k" > in< / span > < span class = "nx" > pairs< / span >
< span class = "p" > [< / span > < span class = "nx" > open< / span > < span class = "p" > ,< / span > < span class = "nx" > close< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "nx" > pair< / span >
2010-03-23 00:18:50 -04:00
< span class = "nx" > levels< / span > < span class = "p" > [< / span > < span class = "nx" > open< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "o" > or< / span > < span class = "mi" > 0< / span >
2010-03-16 23:18:54 -04:00
< span class = "k" > if< / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "nx" > open< / span >
< span class = "nx" > open_line< / span > < span class = "p" > [< / span > < span class = "nx" > open< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 2< / span > < span class = "p" > ]< / span > < span class = "k" > if< / span > < span class = "nx" > levels< / span > < span class = "p" > [< / span > < span class = "nx" > open< / span > < span class = "p" > ]< / span > < span class = "o" > ==< / span > < span class = "mi" > 0< / span >
2010-03-23 00:18:50 -04:00
< span class = "nx" > levels< / span > < span class = "p" > [< / span > < span class = "nx" > open< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span >
< span class = "nx" > levels< / span > < span class = "p" > [< / span > < span class = "nx" > open< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "k" > if< / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "nx" > close< / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > throw< / span > < span class = "k" > new< / span > < span class = "nb" > Error< / span > < span class = "p" > (< / span > < span class = "s2" > " too many ${token[1]} on line ${token[2] + 1}" < / span > < span class = "p" > )< / span > < span class = "k" > if< / span > < span class = "nx" > levels< / span > < span class = "p" > [< / span > < span class = "nx" > open< / span > < span class = "p" > ]< / span > < span class = "o" > < < / span > < span class = "mi" > 0< / span >
< span class = "k" > return< / span > < span class = "mi" > 1< / span >
< span class = "nv" > unclosed: < / span > < span class = "nx" > key< / span > < span class = "k" > for< / span > < span class = "nx" > key< / span > < span class = "p" > ,< / span > < span class = "nx" > value< / span > < span class = "k" > of< / span > < span class = "nx" > levels< / span > < span class = "k" > when< / span > < span class = "nx" > value< / span > < span class = "o" > > < / span > < span class = "mi" > 0< / span >
2010-03-16 23:18:54 -04:00
< span class = "k" > if< / span > < span class = "nx" > unclosed< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span >
< span class = "nv" > open: < / span > < span class = "nx" > unclosed< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
< span class = "nv" > line: < / span > < span class = "nx" > open_line< / span > < span class = "p" > [< / span > < span class = "nx" > open< / span > < span class = "p" > ]< / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span >
< span class = "k" > throw< / span > < span class = "k" > new< / span > < span class = "nb" > Error< / span > < span class = "s2" > " unclosed $open on line $line" < / span > < / pre > < / div > < / td > < / tr > < tr id = "section-14" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-14" > #< / a > < / div > < p > We'd like to support syntax like this:< / p >
2010-03-07 14:41:52 -05:00
< pre > < code > el.click((event) ->
el.hide())
< / code > < / pre >
< p > In order to accomplish this, move outdents that follow closing parens
2010-03-06 23:48:06 -05:00
inwards, safely. The steps to accomplish this are:< / p >
< ol >
< li > Check that all paired tokens are balanced and in order.< / li >
2010-03-29 21:22:12 -04:00
< li > Rewrite the stream with a stack: if you see an < code > EXPRESSION_START< / code > , add it
to the stack. If you see an < code > EXPRESSION_END< / code > , pop the stack and replace
2010-03-06 23:48:06 -05:00
it with the inverse of what we've just popped.< / li >
2010-03-29 21:22:12 -04:00
< li > Keep track of "debt" for tokens that we manufacture, to make sure we end
2010-03-06 23:48:06 -05:00
up balanced in the end.< / li >
2010-05-04 23:31:28 -04:00
< li > Be careful not to alter array or parentheses delimiters with overzealous
rewriting.< / li >
2010-03-06 23:48:06 -05:00
< / ol > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > rewrite_closing_parens: < / span > < span class = "o" > -> < / span >
< span class = "nv" > stack: < / span > < span class = "p" > []< / span >
< span class = "nv" > debt: < / span > < span class = "p" > {}< / span >
< span class = "p" > (< / span > < span class = "nx" > debt< / span > < span class = "p" > [< / span > < span class = "nx" > key< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "mi" > 0< / span > < span class = "p" > )< / span > < span class = "k" > for< / span > < span class = "nx" > key< / span > < span class = "p" > ,< / span > < span class = "nx" > val< / span > < span class = "k" > of< / span > < span class = "nx" > INVERSES< / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @scan_tokens< / span > < span class = "p" > (< / span > < span class = "nx" > prev< / span > < span class = "p" > ,< / span > < span class = "nx" > token< / span > < span class = "p" > ,< / span > < span class = "nx" > post< / span > < span class = "p" > ,< / span > < span class = "nx" > i< / span > < span class = "p" > )< / span > < span class = "o" > => < / span >
2010-03-06 23:48:06 -05:00
< span class = "nv" > tag: < / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
2010-03-07 14:41:52 -05:00
< span class = "nv" > inv: < / span > < span class = "nx" > INVERSES< / span > < span class = "p" > [< / span > < span class = "nx" > token< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]]< / span >
< span class = "k" > if< / span > < span class = "nx" > include< / span > < span class = "nx" > EXPRESSION_START< / span > < span class = "p" > ,< / span > < span class = "nx" > tag< / span >
< span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "nx" > token< / span >
< span class = "k" > return< / span > < span class = "mi" > 1< / span >
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > include< / span > < span class = "nx" > EXPRESSION_END< / span > < span class = "p" > ,< / span > < span class = "nx" > tag< / span >
< span class = "k" > if< / span > < span class = "nx" > debt< / span > < span class = "p" > [< / span > < span class = "nx" > inv< / span > < span class = "p" > ]< / span > < span class = "o" > > < / span > < span class = "mi" > 0< / span >
2010-03-23 00:18:50 -04:00
< span class = "nx" > debt< / span > < span class = "p" > [< / span > < span class = "nx" > inv< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > splice< / span > < span class = "nx" > i< / span > < span class = "p" > ,< / span > < span class = "mi" > 1< / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > return< / span > < span class = "mi" > 0< / span >
2010-03-07 14:41:52 -05:00
< span class = "k" > else< / span >
< span class = "nv" > match: < / span > < span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span >
< span class = "nv" > mtag: < / span > < span class = "nx" > match< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
2010-05-04 23:31:28 -04:00
< span class = "nv" > oppos: < / span > < span class = "nx" > INVERSES< / span > < span class = "p" > [< / span > < span class = "nx" > mtag< / span > < span class = "p" > ]< / span >
< span class = "k" > return< / span > < span class = "mi" > 1< / span > < span class = "k" > if< / span > < span class = "nx" > tag< / span > < span class = "o" > is< / span > < span class = "nx" > oppos< / span >
2010-03-23 00:18:50 -04:00
< span class = "nx" > debt< / span > < span class = "p" > [< / span > < span class = "nx" > mtag< / span > < span class = "p" > ]< / span > < span class = "o" > :< / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span >
2010-05-04 23:31:28 -04:00
< span class = "nv" > val: < / span > < span class = "p" > [< / span > < span class = "nx" > oppos< / span > < span class = "p" > ,< / span > < span class = "k" > if< / span > < span class = "nx" > mtag< / span > < span class = "o" > is< / span > < span class = "s1" > ' INDENT' < / span > < span class = "k" > then< / span > < span class = "nx" > match< / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "k" > else< / span > < span class = "nx" > oppos< / span > < span class = "p" > ]< / span >
< span class = "k" > if< / span > < span class = "nx" > @tokens< / span > < span class = "p" > [< / span > < span class = "nx" > i< / span > < span class = "o" > +< / span > < span class = "mi" > 2< / span > < span class = "p" > ]< / span > < span class = "o" > ?< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "nx" > mtag< / span >
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > splice< / span > < span class = "nx" > i< / span > < span class = "o" > +< / span > < span class = "mi" > 3< / span > < span class = "p" > ,< / span > < span class = "mi" > 0< / span > < span class = "p" > ,< / span > < span class = "nx" > val< / span >
< span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "p" > (< / span > < span class = "nx" > match< / span > < span class = "p" > )< / span >
< span class = "k" > else< / span >
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > splice< / span > < span class = "nx" > i< / span > < span class = "p" > ,< / span > < span class = "mi" > 0< / span > < span class = "p" > ,< / span > < span class = "nx" > val< / span >
2010-03-07 14:41:52 -05:00
< span class = "k" > return< / span > < span class = "mi" > 1< / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > else< / span >
2010-03-16 23:18:54 -04:00
< span class = "k" > return< / span > < span class = "mi" > 1< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-15" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-15" > #< / a > < / div > < h2 > Constants< / h2 > < / td > < td class = "code" > < div class = "highlight" > < pre > < / pre > < / div > < / td > < / tr > < tr id = "section-16" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-16" > #< / a > < / div > < p > List of the token pairs that must be balanced.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > BALANCED_PAIRS: < / span > < span class = "p" > [[< / span > < span class = "s1" > ' (' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' )' < / span > < span class = "p" > ],< / span > < span class = "p" > [< / span > < span class = "s1" > ' [' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ]' < / span > < span class = "p" > ],< / span > < span class = "p" > [< / span > < span class = "s1" > ' {' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' }' < / span > < span class = "p" > ],< / span > < span class = "p" > [< / span > < span class = "s1" > ' INDENT' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' OUTDENT' < / span > < span class = "p" > ],< / span >
2010-03-07 14:41:52 -05:00
< span class = "p" > [< / span > < span class = "s1" > ' PARAM_START' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' PARAM_END' < / span > < span class = "p" > ],< / span > < span class = "p" > [< / span > < span class = "s1" > ' CALL_START' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' CALL_END' < / span > < span class = "p" > ],< / span >
2010-03-16 23:18:54 -04:00
< span class = "p" > [< / span > < span class = "s1" > ' INDEX_START' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' INDEX_END' < / span > < span class = "p" > ],< / span > < span class = "p" > [< / span > < span class = "s1" > ' SOAKED_INDEX_START' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' SOAKED_INDEX_END' < / span > < span class = "p" > ]]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-17" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-17" > #< / a > < / div > < p > The inverse mappings of < code > BALANCED_PAIRS< / code > we're trying to fix up, so we can
2010-03-07 14:41:52 -05:00
look things up from either end.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > INVERSES: < / span > < span class = "p" > {}< / span >
< span class = "k" > for< / span > < span class = "nx" > pair< / span > < span class = "k" > in< / span > < span class = "nx" > BALANCED_PAIRS< / span >
< span class = "nx" > INVERSES< / span > < span class = "p" > [< / span > < span class = "nx" > pair< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]]< / span > < span class = "o" > :< / span > < span class = "nx" > pair< / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span >
2010-05-15 01:18:05 -04:00
< span class = "nx" > INVERSES< / span > < span class = "p" > [< / span > < span class = "nx" > pair< / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "p" > ]]< / span > < span class = "o" > :< / span > < span class = "nx" > pair< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-18" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-18" > #< / a > < / div > < p > The tokens that signal the start of a balanced pair.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > EXPRESSION_START: < / span > < span class = "nx" > pair< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "k" > for< / span > < span class = "nx" > pair< / span > < span class = "k" > in< / span > < span class = "nx" > BALANCED_PAIRS< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-19" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-19" > #< / a > < / div > < p > The tokens that signal the end of a balanced pair.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > EXPRESSION_END: < / span > < span class = "nx" > pair< / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "k" > for< / span > < span class = "nx" > pair< / span > < span class = "k" > in< / span > < span class = "nx" > BALANCED_PAIRS< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-20" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-20" > #< / a > < / div > < p > Tokens that indicate the close of a clause of an expression.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > EXPRESSION_CLOSE: < / span > < span class = "p" > [< / span > < span class = "s1" > ' CATCH' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' WHEN' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ELSE' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' FINALLY' < / span > < span class = "p" > ].< / span > < span class = "nx" > concat< / span > < span class = "nx" > EXPRESSION_END< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-21" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-21" > #< / a > < / div > < p > Tokens that, if followed by an < code > IMPLICIT_CALL< / code > , indicate a function invocation.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > IMPLICIT_FUNC: < / span > < span class = "p" > [< / span > < span class = "s1" > ' IDENTIFIER' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' SUPER' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' )' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' CALL_END' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ]' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' INDEX_END' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' < -' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' @' < / span > < span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-22" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-22" > #< / a > < / div > < p > If preceded by an < code > IMPLICIT_FUNC< / code > , indicates a function invocation.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > IMPLICIT_CALL: < / span > < span class = "p" > [< / span > < span class = "s1" > ' IDENTIFIER' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' NUMBER' < / span > < span c
2010-03-09 23:53:21 -05:00
< span class = "s1" > ' TRY' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' DELETE' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' TYPEOF' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' SWITCH' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' EXTENSION' < / span > < span class = "p" > ,< / span >
2010-03-07 14:41:52 -05:00
< span class = "s1" > ' TRUE' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' FALSE' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' YES' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' NO' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ON' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' OFF' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' !' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' !!' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' NOT' < / span > < span class = "p" > ,< / span >
2010-03-29 20:48:41 -04:00
< span class = "s1" > ' THIS' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' NULL' < / span > < span class = "p" > ,< / span >
2010-05-04 23:31:28 -04:00
< span class = "s1" > ' @' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' -> ' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' => ' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' [' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' (' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' {' < / span > < span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-23" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-23" > #< / a > < / div > < p > Tokens indicating that the implicit call must enclose a block of expressions.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > IMPLICIT_BLOCK: < / span > < span class = "p" > [< / span > < span class = "s1" > ' -> ' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' => ' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' {' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' [' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ,' < / span > < span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-24" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-24" > #< / a > < / div > < p > Tokens that always mark the end of an implicit call for single-liners.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > IMPLICIT_END: < / span > < span class = "p" > [< / span > < span class = "s1" > ' IF' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' UNLESS' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' FOR' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' WHILE' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' UNTIL' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' TERMINATOR' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' INDENT' < / span > < span class = "p" > ].< / span > < span class = "nx" > concat< / span > < span class = "nx" > EXPRESSION_END< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-25" > < td class = "docs" > < div class = "octowrap" > < a class = "octothorpe" href = "#section-25" > #< / a > < / div > < p > Single-line flavors of block expressions that have unclosed endings.
2010-03-07 14:41:52 -05:00
The grammar can't disambiguate them, so we insert the implicit indentation.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > SINGLE_LINERS: < / span > < span class = "p" > [< / span > < span class = "s1" > ' ELSE' < / span > < span class = "p" > ,< / span > < span class = "s2" > " -> " < / span > < span class = "p" > ,< / span > < span class = "s2" > " => " < / span > < span class = "p" > ,< / span > < span class = "s1" > ' TRY' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' FINALLY' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' THEN' < / span > < span class = "p" > ]< / span >
2010-03-09 23:53:21 -05:00
< span class = "nv" > SINGLE_CLOSERS: < / span > < span class = "p" > [< / span > < span class = "s1" > ' TERMINATOR' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' CATCH' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' FINALLY' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ELSE' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' OUTDENT' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' LEADING_WHEN' < / span > < span class = "p" > ]< / span >
2010-03-06 23:48:06 -05:00
< / pre > < / div > < / td > < / tr > < / tbody > < / table > < / div > < / body > < / html >
