2010-11-20 14:22:28 -05:00
<!DOCTYPE html> < html > < head > < title > lexer.coffee< / title > < meta http-equiv = "content-type" content = "text/html; charset=UTF-8" > < link rel = "stylesheet" media = "all" href = "docco.css" / > < / head > < body > < div id = "container" > < div id = "background" > < / div > < div id = "jump_to" > Jump To … < div id = "jump_wrapper" > < div id = "jump_page" > < a class = "source" href = "browser.html" > browser.coffee < / a > < a class = "source" href = "cake.html" > cake.coffee < / a > < a class = "source" href = "coffee-script.html" > coffee-script.coffee < / a > < a class = "source" href = "command.html" > command.coffee < / a > < a class = "source" href = "grammar.html" > grammar.coffee < / a > < a class = "source" href = "helpers.html" > helpers.coffee < / a > < a class = "source" href = "index.html" > index.coffee < / a > < a class = "source" href = "lexer.html" > lexer.coffee < / a > < a class = "source" href = "nodes.html" > nodes.coffee < / a > < a class = "source" href = "optparse.html" > optparse.coffee < / a > < a class = "source" href = "repl.html" > repl.coffee < / a > < a class = "source" href = "rewriter.html" > rewriter.coffee < / a > < a class = "source" href = "scope.html" > scope.coffee < / a > < / div > < / div > < / div > < table cellpadding = "0" cellspacing = "0" > < thead > < tr > < th class = "docs" > < h1 > lexer.coffee < / h1 > < / th > < th class = "code" > < / th > < / tr > < / thead > < tbody > < tr id = "section-1" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-1" > ¶ < / a > < / div > < p > The CoffeeScript Lexer. Uses a series of token-matching regexes to attempt
2010-03-06 23:48:06 -05:00
matches against the beginning of the source code. When a match is found,
a token is produced, we consume the match, and start again. Tokens are in the
form:< / p >
2010-06-28 00:19:58 -04:00
< pre > < code > [tag, value, lineNumber]
2010-03-06 23:48:06 -05:00
< / code > < / pre >
2011-11-08 18:01:45 -05:00
< p > Which is a format that can be fed directly into < a href = "http://github.com/zaach/jison" > Jison< / a > .< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "p" > {< / span > < span class = "nx" > Rewriter< / span > < span class = "p" > ,< / span > < span class = "nx" > INVERSES< / span > < span class = "p" > }< / span > < span class = "o" > =< / span > < span class = "nx" > require< / span > < span class = "s1" > ' ./rewriter' < / span > < / pre > < / div > < / td > < / tr > < tr id = "section-2" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-2" > ¶ < / a > < / div > < p > Import the helpers we need.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "p" > {< / span > < span class = "nx" > count< / span > < span class = "p" > ,< / span > < span class = "nx" > starts< / span > < span class = "p" > ,< / span > < span class = "nx" > compact< / span > < span class = "p" > ,< / span > < span class = "nx" > last< / span > < span class = "p" > }< / span > < span class = "o" > =< / span > < span class = "nx" > require< / span > < span class = "s1" > ' ./helpers' < / span > < / pre > < / div > < / td > < / tr > < tr id = "section-3" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-3" > ¶ < / a > < / div > < h2 > The Lexer Class< / h2 > < / td > < td class = "code" > < div class = "highlight" > < pre > < / pre > < / div > < / td > < / tr > < tr id = "section-4" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-4" > ¶ < / a > < / div > < p > The Lexer class reads a stream of CoffeeScript and divvies it up into tagged
2010-03-07 14:41:52 -05:00
tokens. Some potential ambiguity in the grammar has been avoided by
2011-12-18 13:21:00 -05:00
pushing some extra smarts into the Lexer.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > exports.Lexer = < / span > < span class = "k" > class< / span > < span class = "nx" > Lexer< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-5" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-5" > ¶ < / a > < / div > < p > < strong > tokenize< / strong > is the Lexer's main method. Scan by attempting to match tokens
2010-03-07 12:47:03 -05:00
one at a time, using a regular expression anchored at the start of the
remaining code, or a custom recursive token-matching method
(for interpolations). When the next token has been recorded, we move forward
within the code past the token, and begin again.< / p >
2010-11-20 14:22:28 -05:00
< p > Each tokenizing method is responsible for returning the number of characters
it has consumed.< / p >
2010-03-07 14:41:52 -05:00
< p > Before returning the token stream, run it through the < a href = "rewriter.html" > Rewriter< / a >
2011-12-18 13:21:00 -05:00
unless explicitly asked not to.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > tokenize: < / span > < span class = "nf" > (code, opts = {}) -> < / span >
2011-01-31 22:39:12 -05:00
< span class = "nv" > code = < / span > < span class = "s2" > " \n#{code}" < / span > < span class = "k" > if< / span > < span class = "nx" > WHITESPACE< / span > < span class = "p" > .< / span > < span class = "nx" > test< / span > < span class = "nx" > code< / span >
2010-11-20 14:22:28 -05:00
< span class = "nv" > code = < / span > < span class = "nx" > code< / span > < span class = "p" > .< / span > < span class = "nx" > replace< / span > < span class = "p" > (< / span > < span class = "sr" > /\r/g< / span > < span class = "p" > ,< / span > < span class = "s1" > ' ' < / span > < span class = "p" > ).< / span > < span class = "nx" > replace< / span > < span class = "nx" > TRAILING_SPACES< / span > < span class = "p" > ,< / span > < span class = "s1" > ' ' < / span >
< span class = "vi" > @code = < / span > < span class = "nx" > code< / span > < span class = "c1" > # The remainder of the source code.< / span >
< span class = "vi" > @line = < / span > < span class = "nx" > opts< / span > < span class = "p" > .< / span > < span class = "nx" > line< / span > < span class = "o" > or< / span > < span class = "mi" > 0< / span > < span class = "c1" > # The current line.< / span >
< span class = "vi" > @indent = < / span > < span class = "mi" > 0< / span > < span class = "c1" > # The current indentation level.< / span >
< span class = "vi" > @indebt = < / span > < span class = "mi" > 0< / span > < span class = "c1" > # The over-indentation at the current level.< / span >
< span class = "vi" > @outdebt = < / span > < span class = "mi" > 0< / span > < span class = "c1" > # The under-outdentation at the current level.< / span >
< span class = "vi" > @indents = < / span > < span class = "p" > []< / span > < span class = "c1" > # The stack of all current indentation levels.< / span >
2011-11-08 18:01:45 -05:00
< span class = "vi" > @ends = < / span > < span class = "p" > []< / span > < span class = "c1" > # The stack for pairing up tokens.< / span >
2010-11-21 12:38:27 -05:00
< span class = "vi" > @tokens = < / span > < span class = "p" > []< / span > < span class = "c1" > # Stream of parsed tokens in the form `[' TYPE' , value, line]`.< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-6" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-6" > ¶ < / a > < / div > < p > At every position, run through this list of attempted matches,
2010-03-07 12:47:03 -05:00
short-circuiting if any of them succeed. Their order determines precedence:
2010-11-20 14:22:28 -05:00
< code > @literalToken< / code > is the fallback catch-all.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > i = < / span > < span class = "mi" > 0< / span >
< span class = "k" > while< / span > < span class = "vi" > @chunk = < / span > < span class = "nx" > code< / span > < span class = "p" > .< / span > < span class = "nx" > slice< / span > < span class = "nx" > i< / span >
< span class = "nx" > i< / span > < span class = "o" > +=< / span > < span class = "nx" > @identifierToken< / span > < span class = "p" > ()< / span > < span class = "o" > or< / span >
< span class = "nx" > @commentToken< / span > < span class = "p" > ()< / span > < span class = "o" > or< / span >
< span class = "nx" > @whitespaceToken< / span > < span class = "p" > ()< / span > < span class = "o" > or< / span >
< span class = "nx" > @lineToken< / span > < span class = "p" > ()< / span > < span class = "o" > or< / span >
< span class = "nx" > @heredocToken< / span > < span class = "p" > ()< / span > < span class = "o" > or< / span >
< span class = "nx" > @stringToken< / span > < span class = "p" > ()< / span > < span class = "o" > or< / span >
< span class = "nx" > @numberToken< / span > < span class = "p" > ()< / span > < span class = "o" > or< / span >
< span class = "nx" > @regexToken< / span > < span class = "p" > ()< / span > < span class = "o" > or< / span >
< span class = "nx" > @jsToken< / span > < span class = "p" > ()< / span > < span class = "o" > or< / span >
< span class = "nx" > @literalToken< / span > < span class = "p" > ()< / span >
< span class = "nx" > @closeIndentation< / span > < span class = "p" > ()< / span >
2011-11-08 18:01:45 -05:00
< span class = "nx" > @error< / span > < span class = "s2" > " missing #{tag}" < / span > < span class = "k" > if< / span > < span class = "nv" > tag = < / span > < span class = "nx" > @ends< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > return< / span > < span class = "nx" > @tokens< / span > < span class = "k" > if< / span > < span class = "nx" > opts< / span > < span class = "p" > .< / span > < span class = "nx" > rewrite< / span > < span class = "o" > is< / span > < span class = "kc" > off< / span >
< span class = "p" > (< / span > < span class = "k" > new< / span > < span class = "nx" > Rewriter< / span > < span class = "p" > ).< / span > < span class = "nx" > rewrite< / span > < span class = "nx" > @tokens< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-7" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-7" > ¶ < / a > < / div > < h2 > Tokenizers< / h2 > < / td > < td class = "code" > < div class = "highlight" > < pre > < / pre > < / div > < / td > < / tr > < tr id = "section-8" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-8" > ¶ < / a > < / div > < p > Matches identifying literals: variables, keywords, method names, etc.
2010-03-07 12:47:03 -05:00
Check to ensure that JavaScript reserved words aren't being used as
identifiers. Because CoffeeScript reserves a handful of keywords that are
allowed in JavaScript, we're careful not to tag them as keywords when
referenced as property names here, so you can still do < code > jQuery.is()< / code > even
2011-12-18 13:21:00 -05:00
though < code > is< / code > means < code > ===< / code > otherwise.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > identifierToken: < / span > < span class = "o" > -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > return< / span > < span class = "mi" > 0< / span > < span class = "nx" > unless< / span > < span class = "nv" > match = < / span > < span class = "nx" > IDENTIFIER< / span > < span class = "p" > .< / span > < span class = "nx" > exec< / span > < span class = "nx" > @chunk< / span >
< span class = "p" > [< / span > < span class = "nx" > input< / span > < span class = "p" > ,< / span > < span class = "nx" > id< / span > < span class = "p" > ,< / span > < span class = "nx" > colon< / span > < span class = "p" > ]< / span > < span class = "o" > =< / span > < span class = "nx" > match< / span >
2010-12-07 00:21:34 -05:00
< span class = "k" > if< / span > < span class = "nx" > id< / span > < span class = "o" > is< / span > < span class = "s1" > ' own' < / span > < span class = "o" > and< / span > < span class = "nx" > @tag< / span > < span class = "p" > ()< / span > < span class = "o" > is< / span > < span class = "s1" > ' FOR' < / span >
< span class = "nx" > @token< / span > < span class = "s1" > ' OWN' < / span > < span class = "p" > ,< / span > < span class = "nx" > id< / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > return< / span > < span class = "nx" > id< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span >
< span class = "nv" > forcedIdentifier = < / span > < span class = "nx" > colon< / span > < span class = "o" > or< / span >
2011-05-01 12:08:15 -04:00
< span class = "p" > (< / span > < span class = "nv" > prev = < / span > < span class = "nx" > last< / span > < span class = "nx" > @tokens< / span > < span class = "p" > )< / span > < span class = "o" > and< / span > < span class = "p" > (< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "k" > in< / span > < span class = "p" > [< / span > < span class = "s1" > ' .' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ?.' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ::' < / span > < span class = "p" > ]< / span > < span class = "o" > or< / span >
< span class = "o" > not< / span > < span class = "nx" > prev< / span > < span class = "p" > .< / span > < span class = "nx" > spaced< / span > < span class = "o" > and< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' @' < / span > < span class = "p" > )< / span >
2010-08-04 23:14:34 -04:00
< span class = "nv" > tag = < / span > < span class = "s1" > ' IDENTIFIER' < / span >
2010-11-20 14:22:28 -05:00
2011-08-04 23:17:23 -04:00
< span class = "k" > if< / span > < span class = "o" > not< / span > < span class = "nx" > forcedIdentifier< / span > < span class = "o" > and< / span > < span class = "p" > (< / span > < span class = "nx" > id< / span > < span class = "k" > in< / span > < span class = "nx" > JS_KEYWORDS< / span > < span class = "o" > or< / span > < span class = "nx" > id< / span > < span class = "k" > in< / span > < span class = "nx" > COFFEE_KEYWORDS< / span > < span class = "p" > )< / span >
2010-11-20 14:22:28 -05:00
< span class = "nv" > tag = < / span > < span class = "nx" > id< / span > < span class = "p" > .< / span > < span class = "nx" > toUpperCase< / span > < span class = "p" > ()< / span >
< span class = "k" > if< / span > < span class = "nx" > tag< / span > < span class = "o" > is< / span > < span class = "s1" > ' WHEN' < / span > < span class = "o" > and< / span > < span class = "nx" > @tag< / span > < span class = "p" > ()< / span > < span class = "k" > in< / span > < span class = "nx" > LINE_BREAK< / span >
< span class = "nv" > tag = < / span > < span class = "s1" > ' LEADING_WHEN' < / span >
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > tag< / span > < span class = "o" > is< / span > < span class = "s1" > ' FOR' < / span >
< span class = "vi" > @seenFor = < / span > < span class = "kc" > yes< / span >
2010-12-24 11:02:10 -08:00
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > tag< / span > < span class = "o" > is< / span > < span class = "s1" > ' UNLESS' < / span >
< span class = "nv" > tag = < / span > < span class = "s1" > ' IF' < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > tag< / span > < span class = "k" > in< / span > < span class = "nx" > UNARY< / span >
< span class = "nv" > tag = < / span > < span class = "s1" > ' UNARY' < / span >
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > tag< / span > < span class = "k" > in< / span > < span class = "nx" > RELATION< / span >
< span class = "k" > if< / span > < span class = "nx" > tag< / span > < span class = "o" > isnt< / span > < span class = "s1" > ' INSTANCEOF' < / span > < span class = "o" > and< / span > < span class = "nx" > @seenFor< / span >
< span class = "nv" > tag = < / span > < span class = "s1" > ' FOR' < / span > < span class = "o" > +< / span > < span class = "nx" > tag< / span >
2010-12-24 11:02:10 -08:00
< span class = "vi" > @seenFor = < / span > < span class = "kc" > no< / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > else< / span >
< span class = "nv" > tag = < / span > < span class = "s1" > ' RELATION' < / span >
< span class = "k" > if< / span > < span class = "nx" > @value< / span > < span class = "p" > ()< / span > < span class = "o" > is< / span > < span class = "s1" > ' !' < / span >
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span >
< span class = "nv" > id = < / span > < span class = "s1" > ' !' < / span > < span class = "o" > +< / span > < span class = "nx" > id< / span >
2011-11-08 18:01:45 -05:00
< span class = "k" > if< / span > < span class = "nx" > id< / span > < span class = "k" > in< / span > < span class = "p" > [< / span > < span class = "s1" > ' eval' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' arguments' < / span > < span class = "p" > ].< / span > < span class = "nx" > concat< / span > < span class = "nx" > JS_FORBIDDEN< / span >
2010-06-28 00:19:58 -04:00
< span class = "k" > if< / span > < span class = "nx" > forcedIdentifier< / span >
2010-11-20 14:22:28 -05:00
< span class = "nv" > tag = < / span > < span class = "s1" > ' IDENTIFIER' < / span >
< span class = "nv" > id = < / span > < span class = "k" > new< / span > < span class = "nb" > String< / span > < span class = "nx" > id< / span >
< span class = "nv" > id.reserved = < / span > < span class = "kc" > yes< / span >
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > id< / span > < span class = "k" > in< / span > < span class = "nx" > RESERVED< / span >
2011-12-18 13:21:00 -05:00
< span class = "nx" > @error< / span > < span class = "s2" > " reserved word \" #{id}\" " < / span >
2010-11-20 14:22:28 -05:00
2010-06-28 00:19:58 -04:00
< span class = "nx" > unless< / span > < span class = "nx" > forcedIdentifier< / span >
2011-05-01 12:08:15 -04:00
< span class = "nv" > id = < / span > < span class = "nx" > COFFEE_ALIAS_MAP< / span > < span class = "p" > [< / span > < span class = "nx" > id< / span > < span class = "p" > ]< / span > < span class = "k" > if< / span > < span class = "nx" > id< / span > < span class = "k" > in< / span > < span class = "nx" > COFFEE_ALIASES< / span >
2010-11-20 14:22:28 -05:00
< span class = "nv" > tag = < / span > < span class = "k" > switch< / span > < span class = "nx" > id< / span >
< span class = "k" > when< / span > < span class = "s1" > ' !' < / span > < span class = "k" > then< / span > < span class = "s1" > ' UNARY' < / span >
< span class = "k" > when< / span > < span class = "s1" > ' ==' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' !=' < / span > < span class = "k" > then< / span > < span class = "s1" > ' COMPARE' < / span >
< span class = "k" > when< / span > < span class = "s1" > ' & & ' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ||' < / span > < span class = "k" > then< / span > < span class = "s1" > ' LOGIC' < / span >
< span class = "k" > when< / span > < span class = "s1" > ' true' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' false' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' null' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' undefined' < / span > < span class = "k" > then< / span > < span class = "s1" > ' BOOL' < / span >
2011-12-18 13:21:00 -05:00
< span class = "k" > when< / span > < span class = "s1" > ' break' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' continue' < / span > < span class = "k" > then< / span > < span class = "s1" > ' STATEMENT' < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > else< / span > < span class = "nx" > tag< / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @token< / span > < span class = "nx" > tag< / span > < span class = "p" > ,< / span > < span class = "nx" > id< / span >
2010-11-20 14:22:28 -05:00
< span class = "nx" > @token< / span > < span class = "s1" > ' :' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' :' < / span > < span class = "k" > if< / span > < span class = "nx" > colon< / span >
< span class = "nx" > input< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-9" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-9" > ¶ < / a > < / div > < p > Matches numbers, including decimals, hex, and exponential notation.
2011-12-18 13:21:00 -05:00
Be careful not to interfere with ranges-in-progress.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > numberToken: < / span > < span class = "o" > -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > return< / span > < span class = "mi" > 0< / span > < span class = "nx" > unless< / span > < span class = "nv" > match = < / span > < span class = "nx" > NUMBER< / span > < span class = "p" > .< / span > < span class = "nx" > exec< / span > < span class = "nx" > @chunk< / span >
< span class = "nv" > number = < / span > < span class = "nx" > match< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
2011-11-08 18:01:45 -05:00
< span class = "nv" > lexedLength = < / span > < span class = "nx" > number< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span >
< span class = "k" > if< / span > < span class = "nv" > binaryLiteral = < / span > < span class = "sr" > /0b([01]+)/< / span > < span class = "p" > .< / span > < span class = "nx" > exec< / span > < span class = "nx" > number< / span >
< span class = "nv" > number = < / span > < span class = "p" > (< / span > < span class = "nb" > parseInt< / span > < span class = "nx" > binaryLiteral< / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "p" > ],< / span > < span class = "mi" > 2< / span > < span class = "p" > ).< / span > < span class = "nx" > toString< / span > < span class = "p" > ()< / span >
2010-06-28 00:19:58 -04:00
< span class = "nx" > @token< / span > < span class = "s1" > ' NUMBER' < / span > < span class = "p" > ,< / span > < span class = "nx" > number< / span >
2011-11-08 18:01:45 -05:00
< span class = "nx" > lexedLength< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-10" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-10" > ¶ < / a > < / div > < p > Matches strings, including multi-line strings. Ensures that quotation marks
2011-12-18 13:21:00 -05:00
are balanced within the string's contents, and within nested interpolations.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > stringToken: < / span > < span class = "o" > -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > switch< / span > < span class = "nx" > @chunk< / span > < span class = "p" > .< / span > < span class = "nx" > charAt< / span > < span class = "mi" > 0< / span >
< span class = "k" > when< / span > < span class = "s2" > " ' " < / span >
< span class = "k" > return< / span > < span class = "mi" > 0< / span > < span class = "nx" > unless< / span > < span class = "nv" > match = < / span > < span class = "nx" > SIMPLESTR< / span > < span class = "p" > .< / span > < span class = "nx" > exec< / span > < span class = "nx" > @chunk< / span >
< span class = "nx" > @token< / span > < span class = "s1" > ' STRING' < / span > < span class = "p" > ,< / span > < span class = "p" > (< / span > < span class = "nv" > string = < / span > < span class = "nx" > match< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]).< / span > < span class = "nx" > replace< / span > < span class = "nx" > MULTILINER< / span > < span class = "p" > ,< / span > < span class = "s1" > ' \\\n' < / span >
< span class = "k" > when< / span > < span class = "s1" > ' " ' < / span >
2010-12-24 11:02:10 -08:00
< span class = "k" > return< / span > < span class = "mi" > 0< / span > < span class = "nx" > unless< / span > < span class = "nv" > string = < / span > < span class = "nx" > @balancedString< / span > < span class = "nx" > @chunk< / span > < span class = "p" > ,< / span > < span class = "s1" > ' " ' < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > if< / span > < span class = "mi" > 0< / span > < span class = "o" > < < / span > < span class = "nx" > string< / span > < span class = "p" > .< / span > < span class = "nx" > indexOf< / span > < span class = "s1" > ' #{' < / span > < span class = "p" > ,< / span > < span class = "mi" > 1< / span >
< span class = "nx" > @interpolateString< / span > < span class = "nx" > string< / span > < span class = "p" > .< / span > < span class = "nx" > slice< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span >
< span class = "k" > else< / span >
< span class = "nx" > @token< / span > < span class = "s1" > ' STRING' < / span > < span class = "p" > ,< / span > < span class = "nx" > @escapeLines< / span > < span class = "nx" > string< / span >
< span class = "k" > else< / span >
< span class = "k" > return< / span > < span class = "mi" > 0< / span >
< span class = "nx" > @line< / span > < span class = "o" > +=< / span > < span class = "nx" > count< / span > < span class = "nx" > string< / span > < span class = "p" > ,< / span > < span class = "s1" > ' \n' < / span >
< span class = "nx" > string< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-11" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-11" > ¶ < / a > < / div > < p > Matches heredocs, adjusting indentation to the correct level, as heredocs
2011-12-18 13:21:00 -05:00
preserve whitespace, but ignore indentation to the left.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > heredocToken: < / span > < span class = "o" > -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > return< / span > < span class = "mi" > 0< / span > < span class = "nx" > unless< / span > < span class = "nv" > match = < / span > < span class = "nx" > HEREDOC< / span > < span class = "p" > .< / span > < span class = "nx" > exec< / span > < span class = "nx" > @chunk< / span >
< span class = "nv" > heredoc = < / span > < span class = "nx" > match< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
< span class = "nv" > quote = < / span > < span class = "nx" > heredoc< / span > < span class = "p" > .< / span > < span class = "nx" > charAt< / span > < span class = "mi" > 0< / span >
2011-12-18 13:21:00 -05:00
< span class = "nv" > doc = < / span > < span class = "nx" > @sanitizeHeredoc< / span > < span class = "nx" > match< / span > < span class = "p" > [< / span > < span class = "mi" > 2< / span > < span class = "p" > ],< / span > < span class = "nv" > quote: < / span > < span class = "nx" > quote< / span > < span class = "p" > ,< / span > < span class = "nv" > indent: < / span > < span class = "kc" > null< / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > if< / span > < span class = "nx" > quote< / span > < span class = "o" > is< / span > < span class = "s1" > ' " ' < / span > < span class = "o" > and< / span > < span class = "mi" > 0< / span > < span class = "o" > < =< / span > < span class = "nx" > doc< / span > < span class = "p" > .< / span > < span class = "nx" > indexOf< / span > < span class = "s1" > ' #{' < / span >
2011-12-18 13:21:00 -05:00
< span class = "nx" > @interpolateString< / span > < span class = "nx" > doc< / span > < span class = "p" > ,< / span > < span class = "nv" > heredoc: < / span > < span class = "kc" > yes< / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > else< / span >
< span class = "nx" > @token< / span > < span class = "s1" > ' STRING' < / span > < span class = "p" > ,< / span > < span class = "nx" > @makeString< / span > < span class = "nx" > doc< / span > < span class = "p" > ,< / span > < span class = "nx" > quote< / span > < span class = "p" > ,< / span > < span class = "kc" > yes< / span >
< span class = "nx" > @line< / span > < span class = "o" > +=< / span > < span class = "nx" > count< / span > < span class = "nx" > heredoc< / span > < span class = "p" > ,< / span > < span class = "s1" > ' \n' < / span >
2011-12-18 13:21:00 -05:00
< span class = "nx" > heredoc< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-12" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-12" > ¶ < / a > < / div > < p > Matches and consumes comments.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > commentToken: < / span > < span class = "o" > -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > return< / span > < span class = "mi" > 0< / span > < span class = "nx" > unless< / span > < span class = "nv" > match = < / span > < span class = "nx" > @chunk< / span > < span class = "p" > .< / span > < span class = "nx" > match< / span > < span class = "nx" > COMMENT< / span >
< span class = "p" > [< / span > < span class = "nx" > comment< / span > < span class = "p" > ,< / span > < span class = "nx" > here< / span > < span class = "p" > ]< / span > < span class = "o" > =< / span > < span class = "nx" > match< / span >
< span class = "k" > if< / span > < span class = "nx" > here< / span >
< span class = "nx" > @token< / span > < span class = "s1" > ' HERECOMMENT' < / span > < span class = "p" > ,< / span > < span class = "nx" > @sanitizeHeredoc< / span > < span class = "nx" > here< / span > < span class = "p" > ,< / span >
2011-12-18 13:21:00 -05:00
< span class = "nv" > herecomment: < / span > < span class = "kc" > true< / span > < span class = "p" > ,< / span > < span class = "nv" > indent: < / span > < span class = "nb" > Array< / span > < span class = "p" > (< / span > < span class = "nx" > @indent< / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span > < span class = "p" > ).< / span > < span class = "nx" > join< / span > < span class = "p" > (< / span > < span class = "s1" > ' ' < / span > < span class = "p" > )< / span >
2010-07-11 10:40:44 -04:00
< span class = "nx" > @token< / span > < span class = "s1" > ' TERMINATOR' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' \n' < / span >
2011-05-01 12:08:15 -04:00
< span class = "nx" > @line< / span > < span class = "o" > +=< / span > < span class = "nx" > count< / span > < span class = "nx" > comment< / span > < span class = "p" > ,< / span > < span class = "s1" > ' \n' < / span >
2011-12-18 13:21:00 -05:00
< span class = "nx" > comment< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-13" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-13" > ¶ < / a > < / div > < p > Matches JavaScript interpolated directly into the source via backticks.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > jsToken: < / span > < span class = "o" > -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > return< / span > < span class = "mi" > 0< / span > < span class = "nx" > unless< / span > < span class = "nx" > @chunk< / span > < span class = "p" > .< / span > < span class = "nx" > charAt< / span > < span class = "p" > (< / span > < span class = "mi" > 0< / span > < span class = "p" > )< / span > < span class = "o" > is< / span > < span class = "s1" > ' `' < / span > < span class = "o" > and< / span > < span class = "nv" > match = < / span > < span class = "nx" > JSTOKEN< / span > < span class = "p" > .< / span > < span class = "nx" > exec< / span > < span class = "nx" > @chunk< / span >
< span class = "nx" > @token< / span > < span class = "s1" > ' JS' < / span > < span class = "p" > ,< / span > < span class = "p" > (< / span > < span class = "nv" > script = < / span > < span class = "nx" > match< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]).< / span > < span class = "nx" > slice< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span >
< span class = "nx" > script< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-14" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-14" > ¶ < / a > < / div > < p > Matches regular expression literals. Lexing regular expressions is difficult
2010-03-07 12:47:03 -05:00
to distinguish from division, so we borrow some basic heuristics from
2011-12-18 13:21:00 -05:00
JavaScript and Ruby.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > regexToken: < / span > < span class = "o" > -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > return< / span > < span class = "mi" > 0< / span > < span class = "k" > if< / span > < span class = "nx" > @chunk< / span > < span class = "p" > .< / span > < span class = "nx" > charAt< / span > < span class = "p" > (< / span > < span class = "mi" > 0< / span > < span class = "p" > )< / span > < span class = "o" > isnt< / span > < span class = "s1" > ' /' < / span >
2011-08-04 23:17:23 -04:00
< span class = "k" > if< / span > < span class = "nv" > match = < / span > < span class = "nx" > HEREGEX< / span > < span class = "p" > .< / span > < span class = "nx" > exec< / span > < span class = "nx" > @chunk< / span >
< span class = "nv" > length = < / span > < span class = "nx" > @heregexToken< / span > < span class = "nx" > match< / span >
< span class = "nx" > @line< / span > < span class = "o" > +=< / span > < span class = "nx" > count< / span > < span class = "nx" > match< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ],< / span > < span class = "s1" > ' \n' < / span >
< span class = "k" > return< / span > < span class = "nx" > length< / span >
2010-11-20 14:22:28 -05:00
< span class = "nv" > prev = < / span > < span class = "nx" > last< / span > < span class = "nx" > @tokens< / span >
< span class = "k" > return< / span > < span class = "mi" > 0< / span > < span class = "k" > if< / span > < span class = "nx" > prev< / span > < span class = "o" > and< / span > < span class = "p" > (< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "k" > in< / span > < span class = "p" > (< / span > < span class = "k" > if< / span > < span class = "nx" > prev< / span > < span class = "p" > .< / span > < span class = "nx" > spaced< / span > < span class = "k" > then< / span > < span class = "nx" > NOT_REGEX< / span > < span class = "k" > else< / span > < span class = "nx" > NOT_SPACED_REGEX< / span > < span class = "p" > ))< / span >
< span class = "k" > return< / span > < span class = "mi" > 0< / span > < span class = "nx" > unless< / span > < span class = "nv" > match = < / span > < span class = "nx" > REGEX< / span > < span class = "p" > .< / span > < span class = "nx" > exec< / span > < span class = "nx" > @chunk< / span >
2011-11-08 18:01:45 -05:00
< span class = "p" > [< / span > < span class = "nx" > match< / span > < span class = "p" > ,< / span > < span class = "nx" > regex< / span > < span class = "p" > ,< / span > < span class = "nx" > flags< / span > < span class = "p" > ]< / span > < span class = "o" > =< / span > < span class = "nx" > match< / span >
< span class = "k" > if< / span > < span class = "nx" > regex< / span > < span class = "p" > [..< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' /*' < / span > < span class = "k" > then< / span > < span class = "nx" > @error< / span > < span class = "s1" > ' regular expressions cannot begin with `*`' < / span >
< span class = "k" > if< / span > < span class = "nx" > regex< / span > < span class = "o" > is< / span > < span class = "s1" > ' //' < / span > < span class = "k" > then< / span > < span class = "nv" > regex = < / span > < span class = "s1" > ' /(?:)/' < / span >
< span class = "nx" > @token< / span > < span class = "s1" > ' REGEX' < / span > < span class = "p" > ,< / span > < span class = "s2" > " #{regex}#{flags}" < / span >
2011-12-18 13:21:00 -05:00
< span class = "nx" > match< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-15" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-15" > ¶ < / a > < / div > < p > Matches multiline extended regular expressions.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > heregexToken: < / span > < span class = "nf" > (match) -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "p" > [< / span > < span class = "nx" > heregex< / span > < span class = "p" > ,< / span > < span class = "nx" > body< / span > < span class = "p" > ,< / span > < span class = "nx" > flags< / span > < span class = "p" > ]< / span > < span class = "o" > =< / span > < span class = "nx" > match< / span >
< span class = "k" > if< / span > < span class = "mi" > 0< / span > < span class = "o" > > < / span > < span class = "nx" > body< / span > < span class = "p" > .< / span > < span class = "nx" > indexOf< / span > < span class = "s1" > ' #{' < / span >
< span class = "nv" > re = < / span > < span class = "nx" > body< / span > < span class = "p" > .< / span > < span class = "nx" > replace< / span > < span class = "p" > (< / span > < span class = "nx" > HEREGEX_OMIT< / span > < span class = "p" > ,< / span > < span class = "s1" > ' ' < / span > < span class = "p" > ).< / span > < span class = "nx" > replace< / span > < span class = "p" > (< / span > < span class = "sr" > /\//g< / span > < span class = "p" > ,< / span > < span class = "s1" > ' \\/' < / span > < span class = "p" > )< / span >
2011-11-08 18:01:45 -05:00
< span class = "k" > if< / span > < span class = "nx" > re< / span > < span class = "p" > .< / span > < span class = "nx" > match< / span > < span class = "sr" > /^\*/< / span > < span class = "k" > then< / span > < span class = "nx" > @error< / span > < span class = "s1" > ' regular expressions cannot begin with `*`' < / span >
2010-11-20 14:22:28 -05:00
< span class = "nx" > @token< / span > < span class = "s1" > ' REGEX' < / span > < span class = "p" > ,< / span > < span class = "s2" > " /#{ re or ' (?:)' }/#{flags}" < / span >
< span class = "k" > return< / span > < span class = "nx" > heregex< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span >
< span class = "nx" > @token< / span > < span class = "s1" > ' IDENTIFIER' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' RegExp' < / span >
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "p" > [< / span > < span class = "s1" > ' CALL_START' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' (' < / span > < span class = "p" > ]< / span >
< span class = "nv" > tokens = < / span > < span class = "p" > []< / span >
2011-12-18 13:21:00 -05:00
< span class = "k" > for< / span > < span class = "p" > [< / span > < span class = "nx" > tag< / span > < span class = "p" > ,< / span > < span class = "nx" > value< / span > < span class = "p" > ]< / span > < span class = "k" > in< / span > < span class = "nx" > @interpolateString< / span > < span class = "p" > (< / span > < span class = "nx" > body< / span > < span class = "p" > ,< / span > < span class = "nv" > regex: < / span > < span class = "kc" > yes< / span > < span class = "p" > )< / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > if< / span > < span class = "nx" > tag< / span > < span class = "o" > is< / span > < span class = "s1" > ' TOKENS' < / span >
< span class = "nx" > tokens< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "nx" > value< / span > < span class = "p" > ...< / span >
< span class = "k" > else< / span >
< span class = "k" > continue< / span > < span class = "nx" > unless< / span > < span class = "nv" > value = < / span > < span class = "nx" > value< / span > < span class = "p" > .< / span > < span class = "nx" > replace< / span > < span class = "nx" > HEREGEX_OMIT< / span > < span class = "p" > ,< / span > < span class = "s1" > ' ' < / span >
< span class = "nv" > value = < / span > < span class = "nx" > value< / span > < span class = "p" > .< / span > < span class = "nx" > replace< / span > < span class = "sr" > /\\/g< / span > < span class = "p" > ,< / span > < span class = "s1" > ' \\\\' < / span >
< span class = "nx" > tokens< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "p" > [< / span > < span class = "s1" > ' STRING' < / span > < span class = "p" > ,< / span > < span class = "nx" > @makeString< / span > < span class = "p" > (< / span > < span class = "nx" > value< / span > < span class = "p" > ,< / span > < span class = "s1" > ' " ' < / span > < span class = "p" > ,< / span > < span class = "kc" > yes< / span > < span class = "p" > )]< / span >
< span class = "nx" > tokens< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "p" > [< / span > < span class = "s1" > ' +' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' +' < / span > < span class = "p" > ]< / span >
< span class = "nx" > tokens< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span >
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "p" > [< / span > < span class = "s1" > ' STRING' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' " " ' < / span > < span class = "p" > ],< / span > < span class = "p" > [< / span > < span class = "s1" > ' +' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' +' < / span > < span class = "p" > ]< / span > < span class = "nx" > unless< / span > < span class = "nx" > tokens< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > ?< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' STRING' < / span >
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "nx" > tokens< / span > < span class = "p" > ...< / span >
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "p" > [< / span > < span class = "s1" > ' ,' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ,' < / span > < span class = "p" > ],< / span > < span class = "p" > [< / span > < span class = "s1" > ' STRING' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' " ' < / span > < span class = "o" > +< / span > < span class = "nx" > flags< / span > < span class = "o" > +< / span > < span class = "s1" > ' " ' < / span > < span class = "p" > ]< / span > < span class = "k" > if< / span > < span class = "nx" > flags< / span >
< span class = "nx" > @token< / span > < span class = "s1" > ' )' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' )' < / span >
< span class = "nx" > heregex< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-16" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-16" > ¶ < / a > < / div > < p > Matches newlines, indents, and outdents, and determines which is which.
2010-03-07 12:47:03 -05:00
If we can detect that the current line is continued onto the the next line,
then the newline is suppressed:< / p >
< pre > < code > elements
.each( ... )
.map( ... )
< / code > < / pre >
< p > Keeps track of the level of indentation, because a single outdent token
2011-12-18 13:21:00 -05:00
can close multiple indents, so we need to know how far in we happen to be.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > lineToken: < / span > < span class = "o" > -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > return< / span > < span class = "mi" > 0< / span > < span class = "nx" > unless< / span > < span class = "nv" > match = < / span > < span class = "nx" > MULTI_DENT< / span > < span class = "p" > .< / span > < span class = "nx" > exec< / span > < span class = "nx" > @chunk< / span >
< span class = "nv" > indent = < / span > < span class = "nx" > match< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
< span class = "nx" > @line< / span > < span class = "o" > +=< / span > < span class = "nx" > count< / span > < span class = "nx" > indent< / span > < span class = "p" > ,< / span > < span class = "s1" > ' \n' < / span >
2011-11-08 18:01:45 -05:00
< span class = "vi" > @seenFor = < / span > < span class = "kc" > no< / span >
2010-11-20 14:22:28 -05:00
< span class = "nv" > prev = < / span > < span class = "nx" > last< / span > < span class = "nx" > @tokens< / span > < span class = "p" > ,< / span > < span class = "mi" > 1< / span >
< span class = "nv" > size = < / span > < span class = "nx" > indent< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "o" > -< / span > < span class = "nx" > indent< / span > < span class = "p" > .< / span > < span class = "nx" > lastIndexOf< / span > < span class = "s1" > ' \n' < / span >
< span class = "nv" > noNewlines = < / span > < span class = "nx" > @unfinished< / span > < span class = "p" > ()< / span >
2010-09-16 00:32:57 -04:00
< span class = "k" > if< / span > < span class = "nx" > size< / span > < span class = "o" > -< / span > < span class = "nx" > @indebt< / span > < span class = "o" > is< / span > < span class = "nx" > @indent< / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > if< / span > < span class = "nx" > noNewlines< / span > < span class = "k" > then< / span > < span class = "nx" > @suppressNewlines< / span > < span class = "p" > ()< / span > < span class = "k" > else< / span > < span class = "nx" > @newlineToken< / span > < span class = "p" > ()< / span >
< span class = "k" > return< / span > < span class = "nx" > indent< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span >
< span class = "k" > if< / span > < span class = "nx" > size< / span > < span class = "o" > > < / span > < span class = "nx" > @indent< / span >
2010-09-16 00:32:57 -04:00
< span class = "k" > if< / span > < span class = "nx" > noNewlines< / span >
< span class = "vi" > @indebt = < / span > < span class = "nx" > size< / span > < span class = "o" > -< / span > < span class = "nx" > @indent< / span >
2010-11-20 14:22:28 -05:00
< span class = "nx" > @suppressNewlines< / span > < span class = "p" > ()< / span >
< span class = "k" > return< / span > < span class = "nx" > indent< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span >
2010-08-23 22:08:33 -04:00
< span class = "nv" > diff = < / span > < span class = "nx" > size< / span > < span class = "o" > -< / span > < span class = "nx" > @indent< / span > < span class = "o" > +< / span > < span class = "nx" > @outdebt< / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @token< / span > < span class = "s1" > ' INDENT' < / span > < span class = "p" > ,< / span > < span class = "nx" > diff< / span >
< span class = "nx" > @indents< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "nx" > diff< / span >
2011-12-18 13:21:00 -05:00
< span class = "nx" > @ends< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "s1" > ' OUTDENT' < / span >
2010-09-16 00:32:57 -04:00
< span class = "vi" > @outdebt = @indebt = < / span > < span class = "mi" > 0< / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > else< / span >
2010-09-16 00:32:57 -04:00
< span class = "vi" > @indebt = < / span > < span class = "mi" > 0< / span >
2010-06-28 00:19:58 -04:00
< span class = "nx" > @outdentToken< / span > < span class = "nx" > @indent< / span > < span class = "o" > -< / span > < span class = "nx" > size< / span > < span class = "p" > ,< / span > < span class = "nx" > noNewlines< / span >
2010-08-04 23:14:34 -04:00
< span class = "vi" > @indent = < / span > < span class = "nx" > size< / span >
2010-11-20 14:22:28 -05:00
< span class = "nx" > indent< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-17" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-17" > ¶ < / a > < / div > < p > Record an outdent token or multiple tokens, if we happen to be moving back
2011-12-18 13:21:00 -05:00
inwards past several recorded indents.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > outdentToken: < / span > < span class = "nf" > (moveOut, noNewlines) -> < / span >
2010-08-03 00:06:34 -04:00
< span class = "k" > while< / span > < span class = "nx" > moveOut< / span > < span class = "o" > > < / span > < span class = "mi" > 0< / span >
2010-08-04 23:14:34 -04:00
< span class = "nv" > len = < / span > < span class = "nx" > @indents< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span >
2010-08-03 00:06:34 -04:00
< span class = "k" > if< / span > < span class = "nx" > @indents< / span > < span class = "p" > [< / span > < span class = "nx" > len< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "kc" > undefined< / span >
2010-08-04 23:14:34 -04:00
< span class = "nv" > moveOut = < / span > < span class = "mi" > 0< / span >
2010-08-03 00:06:34 -04:00
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > @indents< / span > < span class = "p" > [< / span > < span class = "nx" > len< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "nx" > @outdebt< / span >
< span class = "nx" > moveOut< / span > < span class = "o" > -=< / span > < span class = "nx" > @outdebt< / span >
2010-08-04 23:14:34 -04:00
< span class = "vi" > @outdebt = < / span > < span class = "mi" > 0< / span >
2010-08-03 00:06:34 -04:00
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > @indents< / span > < span class = "p" > [< / span > < span class = "nx" > len< / span > < span class = "p" > ]< / span > < span class = "o" > < < / span > < span class = "nx" > @outdebt< / span >
< span class = "nx" > @outdebt< / span > < span class = "o" > -=< / span > < span class = "nx" > @indents< / span > < span class = "p" > [< / span > < span class = "nx" > len< / span > < span class = "p" > ]< / span >
2010-11-20 14:22:28 -05:00
< span class = "nx" > moveOut< / span > < span class = "o" > -=< / span > < span class = "nx" > @indents< / span > < span class = "p" > [< / span > < span class = "nx" > len< / span > < span class = "p" > ]< / span >
2010-08-03 00:06:34 -04:00
< span class = "k" > else< / span >
2010-11-20 14:22:28 -05:00
< span class = "nv" > dent = < / span > < span class = "nx" > @indents< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span > < span class = "o" > -< / span > < span class = "nx" > @outdebt< / span >
2010-08-03 00:06:34 -04:00
< span class = "nx" > moveOut< / span > < span class = "o" > -=< / span > < span class = "nx" > dent< / span >
2010-08-04 23:14:34 -04:00
< span class = "vi" > @outdebt = < / span > < span class = "mi" > 0< / span >
2011-11-08 18:01:45 -05:00
< span class = "nx" > @pair< / span > < span class = "s1" > ' OUTDENT' < / span >
2010-08-03 00:06:34 -04:00
< span class = "nx" > @token< / span > < span class = "s1" > ' OUTDENT' < / span > < span class = "p" > ,< / span > < span class = "nx" > dent< / span >
< span class = "nx" > @outdebt< / span > < span class = "o" > -=< / span > < span class = "nx" > moveOut< / span > < span class = "k" > if< / span > < span class = "nx" > dent< / span >
2011-11-08 18:01:45 -05:00
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span > < span class = "k" > while< / span > < span class = "nx" > @value< / span > < span class = "p" > ()< / span > < span class = "o" > is< / span > < span class = "s1" > ' ;' < / span >
2010-11-20 14:22:28 -05:00
< span class = "nx" > @token< / span > < span class = "s1" > ' TERMINATOR' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' \n' < / span > < span class = "nx" > unless< / span > < span class = "nx" > @tag< / span > < span class = "p" > ()< / span > < span class = "o" > is< / span > < span class = "s1" > ' TERMINATOR' < / span > < span class = "o" > or< / span > < span class = "nx" > noNewlines< / span >
< span class = "k" > this< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-18" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-18" > ¶ < / a > < / div > < p > Matches and consumes non-meaningful whitespace. Tag the previous token
2011-12-18 13:21:00 -05:00
as being "spaced", because there are some cases where it makes a difference.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > whitespaceToken: < / span > < span class = "o" > -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > return< / span > < span class = "mi" > 0< / span > < span class = "nx" > unless< / span > < span class = "p" > (< / span > < span class = "nv" > match = < / span > < span class = "nx" > WHITESPACE< / span > < span class = "p" > .< / span > < span class = "nx" > exec< / span > < span class = "nx" > @chunk< / span > < span class = "p" > )< / span > < span class = "o" > or< / span >
< span class = "p" > (< / span > < span class = "nv" > nline = < / span > < span class = "nx" > @chunk< / span > < span class = "p" > .< / span > < span class = "nx" > charAt< / span > < span class = "p" > (< / span > < span class = "mi" > 0< / span > < span class = "p" > )< / span > < span class = "o" > is< / span > < span class = "s1" > ' \n' < / span > < span class = "p" > )< / span >
< span class = "nv" > prev = < / span > < span class = "nx" > last< / span > < span class = "nx" > @tokens< / span >
< span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "k" > if< / span > < span class = "nx" > match< / span > < span class = "k" > then< / span > < span class = "s1" > ' spaced' < / span > < span class = "k" > else< / span > < span class = "s1" > ' newLine' < / span > < span class = "p" > ]< / span > < span class = "o" > =< / span > < span class = "kc" > true< / span > < span class = "k" > if< / span > < span class = "nx" > prev< / span >
2011-12-18 13:21:00 -05:00
< span class = "k" > if< / span > < span class = "nx" > match< / span > < span class = "k" > then< / span > < span class = "nx" > match< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ].< / span > < span class = "nx" > length< / span > < span class = "k" > else< / span > < span class = "mi" > 0< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-19" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-19" > ¶ < / a > < / div > < p > Generate a newline token. Consecutive newlines get merged together.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > newlineToken: < / span > < span class = "o" > -> < / span >
2011-11-08 18:01:45 -05:00
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span > < span class = "k" > while< / span > < span class = "nx" > @value< / span > < span class = "p" > ()< / span > < span class = "o" > is< / span > < span class = "s1" > ' ;' < / span >
2010-11-20 14:22:28 -05:00
< span class = "nx" > @token< / span > < span class = "s1" > ' TERMINATOR' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' \n' < / span > < span class = "nx" > unless< / span > < span class = "nx" > @tag< / span > < span class = "p" > ()< / span > < span class = "o" > is< / span > < span class = "s1" > ' TERMINATOR' < / span >
< span class = "k" > this< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-20" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-20" > ¶ < / a > < / div > < p > Use a < code > \< / code > at a line-ending to suppress the newline.
2011-12-18 13:21:00 -05:00
The slash is removed here once its job is done.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > suppressNewlines: < / span > < span class = "o" > -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span > < span class = "k" > if< / span > < span class = "nx" > @value< / span > < span class = "p" > ()< / span > < span class = "o" > is< / span > < span class = "s1" > ' \\' < / span >
< span class = "k" > this< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-21" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-21" > ¶ < / a > < / div > < p > We treat all other single characters as a token. E.g.: < code > ( ) , . !< / code >
2010-03-06 23:48:06 -05:00
Multi-character operators are also literal tokens, so that Jison can assign
2010-03-07 12:47:03 -05:00
the proper order of operations. There are some symbols that we tag specially
here. < code > ;< / code > and newlines are both treated as a < code > TERMINATOR< / code > , we distinguish
2011-12-18 13:21:00 -05:00
parentheses that indicate a method call from regular parentheses, and so on.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > literalToken: < / span > < span class = "o" > -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > if< / span > < span class = "nv" > match = < / span > < span class = "nx" > OPERATOR< / span > < span class = "p" > .< / span > < span class = "nx" > exec< / span > < span class = "nx" > @chunk< / span >
< span class = "p" > [< / span > < span class = "nx" > value< / span > < span class = "p" > ]< / span > < span class = "o" > =< / span > < span class = "nx" > match< / span >
< span class = "nx" > @tagParameters< / span > < span class = "p" > ()< / span > < span class = "k" > if< / span > < span class = "nx" > CODE< / span > < span class = "p" > .< / span > < span class = "nx" > test< / span > < span class = "nx" > value< / span >
< span class = "k" > else< / span >
< span class = "nv" > value = < / span > < span class = "nx" > @chunk< / span > < span class = "p" > .< / span > < span class = "nx" > charAt< / span > < span class = "mi" > 0< / span >
< span class = "nv" > tag = < / span > < span class = "nx" > value< / span >
< span class = "nv" > prev = < / span > < span class = "nx" > last< / span > < span class = "nx" > @tokens< / span >
< span class = "k" > if< / span > < span class = "nx" > value< / span > < span class = "o" > is< / span > < span class = "s1" > ' =' < / span > < span class = "o" > and< / span > < span class = "nx" > prev< / span >
2011-11-08 18:01:45 -05:00
< span class = "k" > if< / span > < span class = "o" > not< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "p" > ].< / span > < span class = "nx" > reserved< / span > < span class = "o" > and< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "k" > in< / span > < span class = "nx" > JS_FORBIDDEN< / span >
< span class = "nx" > @error< / span > < span class = "s2" > " reserved word \" #{@value()}\" can' t be assigned" < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > if< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "k" > in< / span > < span class = "p" > [< / span > < span class = "s1" > ' ||' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' & & ' < / span > < span class = "p" > ]< / span >
< span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > =< / span > < span class = "s1" > ' COMPOUND_ASSIGN' < / span >
< span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "o" > +=< / span > < span class = "s1" > ' =' < / span >
< span class = "k" > return< / span > < span class = "nx" > value< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span >
2011-11-08 18:01:45 -05:00
< span class = "k" > if< / span > < span class = "nx" > value< / span > < span class = "o" > is< / span > < span class = "s1" > ' ;' < / span >
< span class = "vi" > @seenFor = < / span > < span class = "kc" > no< / span >
< span class = "nv" > tag = < / span > < span class = "s1" > ' TERMINATOR' < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > value< / span > < span class = "k" > in< / span > < span class = "nx" > MATH< / span > < span class = "k" > then< / span > < span class = "nv" > tag = < / span > < span class = "s1" > ' MATH' < / span >
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > value< / span > < span class = "k" > in< / span > < span class = "nx" > COMPARE< / span > < span class = "k" > then< / span > < span class = "nv" > tag = < / span > < span class = "s1" > ' COMPARE' < / span >
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > value< / span > < span class = "k" > in< / span > < span class = "nx" > COMPOUND_ASSIGN< / span > < span class = "k" > then< / span > < span class = "nv" > tag = < / span > < span class = "s1" > ' COMPOUND_ASSIGN' < / span >
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > value< / span > < span class = "k" > in< / span > < span class = "nx" > UNARY< / span > < span class = "k" > then< / span > < span class = "nv" > tag = < / span > < span class = "s1" > ' UNARY' < / span >
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > value< / span > < span class = "k" > in< / span > < span class = "nx" > SHIFT< / span > < span class = "k" > then< / span > < span class = "nv" > tag = < / span > < span class = "s1" > ' SHIFT' < / span >
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > value< / span > < span class = "k" > in< / span > < span class = "nx" > LOGIC< / span > < span class = "o" > or< / span > < span class = "nx" > value< / span > < span class = "o" > is< / span > < span class = "s1" > ' ?' < / span > < span class = "o" > and< / span > < span class = "nx" > prev< / span > < span class = "o" > ?< / span > < span class = "p" > .< / span > < span class = "nx" > spaced< / span > < span class = "k" > then< / span > < span class = "nv" > tag = < / span > < span class = "s1" > ' LOGIC' < / span >
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > prev< / span > < span class = "o" > and< / span > < span class = "o" > not< / span > < span class = "nx" > prev< / span > < span class = "p" > .< / span > < span class = "nx" > spaced< / span >
< span class = "k" > if< / span > < span class = "nx" > value< / span > < span class = "o" > is< / span > < span class = "s1" > ' (' < / span > < span class = "o" > and< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "k" > in< / span > < span class = "nx" > CALLABLE< / span >
2010-09-16 00:32:57 -04:00
< span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > =< / span > < span class = "s1" > ' FUNC_EXIST' < / span > < span class = "k" > if< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' ?' < / span >
2010-08-04 23:14:34 -04:00
< span class = "nv" > tag = < / span > < span class = "s1" > ' CALL_START' < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > value< / span > < span class = "o" > is< / span > < span class = "s1" > ' [' < / span > < span class = "o" > and< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "k" > in< / span > < span class = "nx" > INDEXABLE< / span >
2010-08-04 23:14:34 -04:00
< span class = "nv" > tag = < / span > < span class = "s1" > ' INDEX_START' < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > switch< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
< span class = "k" > when< / span > < span class = "s1" > ' ?' < / span > < span class = "k" > then< / span > < span class = "nx" > prev< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > =< / span > < span class = "s1" > ' INDEX_SOAK' < / span >
2011-11-08 18:01:45 -05:00
< span class = "k" > switch< / span > < span class = "nx" > value< / span >
< span class = "k" > when< / span > < span class = "s1" > ' (' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' {' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' [' < / span > < span class = "k" > then< / span > < span class = "nx" > @ends< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "nx" > INVERSES< / span > < span class = "p" > [< / span > < span class = "nx" > value< / span > < span class = "p" > ]< / span >
< span class = "k" > when< / span > < span class = "s1" > ' )' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' }' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ]' < / span > < span class = "k" > then< / span > < span class = "nx" > @pair< / span > < span class = "nx" > value< / span >
2010-04-03 12:01:46 -04:00
< span class = "nx" > @token< / span > < span class = "nx" > tag< / span > < span class = "p" > ,< / span > < span class = "nx" > value< / span >
2010-11-20 14:22:28 -05:00
< span class = "nx" > value< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-22" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-22" > ¶ < / a > < / div > < h2 > Token Manipulators< / h2 > < / td > < td class = "code" > < div class = "highlight" > < pre > < / pre > < / div > < / td > < / tr > < tr id = "section-23" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-23" > ¶ < / a > < / div > < p > Sanitize a heredoc or herecomment by
2011-12-18 13:21:00 -05:00
erasing all external indentation on the left-hand side.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > sanitizeHeredoc: < / span > < span class = "nf" > (doc, options) -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "p" > {< / span > < span class = "nx" > indent< / span > < span class = "p" > ,< / span > < span class = "nx" > herecomment< / span > < span class = "p" > }< / span > < span class = "o" > =< / span > < span class = "nx" > options< / span >
2011-05-01 12:08:15 -04:00
< span class = "k" > if< / span > < span class = "nx" > herecomment< / span >
< span class = "k" > if< / span > < span class = "nx" > HEREDOC_ILLEGAL< / span > < span class = "p" > .< / span > < span class = "nx" > test< / span > < span class = "nx" > doc< / span >
2011-11-08 18:01:45 -05:00
< span class = "nx" > @error< / span > < span class = "s2" > " block comment cannot contain \" */\" , starting" < / span >
2011-05-01 12:08:15 -04:00
< span class = "k" > return< / span > < span class = "nx" > doc< / span > < span class = "k" > if< / span > < span class = "nx" > doc< / span > < span class = "p" > .< / span > < span class = "nx" > indexOf< / span > < span class = "p" > (< / span > < span class = "s1" > ' \n' < / span > < span class = "p" > )< / span > < span class = "o" > < =< / span > < span class = "mi" > 0< / span >
< span class = "k" > else< / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > while< / span > < span class = "nv" > match = < / span > < span class = "nx" > HEREDOC_INDENT< / span > < span class = "p" > .< / span > < span class = "nx" > exec< / span > < span class = "nx" > doc< / span >
< span class = "nv" > attempt = < / span > < span class = "nx" > match< / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span >
< span class = "nv" > indent = < / span > < span class = "nx" > attempt< / span > < span class = "k" > if< / span > < span class = "nx" > indent< / span > < span class = "o" > is< / span > < span class = "kc" > null< / span > < span class = "o" > or< / span > < span class = "mi" > 0< / span > < span class = "o" > < < / span > < span class = "nx" > attempt< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > < < / span > < span class = "nx" > indent< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span >
< span class = "nv" > doc = < / span > < span class = "nx" > doc< / span > < span class = "p" > .< / span > < span class = "nx" > replace< / span > < span class = "err" > /// \n #{indent} ///g, ' \n' if indent< / span >
< span class = "nv" > doc = < / span > < span class = "nx" > doc< / span > < span class = "p" > .< / span > < span class = "nx" > replace< / span > < span class = "sr" > /^\n/< / span > < span class = "p" > ,< / span > < span class = "s1" > ' ' < / span > < span class = "nx" > unless< / span > < span class = "nx" > herecomment< / span >
< span class = "nx" > doc< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-24" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-24" > ¶ < / a > < / div > < p > A source of ambiguity in our grammar used to be parameter lists in function
2010-03-07 12:47:03 -05:00
definitions versus argument lists in function calls. Walk backwards, tagging
2011-12-18 13:21:00 -05:00
parameters specially in order to make things easier for the parser.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > tagParameters: < / span > < span class = "o" > -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > return< / span > < span class = "k" > this< / span > < span class = "k" > if< / span > < span class = "nx" > @tag< / span > < span class = "p" > ()< / span > < span class = "o" > isnt< / span > < span class = "s1" > ' )' < / span >
< span class = "nv" > stack = < / span > < span class = "p" > []< / span >
< span class = "p" > {< / span > < span class = "nx" > tokens< / span > < span class = "p" > }< / span > < span class = "o" > =< / span > < span class = "k" > this< / span >
< span class = "nv" > i = < / span > < span class = "nx" > tokens< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span >
< span class = "nx" > tokens< / span > < span class = "p" > [< / span > < span class = "o" > --< / span > < span class = "nx" > i< / span > < span class = "p" > ][< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > =< / span > < span class = "s1" > ' PARAM_END' < / span >
< span class = "k" > while< / span > < span class = "nv" > tok = < / span > < span class = "nx" > tokens< / span > < span class = "p" > [< / span > < span class = "o" > --< / span > < span class = "nx" > i< / span > < span class = "p" > ]< / span >
2010-03-06 23:48:06 -05:00
< span class = "k" > switch< / span > < span class = "nx" > tok< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > when< / span > < span class = "s1" > ' )' < / span >
< span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "nx" > tok< / span >
< span class = "k" > when< / span > < span class = "s1" > ' (' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' CALL_START' < / span >
< span class = "k" > if< / span > < span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "k" > then< / span > < span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span >
2011-05-01 12:08:15 -04:00
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > tok< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' (' < / span >
2010-11-20 14:22:28 -05:00
< span class = "nx" > tok< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > =< / span > < span class = "s1" > ' PARAM_START' < / span >
< span class = "k" > return< / span > < span class = "k" > this< / span >
2011-08-04 23:17:23 -04:00
< span class = "k" > else< / span > < span class = "k" > return< / span > < span class = "k" > this< / span >
2011-12-18 13:21:00 -05:00
< span class = "k" > this< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-25" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-25" > ¶ < / a > < / div > < p > Close up all remaining open blocks at the end of the file.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > closeIndentation: < / span > < span class = "o" > -> < / span >
2011-11-08 18:01:45 -05:00
< span class = "nx" > @outdentToken< / span > < span class = "nx" > @indent< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-26" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-26" > ¶ < / a > < / div > < p > Matches a balanced group such as a single or double-quoted string. Pass in
2010-06-28 00:19:58 -04:00
a series of delimiters, all of which must be nested correctly within the
contents of the string. This method allows us to have strings within
2011-12-18 13:21:00 -05:00
interpolations within strings, ad infinitum.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > balancedString: < / span > < span class = "nf" > (str, end) -> < / span >
2010-12-24 11:02:10 -08:00
< span class = "nv" > stack = < / span > < span class = "p" > [< / span > < span class = "nx" > end< / span > < span class = "p" > ]< / span >
2010-11-21 12:38:27 -05:00
< span class = "k" > for< / span > < span class = "nx" > i< / span > < span class = "k" > in< / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "p" > ...< / span > < span class = "nx" > str< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "p" > ]< / span >
2010-12-24 11:02:10 -08:00
< span class = "k" > switch< / span > < span class = "nv" > letter = < / span > < span class = "nx" > str< / span > < span class = "p" > .< / span > < span class = "nx" > charAt< / span > < span class = "nx" > i< / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > when< / span > < span class = "s1" > ' \\' < / span >
< span class = "nx" > i< / span > < span class = "o" > ++< / span >
< span class = "k" > continue< / span >
2010-12-24 11:02:10 -08:00
< span class = "k" > when< / span > < span class = "nx" > end< / span >
2010-11-20 14:22:28 -05:00
< span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span >
2010-12-24 11:02:10 -08:00
< span class = "nx" > unless< / span > < span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span >
< span class = "k" > return< / span > < span class = "nx" > str< / span > < span class = "p" > .< / span > < span class = "nx" > slice< / span > < span class = "mi" > 0< / span > < span class = "p" > ,< / span > < span class = "nx" > i< / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span >
< span class = "nv" > end = < / span > < span class = "nx" > stack< / span > < span class = "p" > [< / span > < span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > continue< / span >
2010-12-24 11:02:10 -08:00
< span class = "k" > if< / span > < span class = "nx" > end< / span > < span class = "o" > is< / span > < span class = "s1" > ' }' < / span > < span class = "o" > and< / span > < span class = "nx" > letter< / span > < span class = "k" > in< / span > < span class = "p" > [< / span > < span class = "s1" > ' " ' < / span > < span class = "p" > ,< / span > < span class = "s2" > " ' " < / span > < span class = "p" > ]< / span >
< span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "nv" > end = < / span > < span class = "nx" > letter< / span >
2011-08-04 23:17:23 -04:00
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > end< / span > < span class = "o" > is< / span > < span class = "s1" > ' }' < / span > < span class = "o" > and< / span > < span class = "nx" > letter< / span > < span class = "o" > is< / span > < span class = "s1" > ' /' < / span > < span class = "o" > and< / span > < span class = "nv" > match = < / span > < span class = "p" > (< / span > < span class = "nx" > HEREGEX< / span > < span class = "p" > .< / span > < span class = "nx" > exec< / span > < span class = "p" > (< / span > < span class = "nx" > str< / span > < span class = "p" > .< / span > < span class = "nx" > slice< / span > < span class = "nx" > i< / span > < span class = "p" > )< / span > < span class = "o" > or< / span > < span class = "nx" > REGEX< / span > < span class = "p" > .< / span > < span class = "nx" > exec< / span > < span class = "p" > (< / span > < span class = "nx" > str< / span > < span class = "p" > .< / span > < span class = "nx" > slice< / span > < span class = "nx" > i< / span > < span class = "p" > ))< / span >
< span class = "nx" > i< / span > < span class = "o" > +=< / span > < span class = "nx" > match< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ].< / span > < span class = "nx" > length< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span >
2010-12-24 11:02:10 -08:00
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > end< / span > < span class = "o" > is< / span > < span class = "s1" > ' }' < / span > < span class = "o" > and< / span > < span class = "nx" > letter< / span > < span class = "o" > is< / span > < span class = "s1" > ' {' < / span >
< span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "nv" > end = < / span > < span class = "s1" > ' }' < / span >
< span class = "k" > else< / span > < span class = "k" > if< / span > < span class = "nx" > end< / span > < span class = "o" > is< / span > < span class = "s1" > ' " ' < / span > < span class = "o" > and< / span > < span class = "nx" > prev< / span > < span class = "o" > is< / span > < span class = "s1" > ' #' < / span > < span class = "o" > and< / span > < span class = "nx" > letter< / span > < span class = "o" > is< / span > < span class = "s1" > ' {' < / span >
< span class = "nx" > stack< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "nv" > end = < / span > < span class = "s1" > ' }' < / span >
< span class = "nv" > prev = < / span > < span class = "nx" > letter< / span >
2011-11-08 18:01:45 -05:00
< span class = "nx" > @error< / span > < span class = "s2" > " missing #{ stack.pop() }, starting" < / span > < / pre > < / div > < / td > < / tr > < tr id = "section-27" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-27" > ¶ < / a > < / div > < p > Expand variables and expressions inside double-quoted strings using
2010-11-20 14:22:28 -05:00
Ruby-like notation for substitution of arbitrary expressions.< / p >
2010-03-06 23:48:06 -05:00
2010-08-11 00:40:15 -04:00
< pre > < code > "Hello #{name.capitalize()}."
2010-03-07 12:47:03 -05:00
< / code > < / pre >
< p > If it encounters an interpolation, this method will recursively create a
new Lexer, tokenize the interpolated contents, and merge them into the
2011-12-18 13:21:00 -05:00
token stream.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > interpolateString: < / span > < span class = "nf" > (str, options = {}) -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "p" > {< / span > < span class = "nx" > heredoc< / span > < span class = "p" > ,< / span > < span class = "nx" > regex< / span > < span class = "p" > }< / span > < span class = "o" > =< / span > < span class = "nx" > options< / span >
< span class = "nv" > tokens = < / span > < span class = "p" > []< / span >
< span class = "nv" > pi = < / span > < span class = "mi" > 0< / span >
< span class = "nv" > i = < / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span >
< span class = "k" > while< / span > < span class = "nv" > letter = < / span > < span class = "nx" > str< / span > < span class = "p" > .< / span > < span class = "nx" > charAt< / span > < span class = "nx" > i< / span > < span class = "o" > +=< / span > < span class = "mi" > 1< / span >
< span class = "k" > if< / span > < span class = "nx" > letter< / span > < span class = "o" > is< / span > < span class = "s1" > ' \\' < / span >
2010-08-03 00:06:34 -04:00
< span class = "nx" > i< / span > < span class = "o" > +=< / span > < span class = "mi" > 1< / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > continue< / span >
< span class = "nx" > unless< / span > < span class = "nx" > letter< / span > < span class = "o" > is< / span > < span class = "s1" > ' #' < / span > < span class = "o" > and< / span > < span class = "nx" > str< / span > < span class = "p" > .< / span > < span class = "nx" > charAt< / span > < span class = "p" > (< / span > < span class = "nx" > i< / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span > < span class = "p" > )< / span > < span class = "o" > is< / span > < span class = "s1" > ' {' < / span > < span class = "o" > and< / span >
2010-12-24 11:02:10 -08:00
< span class = "p" > (< / span > < span class = "nv" > expr = < / span > < span class = "nx" > @balancedString< / span > < span class = "nx" > str< / span > < span class = "p" > .< / span > < span class = "nx" > slice< / span > < span class = "p" > (< / span > < span class = "nx" > i< / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span > < span class = "p" > ),< / span > < span class = "s1" > ' }' < / span > < span class = "p" > )< / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > continue< / span >
< span class = "nx" > tokens< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "p" > [< / span > < span class = "s1" > ' NEOSTRING' < / span > < span class = "p" > ,< / span > < span class = "nx" > str< / span > < span class = "p" > .< / span > < span class = "nx" > slice< / span > < span class = "p" > (< / span > < span class = "nx" > pi< / span > < span class = "p" > ,< / span > < span class = "nx" > i< / span > < span class = "p" > )]< / span > < span class = "k" > if< / span > < span class = "nx" > pi< / span > < span class = "o" > < < / span > < span class = "nx" > i< / span >
2010-11-21 12:38:27 -05:00
< span class = "nv" > inner = < / span > < span class = "nx" > expr< / span > < span class = "p" > .< / span > < span class = "nx" > slice< / span > < span class = "p" > (< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "o" > -< / span > < span class = "mi" > 1< / span > < span class = "p" > )< / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > if< / span > < span class = "nx" > inner< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span >
2011-12-18 13:21:00 -05:00
< span class = "nv" > nested = < / span > < span class = "k" > new< / span > < span class = "nx" > Lexer< / span > < span class = "p" > ().< / span > < span class = "nx" > tokenize< / span > < span class = "nx" > inner< / span > < span class = "p" > ,< / span > < span class = "nv" > line: < / span > < span class = "nx" > @line< / span > < span class = "p" > ,< / span > < span class = "nv" > rewrite: < / span > < span class = "kc" > off< / span >
2010-11-20 14:22:28 -05:00
< span class = "nx" > nested< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span >
2010-11-21 12:38:27 -05:00
< span class = "nx" > nested< / span > < span class = "p" > .< / span > < span class = "nx" > shift< / span > < span class = "p" > ()< / span > < span class = "k" > if< / span > < span class = "nx" > nested< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > ?< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' TERMINATOR' < / span >
2011-01-31 22:39:12 -05:00
< span class = "k" > if< / span > < span class = "nv" > len = < / span > < span class = "nx" > nested< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span >
< span class = "k" > if< / span > < span class = "nx" > len< / span > < span class = "o" > > < / span > < span class = "mi" > 1< / span >
2011-12-18 13:21:00 -05:00
< span class = "nx" > nested< / span > < span class = "p" > .< / span > < span class = "nx" > unshift< / span > < span class = "p" > [< / span > < span class = "s1" > ' (' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' (' < / span > < span class = "p" > ,< / span > < span class = "nx" > @line< / span > < span class = "p" > ]< / span >
< span class = "nx" > nested< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "p" > [< / span > < span class = "s1" > ' )' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' )' < / span > < span class = "p" > ,< / span > < span class = "nx" > @line< / span > < span class = "p" > ]< / span >
2011-01-31 22:39:12 -05:00
< span class = "nx" > tokens< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "p" > [< / span > < span class = "s1" > ' TOKENS' < / span > < span class = "p" > ,< / span > < span class = "nx" > nested< / span > < span class = "p" > ]< / span >
2010-11-20 14:22:28 -05:00
< span class = "nx" > i< / span > < span class = "o" > +=< / span > < span class = "nx" > expr< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span >
< span class = "nv" > pi = < / span > < span class = "nx" > i< / span > < span class = "o" > +< / span > < span class = "mi" > 1< / span >
< span class = "nx" > tokens< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "p" > [< / span > < span class = "s1" > ' NEOSTRING' < / span > < span class = "p" > ,< / span > < span class = "nx" > str< / span > < span class = "p" > .< / span > < span class = "nx" > slice< / span > < span class = "nx" > pi< / span > < span class = "p" > ]< / span > < span class = "k" > if< / span > < span class = "nx" > i< / span > < span class = "o" > > < / span > < span class = "nx" > pi< / span > < span class = "o" > < < / span > < span class = "nx" > str< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span >
< span class = "k" > return< / span > < span class = "nx" > tokens< / span > < span class = "k" > if< / span > < span class = "nx" > regex< / span >
< span class = "k" > return< / span > < span class = "nx" > @token< / span > < span class = "s1" > ' STRING' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' " " ' < / span > < span class = "nx" > unless< / span > < span class = "nx" > tokens< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span >
< span class = "nx" > tokens< / span > < span class = "p" > .< / span > < span class = "nx" > unshift< / span > < span class = "p" > [< / span > < span class = "s1" > ' ' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ' < / span > < span class = "p" > ]< / span > < span class = "nx" > unless< / span > < span class = "nx" > tokens< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ][< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > is< / span > < span class = "s1" > ' NEOSTRING' < / span >
< span class = "nx" > @token< / span > < span class = "s1" > ' (' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' (' < / span > < span class = "k" > if< / span > < span class = "nv" > interpolated = < / span > < span class = "nx" > tokens< / span > < span class = "p" > .< / span > < span class = "nx" > length< / span > < span class = "o" > > < / span > < span class = "mi" > 1< / span >
< span class = "k" > for< / span > < span class = "p" > [< / span > < span class = "nx" > tag< / span > < span class = "p" > ,< / span > < span class = "nx" > value< / span > < span class = "p" > ],< / span > < span class = "nx" > i< / span > < span class = "k" > in< / span > < span class = "nx" > tokens< / span >
< span class = "nx" > @token< / span > < span class = "s1" > ' +' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' +' < / span > < span class = "k" > if< / span > < span class = "nx" > i< / span >
< span class = "k" > if< / span > < span class = "nx" > tag< / span > < span class = "o" > is< / span > < span class = "s1" > ' TOKENS' < / span >
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "nx" > value< / span > < span class = "p" > ...< / span >
< span class = "k" > else< / span >
< span class = "nx" > @token< / span > < span class = "s1" > ' STRING' < / span > < span class = "p" > ,< / span > < span class = "nx" > @makeString< / span > < span class = "nx" > value< / span > < span class = "p" > ,< / span > < span class = "s1" > ' " ' < / span > < span class = "p" > ,< / span > < span class = "nx" > heredoc< / span >
< span class = "nx" > @token< / span > < span class = "s1" > ' )' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' )' < / span > < span class = "k" > if< / span > < span class = "nx" > interpolated< / span >
2011-11-08 18:01:45 -05:00
< span class = "nx" > tokens< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-28" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-28" > ¶ < / a > < / div > < p > Pairs up a closing token, ensuring that all listed pairs of tokens are
2011-12-18 13:21:00 -05:00
correctly balanced throughout the course of the token stream.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > pair: < / span > < span class = "nf" > (tag) -> < / span >
2011-11-08 18:01:45 -05:00
< span class = "nx" > unless< / span > < span class = "nx" > tag< / span > < span class = "o" > is< / span > < span class = "nv" > wanted = < / span > < span class = "nx" > last< / span > < span class = "nx" > @ends< / span >
< span class = "nx" > @error< / span > < span class = "s2" > " unmatched #{tag}" < / span > < span class = "nx" > unless< / span > < span class = "s1" > ' OUTDENT' < / span > < span class = "o" > is< / span > < span class = "nx" > wanted< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-29" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-29" > ¶ < / a > < / div > < p > Auto-close INDENT to support syntax like this:< / p >
< pre > < code > el.click((event) ->
el.hide())
< / code > < / pre > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nx" > @indent< / span > < span class = "o" > -=< / span > < span class = "nv" > size = < / span > < span class = "nx" > last< / span > < span class = "nx" > @indents< / span >
< span class = "nx" > @outdentToken< / span > < span class = "nx" > size< / span > < span class = "p" > ,< / span > < span class = "kc" > true< / span >
< span class = "k" > return< / span > < span class = "nx" > @pair< / span > < span class = "nx" > tag< / span >
2011-12-18 13:21:00 -05:00
< span class = "nx" > @ends< / span > < span class = "p" > .< / span > < span class = "nx" > pop< / span > < span class = "p" > ()< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-30" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-30" > ¶ < / a > < / div > < h2 > Helpers< / h2 > < / td > < td class = "code" > < div class = "highlight" > < pre > < / pre > < / div > < / td > < / tr > < tr id = "section-31" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-31" > ¶ < / a > < / div > < p > Add a token to the results, taking note of the line number.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > token: < / span > < span class = "nf" > (tag, value) -> < / span >
< span class = "nx" > @tokens< / span > < span class = "p" > .< / span > < span class = "nx" > push< / span > < span class = "p" > [< / span > < span class = "nx" > tag< / span > < span class = "p" > ,< / span > < span class = "nx" > value< / span > < span class = "p" > ,< / span > < span class = "nx" > @line< / span > < span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-32" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-32" > ¶ < / a > < / div > < p > Peek at a tag in the current token stream.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > tag: < / span > < span class = "nf" > (index, tag) -> < / span >
< span class = "p" > (< / span > < span class = "nv" > tok = < / span > < span class = "nx" > last< / span > < span class = "nx" > @tokens< / span > < span class = "p" > ,< / span > < span class = "nx" > index< / span > < span class = "p" > )< / span > < span class = "o" > and< / span > < span class = "k" > if< / span > < span class = "nx" > tag< / span > < span class = "k" > then< / span > < span class = "nx" > tok< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > =< / span > < span class = "nx" > tag< / span > < span class = "k" > else< / span > < span class = "nx" > tok< / span > < span class = "p" > [< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-33" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-33" > ¶ < / a > < / div > < p > Peek at a value in the current token stream.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > value: < / span > < span class = "nf" > (index, val) -> < / span >
< span class = "p" > (< / span > < span class = "nv" > tok = < / span > < span class = "nx" > last< / span > < span class = "nx" > @tokens< / span > < span class = "p" > ,< / span > < span class = "nx" > index< / span > < span class = "p" > )< / span > < span class = "o" > and< / span > < span class = "k" > if< / span > < span class = "nx" > val< / span > < span class = "k" > then< / span > < span class = "nx" > tok< / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < span class = "o" > =< / span > < span class = "nx" > val< / span > < span class = "k" > else< / span > < span class = "nx" > tok< / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-34" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-34" > ¶ < / a > < / div > < p > Are we in the midst of an unfinished expression?< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > unfinished: < / span > < span class = "o" > -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "nx" > LINE_CONTINUER< / span > < span class = "p" > .< / span > < span class = "nx" > test< / span > < span class = "p" > (< / span > < span class = "nx" > @chunk< / span > < span class = "p" > )< / span > < span class = "o" > or< / span >
2011-11-08 18:01:45 -05:00
< span class = "nx" > @tag< / span > < span class = "p" > ()< / span > < span class = "k" > in< / span > < span class = "p" > [< / span > < span class = "s1" > ' \\' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' .' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ?.' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' UNARY' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' MATH' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' +' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' -' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' SHIFT' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' RELATION' < / span >
2011-12-18 13:21:00 -05:00
< span class = "s1" > ' COMPARE' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' LOGIC' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' COMPOUND_ASSIGN' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' THROW' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' EXTENDS' < / span > < span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-35" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-35" > ¶ < / a > < / div > < p > Converts newlines for string literals.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > escapeLines: < / span > < span class = "nf" > (str, heredoc) -> < / span >
< span class = "nx" > str< / span > < span class = "p" > .< / span > < span class = "nx" > replace< / span > < span class = "nx" > MULTILINER< / span > < span class = "p" > ,< / span > < span class = "k" > if< / span > < span class = "nx" > heredoc< / span > < span class = "k" > then< / span > < span class = "s1" > ' \\n' < / span > < span class = "k" > else< / span > < span class = "s1" > ' ' < / span > < / pre > < / div > < / td > < / tr > < tr id = "section-36" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-36" > ¶ < / a > < / div > < p > Constructs a string token by escaping quotes and newlines.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > makeString: < / span > < span class = "nf" > (body, quote, heredoc) -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > return< / span > < span class = "nx" > quote< / span > < span class = "o" > +< / span > < span class = "nx" > quote< / span > < span class = "nx" > unless< / span > < span class = "nx" > body< / span >
2011-12-18 13:21:00 -05:00
< span class = "nv" > body = < / span > < span class = "nx" > body< / span > < span class = "p" > .< / span > < span class = "nx" > replace< / span > < span class = "sr" > /\\([\s\S])/g< / span > < span class = "p" > ,< / span > < span class = "nf" > (match, contents) -> < / span >
2010-11-20 14:22:28 -05:00
< span class = "k" > if< / span > < span class = "nx" > contents< / span > < span class = "k" > in< / span > < span class = "p" > [< / span > < span class = "s1" > ' \n' < / span > < span class = "p" > ,< / span > < span class = "nx" > quote< / span > < span class = "p" > ]< / span > < span class = "k" > then< / span > < span class = "nx" > contents< / span > < span class = "k" > else< / span > < span class = "nx" > match< / span >
< span class = "nv" > body = < / span > < span class = "nx" > body< / span > < span class = "p" > .< / span > < span class = "nx" > replace< / span > < span class = "err" > /// #{quote} ///g, ' \\$& ' < / span >
2011-11-08 18:01:45 -05:00
< span class = "nx" > quote< / span > < span class = "o" > +< / span > < span class = "nx" > @escapeLines< / span > < span class = "p" > (< / span > < span class = "nx" > body< / span > < span class = "p" > ,< / span > < span class = "nx" > heredoc< / span > < span class = "p" > )< / span > < span class = "o" > +< / span > < span class = "nx" > quote< / span >
2011-12-18 13:21:00 -05:00
< / pre > < / div > < / td > < / tr > < tr id = "section-37" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-37" > ¶ < / a > < / div > < p > Throws a syntax error on the current < code > @line< / code > .< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > error: < / span > < span class = "nf" > (message) -> < / span >
2011-11-08 18:01:45 -05:00
< span class = "k" > throw< / span > < span class = "nx" > SyntaxError< / span > < span class = "s2" > " #{message} on line #{ @line + 1}" < / span > < / pre > < / div > < / td > < / tr > < tr id = "section-38" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-38" > ¶ < / a > < / div > < h2 > Constants< / h2 > < / td > < td class = "code" > < div class = "highlight" > < pre > < / pre > < / div > < / td > < / tr > < tr id = "section-39" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-39" > ¶ < / a > < / div > < p > Keywords that CoffeeScript shares in common with JavaScript.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > JS_KEYWORDS = < / span > < span class = "p" > [< / span >
2010-11-20 14:22:28 -05:00
< span class = "s1" > ' true' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' false' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' null' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' this' < / span >
< span class = "s1" > ' new' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' delete' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' typeof' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' in' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' instanceof' < / span >
< span class = "s1" > ' return' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' throw' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' break' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' continue' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' debugger' < / span >
2010-12-24 11:02:10 -08:00
< span class = "s1" > ' if' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' else' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' switch' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' for' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' while' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' do' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' try' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' catch' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' finally' < / span >
2010-11-20 14:22:28 -05:00
< span class = "s1" > ' class' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' extends' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' super' < / span >
2011-11-08 18:01:45 -05:00
< span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-40" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-40" > ¶ < / a > < / div > < p > CoffeeScript-only keywords.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > COFFEE_KEYWORDS = < / span > < span class = "p" > [< / span > < span class = "s1" > ' undefined' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' then' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' unless' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' until' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' loop' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' of' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' by' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' when' < / span > < span class = "p" > ]< / span >
2011-05-01 12:08:15 -04:00
< span class = "nv" > COFFEE_ALIAS_MAP =< / span >
2010-11-20 14:22:28 -05:00
< span class = "o" > and< / span > < span class = "o" > :< / span > < span class = "s1" > ' & & ' < / span >
< span class = "o" > or< / span > < span class = "o" > :< / span > < span class = "s1" > ' ||' < / span >
< span class = "o" > is< / span > < span class = "o" > :< / span > < span class = "s1" > ' ==' < / span >
< span class = "o" > isnt< / span > < span class = "o" > :< / span > < span class = "s1" > ' !=' < / span >
< span class = "o" > not< / span > < span class = "o" > :< / span > < span class = "s1" > ' !' < / span >
< span class = "kc" > yes< / span > < span class = "o" > :< / span > < span class = "s1" > ' true' < / span >
< span class = "kc" > no< / span > < span class = "o" > :< / span > < span class = "s1" > ' false' < / span >
< span class = "kc" > on< / span > < span class = "o" > :< / span > < span class = "s1" > ' true' < / span >
2011-05-01 12:08:15 -04:00
< span class = "kc" > off< / span > < span class = "o" > :< / span > < span class = "s1" > ' false' < / span >
2011-08-04 23:17:23 -04:00
2011-05-01 12:08:15 -04:00
< span class = "nv" > COFFEE_ALIASES = < / span > < span class = "p" > (< / span > < span class = "nx" > key< / span > < span class = "k" > for< / span > < span class = "nx" > key< / span > < span class = "k" > of< / span > < span class = "nx" > COFFEE_ALIAS_MAP< / span > < span class = "p" > )< / span >
2011-11-08 18:01:45 -05:00
< span class = "nv" > COFFEE_KEYWORDS = < / span > < span class = "nx" > COFFEE_KEYWORDS< / span > < span class = "p" > .< / span > < span class = "nx" > concat< / span > < span class = "nx" > COFFEE_ALIASES< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-41" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-41" > ¶ < / a > < / div > < p > The list of keywords that are reserved by JavaScript, but not used, or are
2010-03-07 14:41:52 -05:00
used by CoffeeScript internally. We throw an error when these are encountered,
2010-08-04 23:14:34 -04:00
to avoid having a JavaScript error at runtime.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > RESERVED = < / span > < span class = "p" > [< / span >
2010-12-24 11:02:10 -08:00
< span class = "s1" > ' case' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' default' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' function' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' var' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' void' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' with' < / span >
2010-11-20 14:22:28 -05:00
< span class = "s1" > ' const' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' let' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' enum' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' export' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' import' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' native' < / span >
2010-12-24 11:02:10 -08:00
< span class = "s1" > ' __hasProp' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' __extends' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' __slice' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' __bind' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' __indexOf' < / span >
2011-11-08 18:01:45 -05:00
< span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-42" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-42" > ¶ < / a > < / div > < p > The superset of both JavaScript keywords and reserved words, none of which may
2010-12-12 21:41:04 -05:00
be used as identifiers or properties.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > JS_FORBIDDEN = < / span > < span class = "nx" > JS_KEYWORDS< / span > < span class = "p" > .< / span > < span class = "nx" > concat< / span > < span class = "nx" > RESERVED< / span >
2011-11-08 18:01:45 -05:00
< span class = "nv" > exports.RESERVED = < / span > < span class = "nx" > RESERVED< / span > < span class = "p" > .< / span > < span class = "nx" > concat< / span > < span class = "p" > (< / span > < span class = "nx" > JS_KEYWORDS< / span > < span class = "p" > ).< / span > < span class = "nx" > concat< / span > < span class = "p" > (< / span > < span class = "nx" > COFFEE_KEYWORDS< / span > < span class = "p" > )< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-43" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-43" > ¶ < / a > < / div > < p > Token matching regexes.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > IDENTIFIER = < / span > < span class = "err" > /// ^< / span >
2011-01-31 22:39:12 -05:00
< span class = "p" > (< / span > < span class = "p" > [< / span > < span class = "nx" > $A< / span > < span class = "o" > -< / span > < span class = "nx" > Za< / span > < span class = "o" > -< / span > < span class = "nx" > z_< / span > < span class = "err" > \< / span > < span class = "nx" > x7f< / span > < span class = "o" > -< / span > < span class = "err" > \< / span > < span class = "nx" > uffff< / span > < span class = "p" > ][< / span > < span class = "nx" > $< / span > < span class = "err" > \< / span > < span class = "nx" > w< / span > < span class = "err" > \< / span > < span class = "nx" > x7f< / span > < span class = "o" > -< / span > < span class = "err" > \< / span > < span class = "nx" > uffff< / span > < span class = "p" > ]< / span > < span class = "o" > *< / span > < span class = "p" > )< / span >
2010-11-20 14:22:28 -05:00
< span class = "p" > (< / span > < span class = "p" > [< / span > < span class = "o" > ^< / span > < span class = "err" > \< / span > < span class = "nx" > n< / span > < span class = "err" > \< / span > < span class = "nx" > S< / span > < span class = "p" > ]< / span > < span class = "o" > *< / span > < span class = "o" > :< / span > < span class = "p" > (< / span > < span class = "o" > ?!:< / span > < span class = "p" > )< / span > < span class = "p" > )< / span > < span class = "o" > ?< / span > < span class = "c1" > # Is this a property name?< / span >
< span class = "err" > ///< / span >
< span class = "nv" > NUMBER = < / span > < span class = "err" > ///< / span >
< span class = "o" > ^< / span > < span class = "mi" > 0< / span > < span class = "nx" > x< / span > < span class = "p" > [< / span > < span class = "err" > \< / span > < span class = "nx" > da< / span > < span class = "o" > -< / span > < span class = "nx" > f< / span > < span class = "p" > ]< / span > < span class = "o" > +< / span > < span class = "o" > |< / span > < span class = "c1" > # hex< / span >
2011-11-08 18:01:45 -05:00
< span class = "o" > ^< / span > < span class = "mi" > 0< / span > < span class = "nx" > b< / span > < span class = "p" > [< / span > < span class = "mi" > 01< / span > < span class = "p" > ]< / span > < span class = "o" > +< / span > < span class = "o" > |< / span > < span class = "c1" > # binary< / span >
2011-08-04 23:17:23 -04:00
< span class = "o" > ^< / span > < span class = "err" > \< / span > < span class = "nx" > d< / span > < span class = "o" > *< / span > < span class = "err" > \< / span > < span class = "p" > .< / span > < span class = "o" > ?< / span > < span class = "err" > \< / span > < span class = "nx" > d< / span > < span class = "o" > +< / span > < span class = "p" > (< / span > < span class = "o" > ?:< / span > < span class = "nx" > e< / span > < span class = "p" > [< / span > < span class = "o" > +-< / span > < span class = "p" > ]< / span > < span class = "o" > ?< / span > < span class = "err" > \< / span > < span class = "nx" > d< / span > < span class = "o" > +< / span > < span class = "p" > )< / span > < span class = "o" > ?< / span > < span class = "c1" > # decimal< / span >
2010-11-20 14:22:28 -05:00
< span class = "err" > ///i< / span >
< span class = "nv" > HEREDOC = < / span > < span class = "err" > /// ^ (" " " |' ' ' ) ([\s\S]*?) (?:\n[^\n\S]*)? \1 ///< / span >
< span class = "nv" > OPERATOR = < / span > < span class = "err" > /// ^ (< / span >
< span class = "o" > ?:< / span > < span class = "p" > [< / span > < span class = "o" > -=< / span > < span class = "p" > ]< / span > < span class = "o" > > < / span > < span class = "c1" > # function< / span >
< span class = "o" > |< / span > < span class = "p" > [< / span > < span class = "o" > -+*< / span > < span class = "sr" > /%< > & |^!?=]= # compound assign /< / span > < span class = "nx" > compare< / span >
< span class = "o" > |< / span > < span class = "o" > > > > =?< / span > < span class = "c1" > # zero-fill right shift< / span >
< span class = "o" > |< / span > < span class = "p" > ([< / span > < span class = "o" > -+:< / span > < span class = "p" > ])< / span > < span class = "err" > \< / span > < span class = "mi" > 1< / span > < span class = "c1" > # doubles< / span >
< span class = "o" > |< / span > < span class = "p" > ([< / span > < span class = "o" > & |< > < / span > < span class = "p" > ])< / span > < span class = "err" > \< / span > < span class = "mi" > 2< / span > < span class = "o" > =?< / span > < span class = "c1" > # logic / shift< / span >
< span class = "o" > |< / span > < span class = "err" > \< / span > < span class = "o" > ?< / span > < span class = "err" > \< / span > < span class = "p" > .< / span > < span class = "c1" > # soak access< / span >
2010-11-21 21:19:49 -05:00
< span class = "o" > |< / span > < span class = "err" > \< / span > < span class = "p" > .{< / span > < span class = "mi" > 2< / span > < span class = "p" > ,< / span > < span class = "mi" > 3< / span > < span class = "p" > }< / span > < span class = "c1" > # range or splat< / span >
2010-11-20 14:22:28 -05:00
< span class = "p" > )< / span > < span class = "o" > /< / span > < span class = "err" > //< / span >
< span class = "nv" > WHITESPACE = < / span > < span class = "sr" > /^[^\n\S]+/< / span >
2010-12-24 11:02:10 -08:00
< span class = "nv" > COMMENT = < / span > < span class = "sr" > /^###([^#][\s\S]*?)(?:###[^\n\S]*|(?:###)?$)|^(?:\s*#(?!##[^#]).*)+/< / span >
2010-11-20 14:22:28 -05:00
< span class = "nv" > CODE = < / span > < span class = "sr" > /^[-=]> /< / span >
< span class = "nv" > MULTI_DENT = < / span > < span class = "sr" > /^(?:\n[^\n\S]*)+/< / span >
< span class = "nv" > SIMPLESTR = < / span > < span class = "sr" > /^' [^\\' ]*(?:\\.[^\\' ]*)*' /< / span >
2011-11-08 18:01:45 -05:00
< span class = "nv" > JSTOKEN = < / span > < span class = "sr" > /^`[^\\`]*(?:\\.[^\\`]*)*`/< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-44" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-44" > ¶ < / a > < / div > < p > Regex-matching-regexes.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > REGEX = < / span > < span class = "err" > /// ^< / span >
< span class = "p" > (< / span > < span class = "err" > / (?! [\s=] ) # disallow leading whitespace or equals signs< / span >
2010-11-20 14:22:28 -05:00
< span class = "p" > [< / span > < span class = "o" > ^< / span > < span class = "p" > [< / span > < span class = "err" > / \n \\ ]* # every other thing< / span >
< span class = "p" > (< / span > < span class = "o" > ?:< / span >
< span class = "p" > (< / span > < span class = "o" > ?:< / span > < span class = "err" > \\< / span > < span class = "p" > [< / span > < span class = "err" > \< / span > < span class = "nx" > s< / span > < span class = "err" > \< / span > < span class = "nx" > S< / span > < span class = "p" > ]< / span > < span class = "c1" > # anything escaped< / span >
< span class = "o" > |< / span > < span class = "err" > \< / span > < span class = "p" > [< / span > < span class = "c1" > # character class< / span >
< span class = "p" > [< / span > < span class = "o" > ^< / span > < span class = "err" > \< / span > < span class = "p" > ]< / span > < span class = "err" > \< / span > < span class = "nx" > n< / span > < span class = "err" > \\< / span > < span class = "p" > ]< / span > < span class = "o" > *< / span >
< span class = "p" > (< / span > < span class = "o" > ?:< / span > < span class = "err" > \\< / span > < span class = "p" > [< / span > < span class = "err" > \< / span > < span class = "nx" > s< / span > < span class = "err" > \< / span > < span class = "nx" > S< / span > < span class = "p" > ]< / span > < span class = "p" > [< / span > < span class = "o" > ^< / span > < span class = "err" > \< / span > < span class = "p" > ]< / span > < span class = "err" > \< / span > < span class = "nx" > n< / span > < span class = "err" > \\< / span > < span class = "p" > ]< / span > < span class = "o" > *< / span > < span class = "p" > )< / span > < span class = "o" > *< / span >
< span class = "p" > ]< / span >
< span class = "p" > )< / span > < span class = "p" > [< / span > < span class = "o" > ^< / span > < span class = "p" > [< / span > < span class = "err" > / \n \\ ]*< / span >
< span class = "p" > )< / span > < span class = "o" > *< / span >
2011-11-08 18:01:45 -05:00
< span class = "err" > /) ([imgy]{0,4}) (?!\w)< / span >
2010-11-20 14:22:28 -05:00
< span class = "o" > /< / span > < span class = "err" > //< / span >
< span class = "nv" > HEREGEX = < / span > < span class = "err" > /// ^ /{3} ([\s\S]+?) /{3} ([imgy]{0,4}) (?!\w) ///< / span >
2011-11-08 18:01:45 -05:00
< span class = "nv" > HEREGEX_OMIT = < / span > < span class = "sr" > /\s+(?:#.*)?/g< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-45" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-45" > ¶ < / a > < / div > < p > Token cleaning regexes.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > MULTILINER = < / span > < span class = "sr" > /\n/g< / span >
2010-11-20 14:22:28 -05:00
< span class = "nv" > HEREDOC_INDENT = < / span > < span class = "sr" > /\n+([^\n\S]*)/g< / span >
2011-05-01 12:08:15 -04:00
< span class = "nv" > HEREDOC_ILLEGAL = < / span > < span class = "sr" > /\*\//< / span >
< span class = "nv" > LINE_CONTINUER = < / span > < span class = "err" > /// ^ \s* (?: , | \??\.(?![.\d]) | :: ) ///< / span >
2010-11-20 14:22:28 -05:00
2011-11-08 18:01:45 -05:00
< span class = "nv" > TRAILING_SPACES = < / span > < span class = "sr" > /\s+$/< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-46" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-46" > ¶ < / a > < / div > < p > Compound assignment tokens.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > COMPOUND_ASSIGN = < / span > < span class = "p" > [< / span >
2010-11-20 14:22:28 -05:00
< span class = "s1" > ' -=' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' +=' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' /=' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' *=' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' %=' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ||=' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' & & =' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ?=' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' < < =' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' > > =' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' > > > =' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' & =' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ^=' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' |=' < / span >
2011-11-08 18:01:45 -05:00
< span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-47" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-47" > ¶ < / a > < / div > < p > Unary tokens.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > UNARY = < / span > < span class = "p" > [< / span > < span class = "s1" > ' !' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ~' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' NEW' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' TYPEOF' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' DELETE' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' DO' < / span > < span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-48" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-48" > ¶ < / a > < / div > < p > Logical tokens.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > LOGIC = < / span > < span class = "p" > [< / span > < span class = "s1" > ' & & ' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ||' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' & ' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' |' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ^' < / span > < span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-49" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-49" > ¶ < / a > < / div > < p > Bit-shifting tokens.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > SHIFT = < / span > < span class = "p" > [< / span > < span class = "s1" > ' < < ' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' > > ' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' > > > ' < / span > < span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-50" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-50" > ¶ < / a > < / div > < p > Comparison tokens.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > COMPARE = < / span > < span class = "p" > [< / span > < span class = "s1" > ' ==' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' !=' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' < ' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' > ' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' < =' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' > =' < / span > < span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-51" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-51" > ¶ < / a > < / div > < p > Mathematical tokens.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > MATH = < / span > < span class = "p" > [< / span > < span class = "s1" > ' *' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' /' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' %' < / span > < span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-52" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-52" > ¶ < / a > < / div > < p > Relational tokens that are negatable with < code > not< / code > prefix.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "n
2010-03-07 14:41:52 -05:00
a division operator might.< / p >
2010-05-15 01:18:05 -04:00
< p > See: http://www.mozilla.org/js/language/js20-2002-04/rationale/syntax.html#regular-expressions< / p >
2010-03-07 14:41:52 -05:00
2011-11-08 18:01:45 -05:00
< p > Our list is shorter, due to sans-parentheses method calls.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > NOT_REGEX = < / span > < span class = "p" > [< / span > < span class = "s1" > ' NUMBER' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' REGEX' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' BOOL' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ++' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' --' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ]' < / span > < span class = "p" > ]< / span > < / pre > < / div > < / td > < / tr > < tr id = "section-55" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-55" > ¶ < / a > < / div > < p > If the previous token is not spaced, there are more preceding tokens that
force a division parse:< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > NOT_SPACED_REGEX = < / span > < span class = "nx" > NOT_REGEX< / span > < span class = "p" > .< / span > < span class = "nx" > concat< / span > < span class = "s1" > ' )' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' }' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' THIS' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' IDENTIFIER' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' STRING' < / span > < / pre > < / div > < / td > < / tr > < tr id = "section-56" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-56" > ¶ < / a > < / div > < p > Tokens which could legitimately be invoked or indexed. An opening
2010-03-07 14:41:52 -05:00
parentheses or bracket following these tokens will be recorded as the start
2010-11-20 14:22:28 -05:00
of a function invocation or indexing operation.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > CALLABLE = < / span > < span class = "p" > [< / span > < span class = "s1" > ' IDENTIFIER' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' STRING' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' REGEX' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' )' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ]' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' }' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ?' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' ::' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' @' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' THIS' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' SUPER' < / span > < span class = "p" > ]< / span >
2011-11-08 18:01:45 -05:00
< span class = "nv" > INDEXABLE = < / span > < span class = "nx" > CALLABLE< / span > < span class = "p" > .< / span > < span class = "nx" > concat< / span > < span class = "s1" > ' NUMBER' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' BOOL' < / span > < / pre > < / div > < / td > < / tr > < tr id = "section-57" > < td class = "docs" > < div class = "pilwrap" > < a class = "pilcrow" href = "#section-57" > ¶ < / a > < / div > < p > Tokens that, when immediately preceding a < code > WHEN< / code > , indicate that the < code > WHEN< / code >
2010-03-07 14:41:52 -05:00
occurs at the start of a line. We disambiguate these from trailing whens to
2010-11-20 14:22:28 -05:00
avoid an ambiguity in the grammar.< / p > < / td > < td class = "code" > < div class = "highlight" > < pre > < span class = "nv" > LINE_BREAK = < / span > < span class = "p" > [< / span > < span class = "s1" > ' INDENT' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' OUTDENT' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' TERMINATOR' < / span > < span class = "p" > ]< / span >
2010-03-06 23:48:06 -05:00
< / pre > < / div > < / td > < / tr > < / tbody > < / table > < / div > < / body > < / html >
