2018-02-07 06:28:15 +00:00
// Generated by CoffeeScript 2.2.1
2010-07-25 07:15:12 +00:00
( function ( ) {
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// The CoffeeScript Lexer. Uses a series of token-matching regexes to attempt
// matches against the beginning of the source code. When a match is found,
// a token is produced, we consume the match, and start again. Tokens are in the
// form:
// [tag, value, locationData]
// where locationData is {first_line, first_column, last_line, last_column}, which is a
// format that can be fed directly into [Jison](https://github.com/zaach/jison). These
// are read by jison in the `parser.lexer` function defined in coffeescript.coffee.
2017-12-09 06:41:03 +00:00
var BOM , BOOL , CALLABLE , CODE , COFFEE _ALIASES , COFFEE _ALIAS _MAP , COFFEE _KEYWORDS , COMMENT , COMPARABLE _LEFT _SIDE , COMPARE , COMPOUND _ASSIGN , CSX _ATTRIBUTE , CSX _FRAGMENT _IDENTIFIER , CSX _IDENTIFIER , CSX _INTERPOLATION , HERECOMMENT _ILLEGAL , HEREDOC _DOUBLE , HEREDOC _INDENT , HEREDOC _SINGLE , HEREGEX , HEREGEX _OMIT , HERE _JSTOKEN , IDENTIFIER , INDENTABLE _CLOSERS , INDEXABLE , INSIDE _CSX , INVERSES , JSTOKEN , JS _KEYWORDS , LEADING _BLANK _LINE , LINE _BREAK , LINE _CONTINUER , Lexer , MATH , MULTI _DENT , NOT _REGEX , NUMBER , OPERATOR , POSSIBLY _DIVISION , REGEX , REGEX _FLAGS , REGEX _ILLEGAL , REGEX _INVALID _ESCAPE , RELATION , RESERVED , Rewriter , SHIFT , SIMPLE _STRING _OMIT , STRICT _PROSCRIBED , STRING _DOUBLE , STRING _INVALID _ESCAPE , STRING _OMIT , STRING _SINGLE , STRING _START , TRAILING _BLANK _LINE , TRAILING _SPACES , UNARY , UNARY _MATH , UNFINISHED , UNICODE _CODE _POINT _ESCAPE , VALID _FLAGS , WHITESPACE , attachCommentsToNode , compact , count , invertLiterate , isForFrom , isUnassignable , key , locationDataToString , merge , repeat , starts , throwSyntaxError ,
2018-01-16 03:24:21 +00:00
indexOf = [ ] . indexOf ,
slice = [ ] . slice ;
2011-09-18 22:16:39 +00:00
2017-04-06 17:06:45 +00:00
( { Rewriter , INVERSES } = require ( './rewriter' ) ) ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Import the helpers we need.
( { count , starts , compact , repeat , invertLiterate , merge , attachCommentsToNode , locationDataToString , throwSyntaxError } = require ( './helpers' ) ) ;
2013-02-25 17:41:34 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// The Lexer Class
// ---------------
// The Lexer class reads a stream of CoffeeScript and divvies it up into tagged
// tokens. Some potential ambiguity in the grammar has been avoided by
// pushing some extra smarts into the Lexer.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
exports . Lexer = Lexer = class Lexer {
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// **tokenize** is the Lexer's main method. Scan by attempting to match tokens
// one at a time, using a regular expression anchored at the start of the
// remaining code, or a custom recursive token-matching method
// (for interpolations). When the next token has been recorded, we move forward
// within the code past the token, and begin again.
// Each tokenizing method is responsible for returning the number of characters
// it has consumed.
// Before returning the token stream, run it through the [Rewriter](rewriter.html).
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
tokenize ( code , opts = { } ) {
2017-06-30 01:39:05 +00:00
var consumed , end , i , ref ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
this . literate = opts . literate ; // Are we lexing literate CoffeeScript?
this . indent = 0 ; // The current indentation level.
this . baseIndent = 0 ; // The overall minimum indentation level.
this . indebt = 0 ; // The over-indentation at the current level.
this . outdebt = 0 ; // The under-outdentation at the current level.
this . indents = [ ] ; // The stack of all current indentation levels.
this . indentLiteral = '' ; // The indentation.
this . ends = [ ] ; // The stack for pairing up tokens.
this . tokens = [ ] ; // Stream of parsed tokens in the form `['TYPE', value, location data]`.
this . seenFor = false ; // Used to recognize `FORIN`, `FOROF` and `FORFROM` tokens.
this . seenImport = false ; // Used to recognize `IMPORT FROM? AS?` tokens.
this . seenExport = false ; // Used to recognize `EXPORT FROM? AS?` tokens.
this . importSpecifierList = false ; // Used to identify when in an `IMPORT {...} FROM? ...`.
this . exportSpecifierList = false ; // Used to identify when in an `EXPORT {...} FROM? ...`.
this . csxDepth = 0 ; // Used to optimize CSX checks, how deep in CSX we are.
2017-08-03 04:00:39 +00:00
this . csxObjAttribute = { } ; // Used to detect if CSX attributes is wrapped in {} (<div {props...} />).
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
this . chunkLine = opts . line || 0 ; // The start line for the current @chunk.
this . chunkColumn = opts . column || 0 ; // The start column of the current @chunk.
code = this . clean ( code ) ; // The stripped, cleaned original source code.
// At every position, run through this list of attempted matches,
// short-circuiting if any of them succeed. Their order determines precedence:
// `@literalToken` is the fallback catch-all.
2010-10-22 14:47:40 +00:00
i = 0 ;
while ( this . chunk = code . slice ( i ) ) {
2017-06-07 06:33:46 +00:00
consumed = this . identifierToken ( ) || this . commentToken ( ) || this . whitespaceToken ( ) || this . lineToken ( ) || this . stringToken ( ) || this . numberToken ( ) || this . csxToken ( ) || this . regexToken ( ) || this . jsToken ( ) || this . literalToken ( ) ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Update position.
2017-04-06 17:06:45 +00:00
[ this . chunkLine , this . chunkColumn ] = this . getLineAndColumnFromChunk ( consumed ) ;
2012-11-17 00:09:56 +00:00
i += consumed ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
if ( opts . untilBalanced && this . ends . length === 0 ) {
return {
tokens : this . tokens ,
index : i
} ;
}
2010-02-28 00:40:53 +00:00
}
2010-06-12 23:05:13 +00:00
this . closeIndentation ( ) ;
2015-01-03 23:28:23 +00:00
if ( end = this . ends . pop ( ) ) {
2017-06-30 01:39:05 +00:00
this . error ( ` missing ${ end . tag } ` , ( ( ref = end . origin ) != null ? ref : end ) [ 2 ] ) ;
2012-04-10 18:57:45 +00:00
}
if ( opts . rewrite === false ) {
return this . tokens ;
}
2013-04-29 03:09:46 +00:00
return ( new Rewriter ) . rewrite ( this . tokens ) ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Preprocess the code to remove leading and trailing whitespace, carriage
// returns, etc. If we’ re lexing literate CoffeeScript, strip external Markdown
// by removing all lines that aren’ t indented by at least four spaces or a tab.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
clean ( code ) {
2013-01-07 04:56:58 +00:00
if ( code . charCodeAt ( 0 ) === BOM ) {
code = code . slice ( 1 ) ;
}
2013-03-01 18:38:55 +00:00
code = code . replace ( /\r/g , '' ) . replace ( TRAILING _SPACES , '' ) ;
2012-09-26 00:15:40 +00:00
if ( WHITESPACE . test ( code ) ) {
2016-11-28 14:05:51 +00:00
code = ` \n ${ code } ` ;
2013-03-01 18:38:55 +00:00
this . chunkLine -- ;
2012-09-26 00:15:40 +00:00
}
if ( this . literate ) {
2013-03-05 02:45:57 +00:00
code = invertLiterate ( code ) ;
2012-09-26 00:15:40 +00:00
}
return code ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2012-09-26 00:15:40 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Tokenizers
// ----------
// Matches identifying literals: variables, keywords, method names, etc.
// Check to ensure that JavaScript reserved words aren’ t being used as
// identifiers. Because CoffeeScript reserves a handful of keywords that are
// allowed in JavaScript, we’ re careful not to tag them as keywords when
// referenced as property names here, so you can still do `jQuery.is()` even
// though `is` means `===` otherwise.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
identifierToken ( ) {
2018-01-31 05:17:55 +00:00
var alias , colon , colonOffset , colonToken , id , idLength , inCSXTag , input , match , poppedToken , prev , prevprev , ref , ref1 , ref10 , ref2 , ref3 , ref4 , ref5 , ref6 , ref7 , ref8 , ref9 , regExSuper , regex , sup , tag , tagToken ;
2017-06-07 06:33:46 +00:00
inCSXTag = this . atCSXTag ( ) ;
regex = inCSXTag ? CSX _ATTRIBUTE : IDENTIFIER ;
if ( ! ( match = regex . exec ( this . chunk ) ) ) {
2012-04-10 18:57:45 +00:00
return 0 ;
}
2017-04-06 17:06:45 +00:00
[ input , id , colon ] = match ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Preserve length of id for location data
2012-11-17 00:09:56 +00:00
idLength = id . length ;
poppedToken = void 0 ;
2010-11-28 23:33:43 +00:00
if ( id === 'own' && this . tag ( ) === 'FOR' ) {
this . token ( 'OWN' , id ) ;
2010-10-22 14:47:40 +00:00
return id . length ;
2010-07-16 01:18:35 +00:00
}
2014-09-06 11:53:21 +00:00
if ( id === 'from' && this . tag ( ) === 'YIELD' ) {
this . token ( 'FROM' , id ) ;
return id . length ;
}
2016-10-26 12:37:19 +00:00
if ( id === 'as' && this . seenImport ) {
2016-09-15 06:30:58 +00:00
if ( this . value ( ) === '*' ) {
this . tokens [ this . tokens . length - 1 ] [ 0 ] = 'IMPORT_ALL' ;
2018-01-31 14:33:17 +00:00
} else if ( ref = this . value ( true ) , indexOf . call ( COFFEE _KEYWORDS , ref ) >= 0 ) {
prev = this . prev ( ) ;
[ prev [ 0 ] , prev [ 1 ] ] = [ 'IDENTIFIER' , this . value ( true ) ] ;
2016-10-26 12:37:19 +00:00
}
2017-04-06 17:06:45 +00:00
if ( ( ref1 = this . tag ( ) ) === 'DEFAULT' || ref1 === 'IMPORT_ALL' || ref1 === 'IDENTIFIER' ) {
2016-10-26 12:37:19 +00:00
this . token ( 'AS' , id ) ;
return id . length ;
2016-09-15 06:30:58 +00:00
}
}
2017-10-12 18:47:12 +00:00
if ( id === 'as' && this . seenExport ) {
if ( ( ref2 = this . tag ( ) ) === 'IDENTIFIER' || ref2 === 'DEFAULT' ) {
this . token ( 'AS' , id ) ;
return id . length ;
}
if ( ref3 = this . value ( true ) , indexOf . call ( COFFEE _KEYWORDS , ref3 ) >= 0 ) {
prev = this . prev ( ) ;
[ prev [ 0 ] , prev [ 1 ] ] = [ 'IDENTIFIER' , this . value ( true ) ] ;
this . token ( 'AS' , id ) ;
return id . length ;
}
Support import and export of ES2015 modules (#4300)
This pull request adds support for ES2015 modules, by recognizing `import` and `export` statements. The following syntaxes are supported, based on the MDN [import](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import) and [export](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/export) pages:
```js
import "module-name"
import defaultMember from "module-name"
import * as name from "module-name"
import { } from "module-name"
import { member } from "module-name"
import { member as alias } from "module-name"
import { member1, member2 as alias2, … } from "module-name"
import defaultMember, * as name from "module-name"
import defaultMember, { … } from "module-name"
export default expression
export class name
export { }
export { name }
export { name as exportedName }
export { name as default }
export { name1, name2 as exportedName2, name3 as default, … }
export * from "module-name"
export { … } from "module-name"
```
As a subsitute for ECMAScript’s `export var name = …` and `export function name {}`, CoffeeScript also supports:
```js
export name = …
```
CoffeeScript also supports optional commas within `{ … }`.
This PR converts the supported `import` and `export` statements into ES2015 `import` and `export` statements; it **does not resolve the modules**. So any CoffeeScript with `import` or `export` statements will be output as ES2015, and will need to be transpiled by another tool such as Babel before it can be used in a browser. We will need to add a warning to the documentation explaining this.
This should be fully backwards-compatible, as `import` and `export` were previously reserved keywords. No flags are used.
There are extensive tests included, though because no current JavaScript runtime supports `import` or `export`, the tests compare strings of what the compiled CoffeeScript output is against what the expected ES2015 should be. I also conducted two more elaborate tests:
* I forked the [ember-piqu](https://github.com/pauc/piqu-ember) project, which was an Ember CLI app that used ember-cli-coffeescript and [ember-cli-coffees6](https://github.com/alexspeller/ember-cli-coffees6) (which adds “support” for `import`/`export` by wrapping such statements in backticks before passing the result to the CoffeeScript compiler). I removed `ember-cli-coffees6` and replaced the CoffeeScript compiler used in the build chain with this code, and the app built without errors. [Demo here.](https://github.com/GeoffreyBooth/coffeescript-modules-test-piqu)
* I also forked the [CoffeeScript version of Meteor’s Todos example app](https://github.com/meteor/todos/tree/coffeescript), and replaced all of its `require` statements with the `import` and `export` statements from the original ES2015 version of the app on its `master` branch. I then updated the `coffeescript` Meteor package in the app to use this new code, and again the app builds without errors. [Demo here.](https://github.com/GeoffreyBooth/coffeescript-modules-test-meteor-todos)
The discussion history for this work started [here](https://github.com/jashkenas/coffeescript/pull/4160) and continued [here](https://github.com/GeoffreyBooth/coffeescript/pull/2). @lydell provided guidance, and @JimPanic and @rattrayalex contributed essential code.
2016-09-14 18:46:05 +00:00
}
2017-10-12 18:47:12 +00:00
if ( id === 'default' && this . seenExport && ( ( ref4 = this . tag ( ) ) === 'EXPORT' || ref4 === 'AS' ) ) {
Support import and export of ES2015 modules (#4300)
This pull request adds support for ES2015 modules, by recognizing `import` and `export` statements. The following syntaxes are supported, based on the MDN [import](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import) and [export](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/export) pages:
```js
import "module-name"
import defaultMember from "module-name"
import * as name from "module-name"
import { } from "module-name"
import { member } from "module-name"
import { member as alias } from "module-name"
import { member1, member2 as alias2, … } from "module-name"
import defaultMember, * as name from "module-name"
import defaultMember, { … } from "module-name"
export default expression
export class name
export { }
export { name }
export { name as exportedName }
export { name as default }
export { name1, name2 as exportedName2, name3 as default, … }
export * from "module-name"
export { … } from "module-name"
```
As a subsitute for ECMAScript’s `export var name = …` and `export function name {}`, CoffeeScript also supports:
```js
export name = …
```
CoffeeScript also supports optional commas within `{ … }`.
This PR converts the supported `import` and `export` statements into ES2015 `import` and `export` statements; it **does not resolve the modules**. So any CoffeeScript with `import` or `export` statements will be output as ES2015, and will need to be transpiled by another tool such as Babel before it can be used in a browser. We will need to add a warning to the documentation explaining this.
This should be fully backwards-compatible, as `import` and `export` were previously reserved keywords. No flags are used.
There are extensive tests included, though because no current JavaScript runtime supports `import` or `export`, the tests compare strings of what the compiled CoffeeScript output is against what the expected ES2015 should be. I also conducted two more elaborate tests:
* I forked the [ember-piqu](https://github.com/pauc/piqu-ember) project, which was an Ember CLI app that used ember-cli-coffeescript and [ember-cli-coffees6](https://github.com/alexspeller/ember-cli-coffees6) (which adds “support” for `import`/`export` by wrapping such statements in backticks before passing the result to the CoffeeScript compiler). I removed `ember-cli-coffees6` and replaced the CoffeeScript compiler used in the build chain with this code, and the app built without errors. [Demo here.](https://github.com/GeoffreyBooth/coffeescript-modules-test-piqu)
* I also forked the [CoffeeScript version of Meteor’s Todos example app](https://github.com/meteor/todos/tree/coffeescript), and replaced all of its `require` statements with the `import` and `export` statements from the original ES2015 version of the app on its `master` branch. I then updated the `coffeescript` Meteor package in the app to use this new code, and again the app builds without errors. [Demo here.](https://github.com/GeoffreyBooth/coffeescript-modules-test-meteor-todos)
The discussion history for this work started [here](https://github.com/jashkenas/coffeescript/pull/4160) and continued [here](https://github.com/GeoffreyBooth/coffeescript/pull/2). @lydell provided guidance, and @JimPanic and @rattrayalex contributed essential code.
2016-09-14 18:46:05 +00:00
this . token ( 'DEFAULT' , id ) ;
return id . length ;
}
2017-08-03 02:41:01 +00:00
if ( id === 'do' && ( regExSuper = /^(\s*super)(?!\(\))/ . exec ( this . chunk . slice ( 3 ) ) ) ) {
this . token ( 'SUPER' , 'super' ) ;
this . token ( 'CALL_START' , '(' ) ;
this . token ( 'CALL_END' , ')' ) ;
[ input , sup ] = regExSuper ;
return sup . length + 3 ;
}
2017-04-09 04:59:09 +00:00
prev = this . prev ( ) ;
2017-10-12 18:47:12 +00:00
tag = colon || ( prev != null ) && ( ( ( ref5 = prev [ 0 ] ) === '.' || ref5 === '?.' || ref5 === '::' || ref5 === '?::' ) || ! prev . spaced && prev [ 0 ] === '@' ) ? 'PROPERTY' : 'IDENTIFIER' ;
2016-10-26 12:37:19 +00:00
if ( tag === 'IDENTIFIER' && ( indexOf . call ( JS _KEYWORDS , id ) >= 0 || indexOf . call ( COFFEE _KEYWORDS , id ) >= 0 ) && ! ( this . exportSpecifierList && indexOf . call ( COFFEE _KEYWORDS , id ) >= 0 ) ) {
2010-02-28 00:40:53 +00:00
tag = id . toUpperCase ( ) ;
2017-10-12 18:47:12 +00:00
if ( tag === 'WHEN' && ( ref6 = this . tag ( ) , indexOf . call ( LINE _BREAK , ref6 ) >= 0 ) ) {
2010-09-23 05:14:18 +00:00
tag = 'LEADING_WHEN' ;
2018-01-31 14:33:17 +00:00
} else if ( tag === 'FOR' ) {
this . seenFor = true ;
} else if ( tag === 'UNLESS' ) {
tag = 'IF' ;
} else if ( tag === 'IMPORT' ) {
this . seenImport = true ;
} else if ( tag === 'EXPORT' ) {
this . seenExport = true ;
} else if ( indexOf . call ( UNARY , tag ) >= 0 ) {
tag = 'UNARY' ;
} else if ( indexOf . call ( RELATION , tag ) >= 0 ) {
if ( tag !== 'INSTANCEOF' && this . seenFor ) {
tag = 'FOR' + tag ;
this . seenFor = false ;
2010-10-05 19:46:17 +00:00
} else {
2018-01-31 14:33:17 +00:00
tag = 'RELATION' ;
if ( this . value ( ) === '!' ) {
poppedToken = this . tokens . pop ( ) ;
id = '!' + id ;
2010-10-05 19:46:17 +00:00
}
}
2010-09-23 05:14:18 +00:00
}
2018-01-31 14:33:17 +00:00
} else if ( tag === 'IDENTIFIER' && this . seenFor && id === 'from' && isForFrom ( prev ) ) {
tag = 'FORFROM' ;
this . seenFor = false ;
// Throw an error on attempts to use `get` or `set` as keywords, or
// what CoffeeScript would normally interpret as calls to functions named
// `get` or `set`, i.e. `get({foo: function () {}})`.
} else if ( tag === 'PROPERTY' && prev ) {
if ( prev . spaced && ( ref7 = prev [ 0 ] , indexOf . call ( CALLABLE , ref7 ) >= 0 ) && /^[gs]et$/ . test ( prev [ 1 ] ) && this . tokens . length > 1 && ( ( ref8 = this . tokens [ this . tokens . length - 2 ] [ 0 ] ) !== '.' && ref8 !== '?.' && ref8 !== '@' ) ) {
this . error ( ` ' ${ prev [ 1 ] } ' cannot be used as a keyword, or as a function call without parentheses ` , prev [ 2 ] ) ;
} else if ( this . tokens . length > 2 ) {
prevprev = this . tokens [ this . tokens . length - 2 ] ;
if ( ( ( ref9 = prev [ 0 ] ) === '@' || ref9 === 'THIS' ) && prevprev && prevprev . spaced && /^[gs]et$/ . test ( prevprev [ 1 ] ) && ( ( ref10 = this . tokens [ this . tokens . length - 3 ] [ 0 ] ) !== '.' && ref10 !== '?.' && ref10 !== '@' ) ) {
this . error ( ` ' ${ prevprev [ 1 ] } ' cannot be used as a keyword, or as a function call without parentheses ` , prevprev [ 2 ] ) ;
2017-04-09 04:59:09 +00:00
}
}
2010-08-12 02:24:43 +00:00
}
2016-03-05 16:41:15 +00:00
if ( tag === 'IDENTIFIER' && indexOf . call ( RESERVED , id ) >= 0 ) {
2016-11-28 14:05:51 +00:00
this . error ( ` reserved word ' ${ id } ' ` , {
2016-03-05 16:41:15 +00:00
length : id . length
} ) ;
2010-06-15 04:54:02 +00:00
}
2017-10-12 18:47:12 +00:00
if ( ! ( tag === 'PROPERTY' || this . exportSpecifierList ) ) {
2015-01-30 19:33:03 +00:00
if ( indexOf . call ( COFFEE _ALIASES , id ) >= 0 ) {
2015-05-01 12:33:11 +00:00
alias = id ;
2012-04-10 18:57:45 +00:00
id = COFFEE _ALIAS _MAP [ id ] ;
}
2010-12-23 18:50:52 +00:00
tag = ( function ( ) {
2010-11-05 04:04:52 +00:00
switch ( id ) {
case '!' :
return 'UNARY' ;
case '==' :
case '!=' :
return 'COMPARE' ;
case 'true' :
case 'false' :
return 'BOOL' ;
2010-11-14 20:15:13 +00:00
case 'break' :
case 'continue' :
Refactor `Literal` into several subtypes
Previously, the parser created `Literal` nodes for many things. This resulted in
information loss. Instead of being able to check the node type, we had to use
regexes to tell the different types of `Literal`s apart. That was a bit like
parsing literals twice: Once in the lexer, and once (or more) in the compiler.
It also caused problems, such as `` `this` `` and `this` being indistinguishable
(fixes #2009).
Instead returning `new Literal` in the grammar, subtypes of it are now returned
instead, such as `NumberLiteral`, `StringLiteral` and `IdentifierLiteral`. `new
Literal` by itself is only used to represent code chunks that fit no category.
(While mentioning `NumberLiteral`, there's also `InfinityLiteral` now, which is
a subtype of `NumberLiteral`.)
`StringWithInterpolations` has been added as a subtype of `Parens`, and
`RegexWithInterpolations` as a subtype of `Call`. This makes it easier for other
programs to make use of CoffeeScript's "AST" (nodes). For example, it is now
possible to distinguish between `"a #{b} c"` and `"a " + b + " c"`. Fixes #4192.
`SuperCall` has been added as a subtype of `Call`.
Note, though, that some information is still lost, especially in the lexer. For
example, there is no way to distinguish a heredoc from a regular string, or a
heregex without interpolations from a regular regex. Binary and octal number
literals are indistinguishable from hexadecimal literals.
After the new subtypes were added, they were taken advantage of, removing most
regexes in nodes.coffee. `SIMPLENUM` (which matches non-hex integers) had to be
kept, though, because such numbers need special handling in JavaScript (for
example in `1..toString()`).
An especially nice hack to get rid of was using `new String()` for the token
value for reserved identifiers (to be able to set a property on them which could
survive through the parser). Now it's a good old regular string.
In range literals, slices, splices and for loop steps when number literals
are involved, CoffeeScript can do some optimizations, such as precomputing the
value of, say, `5 - 3` (outputting `2` instead of `5 - 3` literally). As a side
bonus, this now also works with hexadecimal number literals, such as `0x02`.
Finally, this also improves the output of `coffee --nodes`:
# Before:
$ bin/coffee -ne 'while true
"#{a}"
break'
Block
While
Value
Bool
Block
Value
Parens
Block
Op +
Value """"
Value
Parens
Block
Value "a" "break"
# After:
$ bin/coffee -ne 'while true
"#{a}"
break'
Block
While
Value BooleanLiteral: true
Block
Value
StringWithInterpolations
Block
Op +
Value StringLiteral: ""
Value
Parens
Block
Value IdentifierLiteral: a
StatementLiteral: break
2016-01-31 19:24:31 +00:00
case 'debugger' :
2010-11-14 20:15:13 +00:00
return 'STATEMENT' ;
Define proper operator precedence for bitwise/logical operators
This is an upstream port for the patch https://github.com/decaffeinate/coffeescript/pull/8
See https://github.com/decaffeinate/decaffeinate/issues/291 for the bug that this fixed.
For the most part, CoffeeScript and JavaScript have the same precedence rules,
but in some cases, the intermediate AST format didn't represent the actual
evaluation order. For example, in the expression `a or b and c`, the `and` is
evaluated first, but the parser treated the two operators with equal precedence.
This was still correct end-to-end because CoffeeScript simply emitted the result
without parens, but any intermediate tools using the CoffeeScript parser could
become confused.
Here are the JS operator precedence rules:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Operator_Precedence
For the most part, CoffeeScript already follows these. `COMPARE` operators
already behave differently due to chained comparisons, so I think we don't need
to worry about following JS precedence for those. So I think the only case where
it was behaving differently in an important way was for the binary/bitwise
operators that are being changed here.
As part of this change, I also introduced a new token tag, `BIN?`, for the
binary form of the `?` operator.
2016-10-09 19:00:38 +00:00
case '&&' :
case '||' :
return id ;
2010-11-05 04:04:52 +00:00
default :
return tag ;
}
2010-12-23 18:50:52 +00:00
} ) ( ) ;
2010-03-22 01:46:06 +00:00
}
2012-11-17 00:09:56 +00:00
tagToken = this . token ( tag , id , 0 , idLength ) ;
2015-05-01 12:33:11 +00:00
if ( alias ) {
tagToken . origin = [ tag , alias , tagToken [ 2 ] ] ;
}
2012-11-17 00:09:56 +00:00
if ( poppedToken ) {
2017-04-06 17:06:45 +00:00
[ tagToken [ 2 ] . first _line , tagToken [ 2 ] . first _column ] = [ poppedToken [ 2 ] . first _line , poppedToken [ 2 ] . first _column ] ;
2012-11-17 00:09:56 +00:00
}
2012-04-10 18:57:45 +00:00
if ( colon ) {
2017-06-07 06:33:46 +00:00
colonOffset = input . lastIndexOf ( inCSXTag ? '=' : ':' ) ;
colonToken = this . token ( ':' , ':' , colonOffset , colon . length ) ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
if ( inCSXTag ) { // used by rewriter
2017-06-07 06:33:46 +00:00
colonToken . csxColon = true ;
}
}
if ( inCSXTag && tag === 'IDENTIFIER' && prev [ 0 ] !== ':' ) {
this . token ( ',' , ',' , 0 , 0 , tagToken ) ;
2012-04-10 18:57:45 +00:00
}
2010-10-22 14:47:40 +00:00
return input . length ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Matches numbers, including decimals, hex, and exponential notation.
// Be careful not to interfere with ranges in progress.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
numberToken ( ) {
2016-10-22 18:51:41 +00:00
var base , lexedLength , match , number , numberValue , tag ;
2012-04-10 18:57:45 +00:00
if ( ! ( match = NUMBER . exec ( this . chunk ) ) ) {
return 0 ;
}
2010-09-25 22:06:14 +00:00
number = match [ 0 ] ;
2015-02-06 09:52:02 +00:00
lexedLength = number . length ;
2016-09-26 14:33:57 +00:00
switch ( false ) {
case ! /^0[BOX]/ . test ( number ) :
2016-11-28 14:05:51 +00:00
this . error ( ` radix prefix in ' ${ number } ' must be lowercase ` , {
2016-09-26 14:33:57 +00:00
offset : 1
} ) ;
break ;
case ! /^(?!0x).*E/ . test ( number ) :
2016-11-28 14:05:51 +00:00
this . error ( ` exponential notation in ' ${ number } ' must be indicated with a lowercase 'e' ` , {
2016-09-26 14:33:57 +00:00
offset : number . indexOf ( 'E' )
} ) ;
break ;
case ! /^0\d*[89]/ . test ( number ) :
2016-11-28 14:05:51 +00:00
this . error ( ` decimal literal ' ${ number } ' must not be prefixed with '0' ` , {
2016-09-26 14:33:57 +00:00
length : lexedLength
} ) ;
break ;
case ! /^0\d+/ . test ( number ) :
2016-11-28 14:05:51 +00:00
this . error ( ` octal literal ' ${ number } ' must be prefixed with '0o' ` , {
2016-09-26 14:33:57 +00:00
length : lexedLength
} ) ;
}
base = ( function ( ) {
switch ( number . charAt ( 1 ) ) {
case 'b' :
return 2 ;
case 'o' :
return 8 ;
case 'x' :
return 16 ;
default :
return null ;
}
} ) ( ) ;
numberValue = base != null ? parseInt ( number . slice ( 2 ) , base ) : parseFloat ( number ) ;
2016-03-05 20:32:20 +00:00
tag = numberValue === 2e308 ? 'INFINITY' : 'NUMBER' ;
Refactor `Literal` into several subtypes
Previously, the parser created `Literal` nodes for many things. This resulted in
information loss. Instead of being able to check the node type, we had to use
regexes to tell the different types of `Literal`s apart. That was a bit like
parsing literals twice: Once in the lexer, and once (or more) in the compiler.
It also caused problems, such as `` `this` `` and `this` being indistinguishable
(fixes #2009).
Instead returning `new Literal` in the grammar, subtypes of it are now returned
instead, such as `NumberLiteral`, `StringLiteral` and `IdentifierLiteral`. `new
Literal` by itself is only used to represent code chunks that fit no category.
(While mentioning `NumberLiteral`, there's also `InfinityLiteral` now, which is
a subtype of `NumberLiteral`.)
`StringWithInterpolations` has been added as a subtype of `Parens`, and
`RegexWithInterpolations` as a subtype of `Call`. This makes it easier for other
programs to make use of CoffeeScript's "AST" (nodes). For example, it is now
possible to distinguish between `"a #{b} c"` and `"a " + b + " c"`. Fixes #4192.
`SuperCall` has been added as a subtype of `Call`.
Note, though, that some information is still lost, especially in the lexer. For
example, there is no way to distinguish a heredoc from a regular string, or a
heregex without interpolations from a regular regex. Binary and octal number
literals are indistinguishable from hexadecimal literals.
After the new subtypes were added, they were taken advantage of, removing most
regexes in nodes.coffee. `SIMPLENUM` (which matches non-hex integers) had to be
kept, though, because such numbers need special handling in JavaScript (for
example in `1..toString()`).
An especially nice hack to get rid of was using `new String()` for the token
value for reserved identifiers (to be able to set a property on them which could
survive through the parser). Now it's a good old regular string.
In range literals, slices, splices and for loop steps when number literals
are involved, CoffeeScript can do some optimizations, such as precomputing the
value of, say, `5 - 3` (outputting `2` instead of `5 - 3` literally). As a side
bonus, this now also works with hexadecimal number literals, such as `0x02`.
Finally, this also improves the output of `coffee --nodes`:
# Before:
$ bin/coffee -ne 'while true
"#{a}"
break'
Block
While
Value
Bool
Block
Value
Parens
Block
Op +
Value """"
Value
Parens
Block
Value "a" "break"
# After:
$ bin/coffee -ne 'while true
"#{a}"
break'
Block
While
Value BooleanLiteral: true
Block
Value
StringWithInterpolations
Block
Op +
Value StringLiteral: ""
Value
Parens
Block
Value IdentifierLiteral: a
StatementLiteral: break
2016-01-31 19:24:31 +00:00
this . token ( tag , number , 0 , lexedLength ) ;
2011-10-24 18:51:41 +00:00
return lexedLength ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Matches strings, including multiline strings, as well as heredocs, with or without
// interpolation.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
stringToken ( ) {
2017-04-09 04:59:09 +00:00
var $ , attempt , delimiter , doc , end , heredoc , i , indent , indentRegex , match , prev , quote , ref , regex , token , tokens ;
2017-04-06 17:06:45 +00:00
[ quote ] = STRING _START . exec ( this . chunk ) || [ ] ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
if ( ! quote ) {
2013-11-27 20:29:45 +00:00
return 0 ;
}
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// If the preceding token is `from` and this is an import or export statement,
// properly tag the `from`.
2017-04-09 04:59:09 +00:00
prev = this . prev ( ) ;
if ( prev && this . value ( ) === 'from' && ( this . seenImport || this . seenExport ) ) {
prev [ 0 ] = 'FROM' ;
Support import and export of ES2015 modules (#4300)
This pull request adds support for ES2015 modules, by recognizing `import` and `export` statements. The following syntaxes are supported, based on the MDN [import](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import) and [export](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/export) pages:
```js
import "module-name"
import defaultMember from "module-name"
import * as name from "module-name"
import { } from "module-name"
import { member } from "module-name"
import { member as alias } from "module-name"
import { member1, member2 as alias2, … } from "module-name"
import defaultMember, * as name from "module-name"
import defaultMember, { … } from "module-name"
export default expression
export class name
export { }
export { name }
export { name as exportedName }
export { name as default }
export { name1, name2 as exportedName2, name3 as default, … }
export * from "module-name"
export { … } from "module-name"
```
As a subsitute for ECMAScript’s `export var name = …` and `export function name {}`, CoffeeScript also supports:
```js
export name = …
```
CoffeeScript also supports optional commas within `{ … }`.
This PR converts the supported `import` and `export` statements into ES2015 `import` and `export` statements; it **does not resolve the modules**. So any CoffeeScript with `import` or `export` statements will be output as ES2015, and will need to be transpiled by another tool such as Babel before it can be used in a browser. We will need to add a warning to the documentation explaining this.
This should be fully backwards-compatible, as `import` and `export` were previously reserved keywords. No flags are used.
There are extensive tests included, though because no current JavaScript runtime supports `import` or `export`, the tests compare strings of what the compiled CoffeeScript output is against what the expected ES2015 should be. I also conducted two more elaborate tests:
* I forked the [ember-piqu](https://github.com/pauc/piqu-ember) project, which was an Ember CLI app that used ember-cli-coffeescript and [ember-cli-coffees6](https://github.com/alexspeller/ember-cli-coffees6) (which adds “support” for `import`/`export` by wrapping such statements in backticks before passing the result to the CoffeeScript compiler). I removed `ember-cli-coffees6` and replaced the CoffeeScript compiler used in the build chain with this code, and the app built without errors. [Demo here.](https://github.com/GeoffreyBooth/coffeescript-modules-test-piqu)
* I also forked the [CoffeeScript version of Meteor’s Todos example app](https://github.com/meteor/todos/tree/coffeescript), and replaced all of its `require` statements with the `import` and `export` statements from the original ES2015 version of the app on its `master` branch. I then updated the `coffeescript` Meteor package in the app to use this new code, and again the app builds without errors. [Demo here.](https://github.com/GeoffreyBooth/coffeescript-modules-test-meteor-todos)
The discussion history for this work started [here](https://github.com/jashkenas/coffeescript/pull/4160) and continued [here](https://github.com/GeoffreyBooth/coffeescript/pull/2). @lydell provided guidance, and @JimPanic and @rattrayalex contributed essential code.
2016-09-14 18:46:05 +00:00
}
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
regex = ( function ( ) {
switch ( quote ) {
case "'" :
return STRING _SINGLE ;
case '"' :
return STRING _DOUBLE ;
case "'''" :
return HEREDOC _SINGLE ;
case '"""' :
return HEREDOC _DOUBLE ;
2014-07-02 15:33:57 +00:00
}
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
} ) ( ) ;
heredoc = quote . length === 3 ;
2017-04-06 17:06:45 +00:00
( {
tokens ,
index : end
} = this . matchWithInterpolations ( regex , quote ) ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
$ = tokens . length - 1 ;
2015-02-22 16:08:15 +00:00
delimiter = quote . charAt ( 0 ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
if ( heredoc ) {
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Find the smallest indentation. It will be removed from all lines later.
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
indent = null ;
doc = ( ( function ( ) {
2015-01-30 19:33:03 +00:00
var j , len , results ;
results = [ ] ;
for ( i = j = 0 , len = tokens . length ; j < len ; i = ++ j ) {
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
token = tokens [ i ] ;
if ( token [ 0 ] === 'NEOSTRING' ) {
2015-01-30 19:33:03 +00:00
results . push ( token [ 1 ] ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
}
}
2015-01-30 19:33:03 +00:00
return results ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
} ) ( ) ) . join ( '#{}' ) ;
while ( match = HEREDOC _INDENT . exec ( doc ) ) {
attempt = match [ 1 ] ;
2017-04-06 17:06:45 +00:00
if ( indent === null || ( 0 < ( ref = attempt . length ) && ref < indent . length ) ) {
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
indent = attempt ;
}
}
if ( indent ) {
2016-11-28 14:05:51 +00:00
indentRegex = RegExp ( ` \\ n ${ indent } ` , "g" ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
}
2017-04-06 17:06:45 +00:00
this . mergeInterpolationTokens ( tokens , { delimiter } , ( value , i ) => {
2017-04-20 19:41:28 +00:00
value = this . formatString ( value , {
delimiter : quote
} ) ;
[CS2] Output ES2015 arrow functions, default parameters, rest parameters (#4311)
* Eliminate wrapper around “bound” (arrow) functions; output `=>` for such functions
* Remove irrelevant (and breaking) tests
* Minor cleanup
* When a function parameter is a splat (i.e., it uses the ES2015 rest parameter syntax) output that parameter as ES2015
* Rearrange function parameters when one of the parameters is a splat and isn’t the last parameter (very WIP)
* Handle params like `@param`, adding assignment expressions for them when they appear; ensure splat parameter is last
* Add parameter names (not a text like `'\nValue IdentifierLiteral: a'`) to the scope, so that parameters can’t be deleted; move body-related lines together; more explanation of what’s going on
* For parameters with a default value, correctly add the parameter name to the function scope
* Handle expansions in function parameters: when an expansion is found, set the parameters to only be the original parameters left of the expansion, then an `...args` parameter; and in the function body define variables for the parameters to the right of the expansion, including setting default values
* Handle splat parameters the same way we handle expansions: if a splat parameter is found, it becomes the last parameter in the function definition, and all following parameters get declared in the function body. Fix the splat/rest parameter values after the post-splat parameters have been extracted from it. Clean up `Code.compileNode` so that we loop through the parameters only once, and we create all expressions using calls like `new IdentifierLiteral` rather than `@makeCode`.
* Fix parameter name when a parameter is a splat attached to `this` (e.g. `@param...`)
* Rather than assigning post-splat parameters based on index, use slice; passes test “Functions with splats being called with too few arguments”
* Dial back our w00t indentation
* Better parsing of parameter names (WIP)
* Refactor processing of splat/expansion parameters
* Fix assignment of default parameters for parameters that come after a splat
* Better check for whether a param is attached to `this`
* More understandable variable names
* For parameters after a splat or expansion, assign them similar to the 1.x destructuring method of using `arguments`, except only concern ourselves with the post-splat parameters instead of all parameters; and use the splat/expansion parameter name, since `arguments` in ES fat arrow functions refers to the parent function’s `arguments` rather than the fat arrow function’s arguments/parameters
* Don’t add unnamed parameters (like `[]` as a parameter) to the function scope
* Disallow multiple splat/expansion parameters in function definitions; disallow lone expansion parameters
* Fix `this` params not getting assigned if the parameter is after a splat parameter
* Allow names of function parameters attached to `this` to be reserved words
* Always add a statement to the function body defining a variable with its default value, if it has one, if the variable `== null`; this covers the case when ES doesn’t apply the default value when `null` is passed in as a value, but CoffeeScript expects `null` and `undefined` to act interchangeably
* Aftermath of having both `undefined` and `null` trigger the use of default values for parameters with default values
* More careful parsing of destructured parameters
* Fall back to processing destructured parameters in the function body, to account for `this` or default values within destructured objects
* Clean up comments
* Restore new bare function test, minus the arrow function part of it
* Test that bound/arrow functions aren’t overwriting the `arguments` object, which should refer to the parent scope’s `arguments` (like `this`)
* Follow ES2015 spec for parameter default values: `null` gets assigned as as `null`, not the default value
* Mimic ES default parameters behavior for parameters after a splat or expansion parameter
* Bound functions cannot be generators: remove no-longer-relevant test, add check to throw error if `yield` appears inside a bound (arrow) function
* Error for bound generator functions should underline the `yield`
2016-10-26 05:26:13 +00:00
if ( indentRegex ) {
value = value . replace ( indentRegex , '\n' ) ;
}
if ( i === 0 ) {
value = value . replace ( LEADING _BLANK _LINE , '' ) ;
}
if ( i === $ ) {
value = value . replace ( TRAILING _BLANK _LINE , '' ) ;
}
return value ;
} ) ;
2010-10-02 22:45:23 +00:00
} else {
2017-04-06 17:06:45 +00:00
this . mergeInterpolationTokens ( tokens , { delimiter } , ( value , i ) => {
2017-04-20 19:41:28 +00:00
value = this . formatString ( value , {
delimiter : quote
} ) ;
[CS2] Output ES2015 arrow functions, default parameters, rest parameters (#4311)
* Eliminate wrapper around “bound” (arrow) functions; output `=>` for such functions
* Remove irrelevant (and breaking) tests
* Minor cleanup
* When a function parameter is a splat (i.e., it uses the ES2015 rest parameter syntax) output that parameter as ES2015
* Rearrange function parameters when one of the parameters is a splat and isn’t the last parameter (very WIP)
* Handle params like `@param`, adding assignment expressions for them when they appear; ensure splat parameter is last
* Add parameter names (not a text like `'\nValue IdentifierLiteral: a'`) to the scope, so that parameters can’t be deleted; move body-related lines together; more explanation of what’s going on
* For parameters with a default value, correctly add the parameter name to the function scope
* Handle expansions in function parameters: when an expansion is found, set the parameters to only be the original parameters left of the expansion, then an `...args` parameter; and in the function body define variables for the parameters to the right of the expansion, including setting default values
* Handle splat parameters the same way we handle expansions: if a splat parameter is found, it becomes the last parameter in the function definition, and all following parameters get declared in the function body. Fix the splat/rest parameter values after the post-splat parameters have been extracted from it. Clean up `Code.compileNode` so that we loop through the parameters only once, and we create all expressions using calls like `new IdentifierLiteral` rather than `@makeCode`.
* Fix parameter name when a parameter is a splat attached to `this` (e.g. `@param...`)
* Rather than assigning post-splat parameters based on index, use slice; passes test “Functions with splats being called with too few arguments”
* Dial back our w00t indentation
* Better parsing of parameter names (WIP)
* Refactor processing of splat/expansion parameters
* Fix assignment of default parameters for parameters that come after a splat
* Better check for whether a param is attached to `this`
* More understandable variable names
* For parameters after a splat or expansion, assign them similar to the 1.x destructuring method of using `arguments`, except only concern ourselves with the post-splat parameters instead of all parameters; and use the splat/expansion parameter name, since `arguments` in ES fat arrow functions refers to the parent function’s `arguments` rather than the fat arrow function’s arguments/parameters
* Don’t add unnamed parameters (like `[]` as a parameter) to the function scope
* Disallow multiple splat/expansion parameters in function definitions; disallow lone expansion parameters
* Fix `this` params not getting assigned if the parameter is after a splat parameter
* Allow names of function parameters attached to `this` to be reserved words
* Always add a statement to the function body defining a variable with its default value, if it has one, if the variable `== null`; this covers the case when ES doesn’t apply the default value when `null` is passed in as a value, but CoffeeScript expects `null` and `undefined` to act interchangeably
* Aftermath of having both `undefined` and `null` trigger the use of default values for parameters with default values
* More careful parsing of destructured parameters
* Fall back to processing destructured parameters in the function body, to account for `this` or default values within destructured objects
* Clean up comments
* Restore new bare function test, minus the arrow function part of it
* Test that bound/arrow functions aren’t overwriting the `arguments` object, which should refer to the parent scope’s `arguments` (like `this`)
* Follow ES2015 spec for parameter default values: `null` gets assigned as as `null`, not the default value
* Mimic ES default parameters behavior for parameters after a splat or expansion parameter
* Bound functions cannot be generators: remove no-longer-relevant test, add check to throw error if `yield` appears inside a bound (arrow) function
* Error for bound generator functions should underline the `yield`
2016-10-26 05:26:13 +00:00
value = value . replace ( SIMPLE _STRING _OMIT , function ( match , offset ) {
if ( ( i === 0 && offset === 0 ) || ( i === $ && offset + match . length === value . length ) ) {
return '' ;
} else {
return ' ' ;
}
} ) ;
return value ;
} ) ;
2010-10-02 22:45:23 +00:00
}
2017-06-07 06:33:46 +00:00
if ( this . atCSXTag ( ) ) {
this . token ( ',' , ',' , 0 , 0 , this . prev ) ;
}
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
return end ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Matches and consumes comments. The comments are taken out of the token
// stream and saved for later, to be reinserted into the output after
// everything has been parsed and the JavaScript code generated.
commentToken ( chunk = this . chunk ) {
var comment , commentAttachments , content , contents , here , i , match , matchIllegal , newLine , placeholderToken , prev ;
if ( ! ( match = chunk . match ( COMMENT ) ) ) {
2012-04-10 18:57:45 +00:00
return 0 ;
}
2017-04-06 17:06:45 +00:00
[ comment , here ] = match ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
contents = null ;
// Does this comment follow code on the same line?
newLine = /^\s*\n+\s*#/ . test ( comment ) ;
2010-09-23 05:14:18 +00:00
if ( here ) {
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
matchIllegal = HERECOMMENT _ILLEGAL . exec ( comment ) ;
if ( matchIllegal ) {
this . error ( ` block comments cannot contain ${ matchIllegal [ 0 ] } ` , {
offset : matchIllegal . index ,
length : matchIllegal [ 0 ] . length
2015-02-06 09:52:02 +00:00
} ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
}
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Parse indentation or outdentation as if this block comment didn’ t exist.
chunk = chunk . replace ( ` ### ${ here } ### ` , '' ) ;
// Remove leading newlines, like `Rewriter::removeLeadingNewlines`, to
// avoid the creation of unwanted `TERMINATOR` tokens.
chunk = chunk . replace ( /^\n+/ , '' ) ;
this . lineToken ( chunk ) ;
// Pull out the ###-style comment’ s content, and format it.
content = here ;
if ( indexOf . call ( content , '\n' ) >= 0 ) {
content = content . replace ( RegExp ( ` \\ n ${ repeat ( ' ' , this . indent ) } ` , "g" ) , '\n' ) ;
}
contents = [ content ] ;
} else {
// The `COMMENT` regex captures successive line comments as one token.
// Remove any leading newlines before the first comment, but preserve
// blank lines between line comments.
content = comment . replace ( /^(\n*)/ , '' ) ;
content = content . replace ( /^([ |\t]*)#/gm , '' ) ;
contents = content . split ( '\n' ) ;
}
commentAttachments = ( function ( ) {
var j , len , results ;
results = [ ] ;
for ( i = j = 0 , len = contents . length ; j < len ; i = ++ j ) {
content = contents [ i ] ;
results . push ( {
content : content ,
here : here != null ,
newLine : newLine || i !== 0 // Line comments after the first one start new lines, by definition.
} ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
}
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
return results ;
} ) ( ) ;
prev = this . prev ( ) ;
if ( ! prev ) {
// If there’ s no previous token, create a placeholder token to attach
// this comment to; and follow with a newline.
commentAttachments [ 0 ] . newLine = true ;
this . lineToken ( this . chunk . slice ( comment . length ) ) ;
placeholderToken = this . makeToken ( 'JS' , '' ) ;
placeholderToken . generated = true ;
placeholderToken . comments = commentAttachments ;
this . tokens . push ( placeholderToken ) ;
this . newlineToken ( 0 ) ;
} else {
attachCommentsToNode ( commentAttachments , prev ) ;
2010-07-02 01:26:33 +00:00
}
2010-10-22 14:47:40 +00:00
return comment . length ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Matches JavaScript interpolated directly into the source via backticks.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
jsToken ( ) {
2010-09-25 22:06:14 +00:00
var match , script ;
2016-11-19 19:13:30 +00:00
if ( ! ( this . chunk . charAt ( 0 ) === '`' && ( match = HERE _JSTOKEN . exec ( this . chunk ) || JSTOKEN . exec ( this . chunk ) ) ) ) {
2011-08-08 16:55:22 +00:00
return 0 ;
}
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Convert escaped backticks to backticks, and escaped backslashes
// just before escaped backticks to backslashes
2016-11-19 19:13:30 +00:00
script = match [ 1 ] . replace ( /\\+(`|$)/g , function ( string ) {
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// `string` is always a value like '\`', '\\\`', '\\\\\`', etc.
// By reducing it to its latter half, we turn '\`' to '`', '\\\`' to '\`', etc.
2016-11-19 19:13:30 +00:00
return string . slice ( - Math . ceil ( string . length / 2 ) ) ;
} ) ;
this . token ( 'JS' , script , 0 , match [ 0 ] . length ) ;
return match [ 0 ] . length ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Matches regular expression literals, as well as multiline extended ones.
// Lexing regular expressions is difficult to distinguish from division, so we
// borrow some basic heuristics from JavaScript and Ruby.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
regexToken ( ) {
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
var body , closed , comment , comments , end , flags , index , j , len , match , origin , prev , ref , ref1 , regex , tokens ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
switch ( false ) {
case ! ( match = REGEX _ILLEGAL . exec ( this . chunk ) ) :
2016-11-28 14:05:51 +00:00
this . error ( ` regular expressions cannot begin with ${ match [ 2 ] } ` , {
2015-02-06 09:52:02 +00:00
offset : match . index + match [ 1 ] . length
} ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
break ;
2015-02-03 17:55:38 +00:00
case ! ( match = this . matchWithInterpolations ( HEREGEX , '///' ) ) :
2017-04-06 17:06:45 +00:00
( { tokens , index } = match ) ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
comments = this . chunk . slice ( 0 , index ) . match ( /\s+(#(?!{).*)/g ) ;
if ( comments ) {
for ( j = 0 , len = comments . length ; j < len ; j ++ ) {
comment = comments [ j ] ;
this . commentToken ( comment ) ;
}
}
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
break ;
case ! ( match = REGEX . exec ( this . chunk ) ) :
2017-04-06 17:06:45 +00:00
[ regex , body , closed ] = match ;
2015-02-12 18:26:41 +00:00
this . validateEscapes ( body , {
isRegex : true ,
offsetInChunk : 1
2015-02-05 16:23:03 +00:00
} ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
index = regex . length ;
2017-04-09 04:59:09 +00:00
prev = this . prev ( ) ;
2015-01-10 00:48:00 +00:00
if ( prev ) {
2017-04-09 04:59:09 +00:00
if ( prev . spaced && ( ref = prev [ 0 ] , indexOf . call ( CALLABLE , ref ) >= 0 ) ) {
2015-01-10 00:48:00 +00:00
if ( ! closed || POSSIBLY _DIVISION . test ( regex ) ) {
return 0 ;
}
2018-01-31 14:33:17 +00:00
} else if ( ref1 = prev [ 0 ] , indexOf . call ( NOT _REGEX , ref1 ) >= 0 ) {
return 0 ;
2015-01-10 00:48:00 +00:00
}
}
if ( ! closed ) {
this . error ( 'missing / (unclosed regex)' ) ;
2012-04-10 18:57:45 +00:00
}
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
break ;
default :
return 0 ;
}
2017-04-06 17:06:45 +00:00
[ flags ] = REGEX _FLAGS . exec ( this . chunk . slice ( index ) ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
end = index + flags . length ;
Fix #3597: Allow interpolations in object keys
The following is now allowed:
o =
a: 1
b: 2
"#{'c'}": 3
"#{'d'}": 4
e: 5
"#{'f'}": 6
g: 7
It compiles to:
o = (
obj = {
a: 1,
b: 2
},
obj["" + 'c'] = 3,
obj["" + 'd'] = 4,
obj.e = 5,
obj["" + 'f'] = 6,
obj.g = 7,
obj
);
- Closes #3039. Empty interpolations in object keys are now _supposed_ to be
allowed.
- Closes #1131. No need to improve error messages for attempted key
interpolation anymore.
- Implementing this required fixing the following bug: `("" + a): 1` used to
error out on the colon, saying "unexpected colon". But really, it is the
attempted object key that is unexpected. Now the error is on the opening
parenthesis instead.
- However, the above fix broke some error message tests for regexes. The easiest
way to fix this was to make a seemingly unrelated change: The error messages
for unexpected identifiers, numbers, strings and regexes now say for example
'unexpected string' instead of 'unexpected """some #{really long} string"""'.
In other words, the tag _name_ is used instead of the tag _value_.
This was way easier to implement, and is more helpful to the user. Using the
tag value is good for operators, reserved words and the like, but not for
tokens which can contain any text. For example, 'unexpected identifier' is
better than 'unexpected expected' (if a variable called 'expected' was used
erraneously).
- While writing tests for the above point I found a few minor bugs with string
locations which have been fixed.
2015-02-07 19:16:59 +00:00
origin = this . makeToken ( 'REGEX' , null , 0 , end ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
switch ( false ) {
case ! ! VALID _FLAGS . test ( flags ) :
2016-11-28 14:05:51 +00:00
this . error ( ` invalid regular expression flags ${ flags } ` , {
2015-02-06 09:52:02 +00:00
offset : index ,
length : flags . length
} ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
break ;
2015-02-05 16:23:03 +00:00
case ! ( regex || tokens . length === 1 ) :
2017-04-25 17:15:08 +00:00
if ( body ) {
body = this . formatRegex ( body , {
flags ,
delimiter : '/'
} ) ;
} else {
body = this . formatHeregex ( tokens [ 0 ] [ 1 ] , { flags } ) ;
2015-02-05 16:23:03 +00:00
}
2016-11-28 14:05:51 +00:00
this . token ( 'REGEX' , ` ${ this . makeDelimitedLiteral ( body , {
2015-02-05 16:23:03 +00:00
delimiter : '/'
2016-11-28 14:05:51 +00:00
} ) } $ { flags } ` , 0, end, origin);
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
break ;
default :
Fix #3597: Allow interpolations in object keys
The following is now allowed:
o =
a: 1
b: 2
"#{'c'}": 3
"#{'d'}": 4
e: 5
"#{'f'}": 6
g: 7
It compiles to:
o = (
obj = {
a: 1,
b: 2
},
obj["" + 'c'] = 3,
obj["" + 'd'] = 4,
obj.e = 5,
obj["" + 'f'] = 6,
obj.g = 7,
obj
);
- Closes #3039. Empty interpolations in object keys are now _supposed_ to be
allowed.
- Closes #1131. No need to improve error messages for attempted key
interpolation anymore.
- Implementing this required fixing the following bug: `("" + a): 1` used to
error out on the colon, saying "unexpected colon". But really, it is the
attempted object key that is unexpected. Now the error is on the opening
parenthesis instead.
- However, the above fix broke some error message tests for regexes. The easiest
way to fix this was to make a seemingly unrelated change: The error messages
for unexpected identifiers, numbers, strings and regexes now say for example
'unexpected string' instead of 'unexpected """some #{really long} string"""'.
In other words, the tag _name_ is used instead of the tag _value_.
This was way easier to implement, and is more helpful to the user. Using the
tag value is good for operators, reserved words and the like, but not for
tokens which can contain any text. For example, 'unexpected identifier' is
better than 'unexpected expected' (if a variable called 'expected' was used
erraneously).
- While writing tests for the above point I found a few minor bugs with string
locations which have been fixed.
2015-02-07 19:16:59 +00:00
this . token ( 'REGEX_START' , '(' , 0 , 0 , origin ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
this . token ( 'IDENTIFIER' , 'RegExp' , 0 , 0 ) ;
Fix #3597: Allow interpolations in object keys
The following is now allowed:
o =
a: 1
b: 2
"#{'c'}": 3
"#{'d'}": 4
e: 5
"#{'f'}": 6
g: 7
It compiles to:
o = (
obj = {
a: 1,
b: 2
},
obj["" + 'c'] = 3,
obj["" + 'd'] = 4,
obj.e = 5,
obj["" + 'f'] = 6,
obj.g = 7,
obj
);
- Closes #3039. Empty interpolations in object keys are now _supposed_ to be
allowed.
- Closes #1131. No need to improve error messages for attempted key
interpolation anymore.
- Implementing this required fixing the following bug: `("" + a): 1` used to
error out on the colon, saying "unexpected colon". But really, it is the
attempted object key that is unexpected. Now the error is on the opening
parenthesis instead.
- However, the above fix broke some error message tests for regexes. The easiest
way to fix this was to make a seemingly unrelated change: The error messages
for unexpected identifiers, numbers, strings and regexes now say for example
'unexpected string' instead of 'unexpected """some #{really long} string"""'.
In other words, the tag _name_ is used instead of the tag _value_.
This was way easier to implement, and is more helpful to the user. Using the
tag value is good for operators, reserved words and the like, but not for
tokens which can contain any text. For example, 'unexpected identifier' is
better than 'unexpected expected' (if a variable called 'expected' was used
erraneously).
- While writing tests for the above point I found a few minor bugs with string
locations which have been fixed.
2015-02-07 19:16:59 +00:00
this . token ( 'CALL_START' , '(' , 0 , 0 ) ;
2015-02-05 16:23:03 +00:00
this . mergeInterpolationTokens ( tokens , {
delimiter : '"' ,
double : true
2017-04-25 17:15:08 +00:00
} , ( str ) => {
return this . formatHeregex ( str , { flags } ) ;
} ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
if ( flags ) {
2017-02-18 23:14:47 +00:00
this . token ( ',' , ',' , index - 1 , 0 ) ;
this . token ( 'STRING' , '"' + flags + '"' , index - 1 , flags . length ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
}
2017-02-18 23:14:47 +00:00
this . token ( ')' , ')' , end - 1 , 0 ) ;
this . token ( 'REGEX_END' , ')' , end - 1 , 0 ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
}
return end ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Matches newlines, indents, and outdents, and determines which is which.
// If we can detect that the current line is continued onto the next line,
// then the newline is suppressed:
// elements
// .each( ... )
// .map( ... )
// Keeps track of the level of indentation, because a single outdent token
// can close multiple indents, so we need to know how far in we happen to be.
lineToken ( chunk = this . chunk ) {
2018-01-28 10:38:39 +00:00
var backslash , diff , indent , match , minLiteralLength , newIndentLiteral , noNewlines , prev , size ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
if ( ! ( match = MULTI _DENT . exec ( chunk ) ) ) {
2012-04-10 18:57:45 +00:00
return 0 ;
}
2010-09-25 22:06:14 +00:00
indent = match [ 0 ] ;
2018-01-28 10:38:39 +00:00
prev = this . prev ( ) ;
2018-02-09 05:11:11 +00:00
backslash = ( prev != null ? prev [ 0 ] : void 0 ) === '\\' ;
2018-01-28 10:38:39 +00:00
if ( ! ( backslash && this . seenFor ) ) {
2018-02-09 05:11:11 +00:00
// console.log backslash, @seenExport
2018-01-28 10:38:39 +00:00
this . seenFor = false ;
}
2018-02-09 05:11:11 +00:00
if ( ! ( ( backslash && this . seenImport ) || this . importSpecifierList ) ) {
2017-04-09 09:53:43 +00:00
this . seenImport = false ;
}
2018-02-09 05:11:11 +00:00
if ( ! ( ( backslash && this . seenExport ) || this . exportSpecifierList ) ) {
2017-04-09 09:53:43 +00:00
this . seenExport = false ;
}
2010-09-23 05:14:18 +00:00
size = indent . length - 1 - indent . lastIndexOf ( '\n' ) ;
2010-10-25 18:56:02 +00:00
noNewlines = this . unfinished ( ) ;
2016-10-22 18:51:41 +00:00
newIndentLiteral = size > 0 ? indent . slice ( - size ) : '' ;
if ( ! /^(.?)\1*$/ . exec ( newIndentLiteral ) ) {
this . error ( 'mixed indentation' , {
offset : indent . length
} ) ;
return indent . length ;
}
minLiteralLength = Math . min ( newIndentLiteral . length , this . indentLiteral . length ) ;
if ( newIndentLiteral . slice ( 0 , minLiteralLength ) !== this . indentLiteral . slice ( 0 , minLiteralLength ) ) {
this . error ( 'indentation mismatch' , {
offset : indent . length
} ) ;
return indent . length ;
}
2010-09-09 02:46:13 +00:00
if ( size - this . indebt === this . indent ) {
2010-06-12 23:05:13 +00:00
if ( noNewlines ) {
2010-10-22 14:47:40 +00:00
this . suppressNewlines ( ) ;
} else {
2012-11-17 00:09:56 +00:00
this . newlineToken ( 0 ) ;
2010-02-28 17:49:37 +00:00
}
2010-10-22 14:47:40 +00:00
return indent . length ;
}
if ( size > this . indent ) {
2017-07-09 17:28:50 +00:00
if ( noNewlines ) {
2010-09-09 02:46:13 +00:00
this . indebt = size - this . indent ;
2010-10-22 14:47:40 +00:00
this . suppressNewlines ( ) ;
return indent . length ;
2010-02-28 17:49:37 +00:00
}
2013-06-13 22:21:47 +00:00
if ( ! this . tokens . length ) {
this . baseIndent = this . indent = size ;
2016-10-22 18:51:41 +00:00
this . indentLiteral = newIndentLiteral ;
2013-06-13 22:21:47 +00:00
return indent . length ;
}
2010-08-21 16:13:43 +00:00
diff = size - this . indent + this . outdebt ;
2013-03-04 23:03:08 +00:00
this . token ( 'INDENT' , diff , indent . length - size , size ) ;
2010-02-28 00:40:53 +00:00
this . indents . push ( diff ) ;
2015-01-03 23:28:23 +00:00
this . ends . push ( {
tag : 'OUTDENT'
} ) ;
2010-10-20 17:29:06 +00:00
this . outdebt = this . indebt = 0 ;
2014-01-22 18:30:13 +00:00
this . indent = size ;
2016-10-22 18:51:41 +00:00
this . indentLiteral = newIndentLiteral ;
2018-01-31 14:33:17 +00:00
} else if ( size < this . baseIndent ) {
this . error ( 'missing indentation' , {
offset : indent . length
} ) ;
2010-02-28 00:40:53 +00:00
} else {
2018-01-31 14:33:17 +00:00
this . indebt = 0 ;
this . outdentToken ( this . indent - size , noNewlines , indent . length ) ;
2010-02-28 00:40:53 +00:00
}
2010-10-22 14:47:40 +00:00
return indent . length ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Record an outdent token or multiple tokens, if we happen to be moving back
// inwards past several recorded indents. Sets new @indent value.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
outdentToken ( moveOut , noNewlines , outdentLength ) {
2017-04-06 17:06:45 +00:00
var decreasedIndent , dent , lastIndent , ref ;
2014-01-22 18:30:13 +00:00
decreasedIndent = this . indent - moveOut ;
2010-07-31 02:06:22 +00:00
while ( moveOut > 0 ) {
2014-01-22 18:30:13 +00:00
lastIndent = this . indents [ this . indents . length - 1 ] ;
if ( ! lastIndent ) {
2017-06-30 01:39:05 +00:00
this . outdebt = moveOut = 0 ;
2018-01-31 14:33:17 +00:00
} else if ( this . outdebt && moveOut <= this . outdebt ) {
this . outdebt -= moveOut ;
moveOut = 0 ;
2010-07-31 02:06:22 +00:00
} else {
2018-01-31 14:33:17 +00:00
dent = this . indents . pop ( ) + this . outdebt ;
if ( outdentLength && ( ref = this . chunk [ outdentLength ] , indexOf . call ( INDENTABLE _CLOSERS , ref ) >= 0 ) ) {
decreasedIndent -= dent - moveOut ;
moveOut = dent ;
2014-01-22 18:30:13 +00:00
}
2018-01-31 14:33:17 +00:00
this . outdebt = 0 ;
// pair might call outdentToken, so preserve decreasedIndent
this . pair ( 'OUTDENT' ) ;
this . token ( 'OUTDENT' , moveOut , 0 , outdentLength ) ;
moveOut -= dent ;
2010-06-02 03:32:46 +00:00
}
}
2012-04-10 18:57:45 +00:00
if ( dent ) {
this . outdebt -= moveOut ;
}
2017-09-01 14:09:36 +00:00
this . suppressSemicolons ( ) ;
2011-08-08 16:55:22 +00:00
if ( ! ( this . tag ( ) === 'TERMINATOR' || noNewlines ) ) {
2012-11-17 00:09:56 +00:00
this . token ( 'TERMINATOR' , '\n' , outdentLength , 0 ) ;
2011-08-08 16:55:22 +00:00
}
2014-01-23 20:52:26 +00:00
this . indent = decreasedIndent ;
2016-10-22 18:51:41 +00:00
this . indentLiteral = this . indentLiteral . slice ( 0 , decreasedIndent ) ;
2010-10-22 14:47:40 +00:00
return this ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Matches and consumes non-meaningful whitespace. Tag the previous token
// as being “spaced”, because there are some cases where it makes a difference.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
whitespaceToken ( ) {
2017-04-09 04:59:09 +00:00
var match , nline , prev ;
2011-08-08 16:55:22 +00:00
if ( ! ( ( match = WHITESPACE . exec ( this . chunk ) ) || ( nline = this . chunk . charAt ( 0 ) === '\n' ) ) ) {
return 0 ;
}
2017-04-09 04:59:09 +00:00
prev = this . prev ( ) ;
2012-04-10 18:57:45 +00:00
if ( prev ) {
prev [ match ? 'spaced' : 'newLine' ] = true ;
}
2010-11-09 04:07:51 +00:00
if ( match ) {
return match [ 0 ] . length ;
} else {
return 0 ;
}
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Generate a newline token. Consecutive newlines get merged together.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
newlineToken ( offset ) {
2017-09-01 14:09:36 +00:00
this . suppressSemicolons ( ) ;
2012-04-10 18:57:45 +00:00
if ( this . tag ( ) !== 'TERMINATOR' ) {
2012-11-17 00:09:56 +00:00
this . token ( 'TERMINATOR' , '\n' , offset , 0 ) ;
2012-04-10 18:57:45 +00:00
}
2010-10-22 14:47:40 +00:00
return this ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Use a `\` at a line-ending to suppress the newline.
// The slash is removed here once its job is done.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
suppressNewlines ( ) {
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
var prev ;
prev = this . prev ( ) ;
if ( prev [ 1 ] === '\\' ) {
if ( prev . comments && this . tokens . length > 1 ) {
// `@tokens.length` should be at least 2 (some code, then `\`).
// If something puts a `\` after nothing, they deserve to lose any
// comments that trail it.
attachCommentsToNode ( prev . comments , this . tokens [ this . tokens . length - 2 ] ) ;
}
2012-04-10 18:57:45 +00:00
this . tokens . pop ( ) ;
}
2010-10-22 14:47:40 +00:00
return this ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// CSX is like JSX but for CoffeeScript.
2017-06-07 06:33:46 +00:00
csxToken ( ) {
2017-08-03 04:00:39 +00:00
var afterTag , colon , csxTag , end , firstChar , id , input , match , origin , prev , prevChar , ref , token , tokens ;
2017-06-07 06:33:46 +00:00
firstChar = this . chunk [ 0 ] ;
2017-08-03 04:00:39 +00:00
// Check the previous token to detect if attribute is spread.
prevChar = this . tokens . length > 0 ? this . tokens [ this . tokens . length - 1 ] [ 0 ] : '' ;
2017-06-07 06:33:46 +00:00
if ( firstChar === '<' ) {
2017-12-09 06:41:03 +00:00
match = CSX _IDENTIFIER . exec ( this . chunk . slice ( 1 ) ) || CSX _FRAGMENT _IDENTIFIER . exec ( this . chunk . slice ( 1 ) ) ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Not the right hand side of an unspaced comparison (i.e. `a<b`).
2017-06-07 06:33:46 +00:00
if ( ! ( match && ( this . csxDepth > 0 || ! ( prev = this . prev ( ) ) || prev . spaced || ( ref = prev [ 0 ] , indexOf . call ( COMPARABLE _LEFT _SIDE , ref ) < 0 ) ) ) ) {
return 0 ;
}
[ input , id , colon ] = match ;
origin = this . token ( 'CSX_TAG' , id , 1 , id . length ) ;
this . token ( 'CALL_START' , '(' ) ;
2017-08-03 04:00:39 +00:00
this . token ( '[' , '[' ) ;
2017-06-07 06:33:46 +00:00
this . ends . push ( {
tag : '/>' ,
origin : origin ,
name : id
} ) ;
this . csxDepth ++ ;
return id . length + 1 ;
2018-01-31 14:33:17 +00:00
} else if ( csxTag = this . atCSXTag ( ) ) {
if ( this . chunk . slice ( 0 , 2 ) === '/>' ) {
this . pair ( '/>' ) ;
this . token ( ']' , ']' , 0 , 2 ) ;
this . token ( 'CALL_END' , ')' , 0 , 2 ) ;
this . csxDepth -- ;
return 2 ;
} else if ( firstChar === '{' ) {
if ( prevChar === ':' ) {
token = this . token ( '(' , '(' ) ;
this . csxObjAttribute [ this . csxDepth ] = false ;
2017-08-03 04:00:39 +00:00
} else {
2018-01-31 14:33:17 +00:00
token = this . token ( '{' , '{' ) ;
this . csxObjAttribute [ this . csxDepth ] = true ;
2017-06-07 06:33:46 +00:00
}
2018-01-31 14:33:17 +00:00
this . ends . push ( {
tag : '}' ,
origin : token
} ) ;
return 1 ;
} else if ( firstChar === '>' ) {
// Ignore terminators inside a tag.
this . pair ( '/>' ) ; // As if the current tag was self-closing.
origin = this . token ( ']' , ']' ) ;
this . token ( ',' , ',' ) ;
( {
tokens ,
index : end
} = this . matchWithInterpolations ( INSIDE _CSX , '>' , '</' , CSX _INTERPOLATION ) ) ;
this . mergeInterpolationTokens ( tokens , {
delimiter : '"'
} , ( value , i ) => {
return this . formatString ( value , {
delimiter : '>'
} ) ;
} ) ;
match = CSX _IDENTIFIER . exec ( this . chunk . slice ( end ) ) || CSX _FRAGMENT _IDENTIFIER . exec ( this . chunk . slice ( end ) ) ;
if ( ! match || match [ 1 ] !== csxTag . name ) {
this . error ( ` expected corresponding CSX closing tag for ${ csxTag . name } ` , csxTag . origin [ 2 ] ) ;
}
afterTag = end + csxTag . name . length ;
if ( this . chunk [ afterTag ] !== '>' ) {
this . error ( "missing closing > after tag name" , {
offset : afterTag ,
length : 1
} ) ;
}
// +1 for the closing `>`.
this . token ( 'CALL_END' , ')' , end , csxTag . name . length + 1 ) ;
this . csxDepth -- ;
return afterTag + 1 ;
2017-06-07 06:33:46 +00:00
} else {
2018-01-31 14:33:17 +00:00
return 0 ;
}
} else if ( this . atCSXTag ( 1 ) ) {
if ( firstChar === '}' ) {
this . pair ( firstChar ) ;
if ( this . csxObjAttribute [ this . csxDepth ] ) {
this . token ( '}' , '}' ) ;
this . csxObjAttribute [ this . csxDepth ] = false ;
2017-08-03 04:00:39 +00:00
} else {
2018-01-31 14:33:17 +00:00
this . token ( ')' , ')' ) ;
2017-08-03 04:00:39 +00:00
}
2018-01-31 14:33:17 +00:00
this . token ( ',' , ',' ) ;
return 1 ;
} else {
return 0 ;
2017-06-07 06:33:46 +00:00
}
2018-01-31 14:33:17 +00:00
} else {
return 0 ;
2017-06-07 06:33:46 +00:00
}
}
atCSXTag ( depth = 0 ) {
var i , last , ref ;
if ( this . csxDepth === 0 ) {
return false ;
}
i = this . ends . length - 1 ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
while ( ( ( ref = this . ends [ i ] ) != null ? ref . tag : void 0 ) === 'OUTDENT' || depth -- > 0 ) { // Ignore indents.
2017-06-07 06:33:46 +00:00
i -- ;
}
last = this . ends [ i ] ;
return ( last != null ? last . tag : void 0 ) === '/>' && last ;
}
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// We treat all other single characters as a token. E.g.: `( ) , . !`
// Multi-character operators are also literal tokens, so that Jison can assign
// the proper order of operations. There are some symbols that we tag specially
// here. `;` and newlines are both treated as a `TERMINATOR`, we distinguish
// parentheses that indicate a method call from regular parentheses, and so on.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
literalToken ( ) {
2017-09-01 14:09:36 +00:00
var match , message , origin , prev , ref , ref1 , ref2 , ref3 , ref4 , skipToken , tag , token , value ;
2010-10-05 14:31:48 +00:00
if ( match = OPERATOR . exec ( this . chunk ) ) {
2017-04-06 17:06:45 +00:00
[ value ] = match ;
2012-04-10 18:57:45 +00:00
if ( CODE . test ( value ) ) {
this . tagParameters ( ) ;
}
2010-09-23 05:14:18 +00:00
} else {
value = this . chunk . charAt ( 0 ) ;
2010-02-28 00:40:53 +00:00
}
tag = value ;
2017-04-09 04:59:09 +00:00
prev = this . prev ( ) ;
Compile splats in arrays and function calls to ES2015 splats (#4353)
Rather than compiling splats to arrays built using `Array#concat`, splats
are now compiled directly to ES2015 splats, e.g.
f foo, arguments..., bar
[ foo, arguments..., bar ]
Which used to be compiled to:
f.apply(null, [foo].concat(slice.call(arguments), [bar]));
[foo].concat(slice.call(arguments), [bar]);
Is now compiled to:
f(foo, ...arguments, bar);
[ foo, ...arguments, bar ];
2016-11-06 16:30:04 +00:00
if ( prev && indexOf . call ( [ '=' , ... COMPOUND _ASSIGN ] , value ) >= 0 ) {
2016-03-05 19:59:39 +00:00
skipToken = false ;
2017-04-09 04:59:09 +00:00
if ( value === '=' && ( ( ref = prev [ 1 ] ) === '||' || ref === '&&' ) && ! prev . spaced ) {
2010-10-05 14:31:48 +00:00
prev [ 0 ] = 'COMPOUND_ASSIGN' ;
2010-10-11 10:10:30 +00:00
prev [ 1 ] += '=' ;
2016-03-05 19:59:39 +00:00
prev = this . tokens [ this . tokens . length - 2 ] ;
skipToken = true ;
}
2016-03-05 16:41:15 +00:00
if ( prev && prev [ 0 ] !== 'PROPERTY' ) {
2017-04-09 04:59:09 +00:00
origin = ( ref1 = prev . origin ) != null ? ref1 : prev ;
2016-03-05 19:59:39 +00:00
message = isUnassignable ( prev [ 1 ] , origin [ 1 ] ) ;
if ( message ) {
this . error ( message , origin [ 2 ] ) ;
}
}
if ( skipToken ) {
2010-10-22 14:47:40 +00:00
return value . length ;
2010-07-25 05:36:50 +00:00
}
2010-07-25 05:23:37 +00:00
}
2017-04-09 09:53:43 +00:00
if ( value === '{' && this . seenImport ) {
this . importSpecifierList = true ;
2018-01-31 14:33:17 +00:00
} else if ( this . importSpecifierList && value === '}' ) {
this . importSpecifierList = false ;
} else if ( value === '{' && ( prev != null ? prev [ 0 ] : void 0 ) === 'EXPORT' ) {
this . exportSpecifierList = true ;
} else if ( this . exportSpecifierList && value === '}' ) {
this . exportSpecifierList = false ;
2016-10-26 12:37:19 +00:00
}
2010-10-20 02:04:38 +00:00
if ( value === ';' ) {
2017-09-01 14:09:36 +00:00
if ( ref2 = prev != null ? prev [ 0 ] : void 0 , indexOf . call ( [ '=' , ... UNFINISHED ] , ref2 ) >= 0 ) {
this . error ( 'unexpected ;' ) ;
}
Support import and export of ES2015 modules (#4300)
This pull request adds support for ES2015 modules, by recognizing `import` and `export` statements. The following syntaxes are supported, based on the MDN [import](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import) and [export](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/export) pages:
```js
import "module-name"
import defaultMember from "module-name"
import * as name from "module-name"
import { } from "module-name"
import { member } from "module-name"
import { member as alias } from "module-name"
import { member1, member2 as alias2, … } from "module-name"
import defaultMember, * as name from "module-name"
import defaultMember, { … } from "module-name"
export default expression
export class name
export { }
export { name }
export { name as exportedName }
export { name as default }
export { name1, name2 as exportedName2, name3 as default, … }
export * from "module-name"
export { … } from "module-name"
```
As a subsitute for ECMAScript’s `export var name = …` and `export function name {}`, CoffeeScript also supports:
```js
export name = …
```
CoffeeScript also supports optional commas within `{ … }`.
This PR converts the supported `import` and `export` statements into ES2015 `import` and `export` statements; it **does not resolve the modules**. So any CoffeeScript with `import` or `export` statements will be output as ES2015, and will need to be transpiled by another tool such as Babel before it can be used in a browser. We will need to add a warning to the documentation explaining this.
This should be fully backwards-compatible, as `import` and `export` were previously reserved keywords. No flags are used.
There are extensive tests included, though because no current JavaScript runtime supports `import` or `export`, the tests compare strings of what the compiled CoffeeScript output is against what the expected ES2015 should be. I also conducted two more elaborate tests:
* I forked the [ember-piqu](https://github.com/pauc/piqu-ember) project, which was an Ember CLI app that used ember-cli-coffeescript and [ember-cli-coffees6](https://github.com/alexspeller/ember-cli-coffees6) (which adds “support” for `import`/`export` by wrapping such statements in backticks before passing the result to the CoffeeScript compiler). I removed `ember-cli-coffees6` and replaced the CoffeeScript compiler used in the build chain with this code, and the app built without errors. [Demo here.](https://github.com/GeoffreyBooth/coffeescript-modules-test-piqu)
* I also forked the [CoffeeScript version of Meteor’s Todos example app](https://github.com/meteor/todos/tree/coffeescript), and replaced all of its `require` statements with the `import` and `export` statements from the original ES2015 version of the app on its `master` branch. I then updated the `coffeescript` Meteor package in the app to use this new code, and again the app builds without errors. [Demo here.](https://github.com/GeoffreyBooth/coffeescript-modules-test-meteor-todos)
The discussion history for this work started [here](https://github.com/jashkenas/coffeescript/pull/4160) and continued [here](https://github.com/GeoffreyBooth/coffeescript/pull/2). @lydell provided guidance, and @JimPanic and @rattrayalex contributed essential code.
2016-09-14 18:46:05 +00:00
this . seenFor = this . seenImport = this . seenExport = false ;
2010-02-28 00:40:53 +00:00
tag = 'TERMINATOR' ;
2018-01-31 14:33:17 +00:00
} else if ( value === '*' && ( prev != null ? prev [ 0 ] : void 0 ) === 'EXPORT' ) {
tag = 'EXPORT_ALL' ;
} else if ( indexOf . call ( MATH , value ) >= 0 ) {
tag = 'MATH' ;
} else if ( indexOf . call ( COMPARE , value ) >= 0 ) {
tag = 'COMPARE' ;
} else if ( indexOf . call ( COMPOUND _ASSIGN , value ) >= 0 ) {
tag = 'COMPOUND_ASSIGN' ;
} else if ( indexOf . call ( UNARY , value ) >= 0 ) {
tag = 'UNARY' ;
} else if ( indexOf . call ( UNARY _MATH , value ) >= 0 ) {
tag = 'UNARY_MATH' ;
} else if ( indexOf . call ( SHIFT , value ) >= 0 ) {
tag = 'SHIFT' ;
} else if ( value === '?' && ( prev != null ? prev . spaced : void 0 ) ) {
tag = 'BIN?' ;
} else if ( prev ) {
if ( value === '(' && ! prev . spaced && ( ref3 = prev [ 0 ] , indexOf . call ( CALLABLE , ref3 ) >= 0 ) ) {
if ( prev [ 0 ] === '?' ) {
prev [ 0 ] = 'FUNC_EXIST' ;
}
tag = 'CALL_START' ;
} else if ( value === '[' && ( ( ( ref4 = prev [ 0 ] , indexOf . call ( INDEXABLE , ref4 ) >= 0 ) && ! prev . spaced ) || ( prev [ 0 ] === '::' ) ) ) { // `.prototype` can’ t be a method you can call.
tag = 'INDEX_START' ;
switch ( prev [ 0 ] ) {
case '?' :
prev [ 0 ] = 'INDEX_SOAK' ;
2010-06-01 02:32:43 +00:00
}
2010-02-28 00:40:53 +00:00
}
}
2015-01-03 23:28:23 +00:00
token = this . makeToken ( tag , value ) ;
2011-09-16 23:26:04 +00:00
switch ( value ) {
case '(' :
case '{' :
case '[' :
2015-01-03 23:28:23 +00:00
this . ends . push ( {
tag : INVERSES [ value ] ,
origin : token
} ) ;
2011-09-16 23:26:04 +00:00
break ;
case ')' :
case '}' :
case ']' :
this . pair ( value ) ;
}
2017-06-07 06:33:46 +00:00
this . tokens . push ( this . makeToken ( tag , value ) ) ;
2010-10-22 14:47:40 +00:00
return value . length ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Token Manipulators
// ------------------
// A source of ambiguity in our grammar used to be parameter lists in function
// definitions versus argument lists in function calls. Walk backwards, tagging
// parameters specially in order to make things easier for the parser.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
tagParameters ( ) {
2017-05-30 00:29:45 +00:00
var i , paramEndToken , stack , tok , tokens ;
2012-04-10 18:57:45 +00:00
if ( this . tag ( ) !== ')' ) {
return this ;
}
2010-10-26 04:09:46 +00:00
stack = [ ] ;
2017-04-06 17:06:45 +00:00
( { tokens } = this ) ;
2010-10-26 04:09:46 +00:00
i = tokens . length ;
2017-05-30 00:29:45 +00:00
paramEndToken = tokens [ -- i ] ;
paramEndToken [ 0 ] = 'PARAM_END' ;
2010-10-26 04:09:46 +00:00
while ( tok = tokens [ -- i ] ) {
2010-09-16 03:46:01 +00:00
switch ( tok [ 0 ] ) {
2010-09-22 03:58:05 +00:00
case ')' :
2010-10-26 04:09:46 +00:00
stack . push ( tok ) ;
2010-09-22 03:58:05 +00:00
break ;
case '(' :
2011-02-23 00:20:01 +00:00
case 'CALL_START' :
2010-10-26 04:09:46 +00:00
if ( stack . length ) {
stack . pop ( ) ;
2018-01-31 14:33:17 +00:00
} else if ( tok [ 0 ] === '(' ) {
tok [ 0 ] = 'PARAM_START' ;
return this ;
2011-06-07 07:58:36 +00:00
} else {
2018-01-31 14:33:17 +00:00
paramEndToken [ 0 ] = 'CALL_END' ;
return this ;
2010-10-26 04:09:46 +00:00
}
2010-03-01 01:44:33 +00:00
}
}
2010-10-22 14:47:40 +00:00
return this ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Close up all remaining open blocks at the end of the file.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
closeIndentation ( ) {
2010-06-12 23:05:13 +00:00
return this . outdentToken ( this . indent ) ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Match the contents of a delimited token and expand variables and expressions
// inside it using Ruby-like notation for substitution of arbitrary
// expressions.
// "Hello #{name.capitalize()}."
// If it encounters an interpolation, this method will recursively create a new
// Lexer and tokenize until the `{` of `#{` is balanced with a `}`.
// - `regex` matches the contents of a token (but not `delimiter`, and not
// `#{` if interpolations are desired).
// - `delimiter` is the delimiter of the token. Examples are `'`, `"`, `'''`,
// `"""` and `///`.
// - `closingDelimiter` is different from `delimiter` only in CSX
// - `interpolators` matches the start of an interpolation, for CSX it's both
// `{` and `<` (i.e. nested CSX tag)
// This method allows us to have strings within interpolations within strings,
// ad infinitum.
2017-06-07 06:33:46 +00:00
matchWithInterpolations ( regex , delimiter , closingDelimiter , interpolators ) {
var braceInterpolator , close , column , firstToken , index , interpolationOffset , interpolator , lastToken , line , match , nested , offsetInChunk , open , ref , rest , str , strPart , tokens ;
if ( closingDelimiter == null ) {
closingDelimiter = delimiter ;
}
if ( interpolators == null ) {
interpolators = /^#\{/ ;
}
2010-09-25 22:06:14 +00:00
tokens = [ ] ;
2015-02-03 17:55:38 +00:00
offsetInChunk = delimiter . length ;
if ( this . chunk . slice ( 0 , offsetInChunk ) !== delimiter ) {
return null ;
}
str = this . chunk . slice ( offsetInChunk ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
while ( true ) {
2017-04-06 17:06:45 +00:00
[ strPart ] = regex . exec ( str ) ;
2015-02-05 16:23:03 +00:00
this . validateEscapes ( strPart , {
isRegex : delimiter . charAt ( 0 ) === '/' ,
2017-04-06 17:06:45 +00:00
offsetInChunk
2015-02-05 16:23:03 +00:00
} ) ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Push a fake `'NEOSTRING'` token, which will get turned into a real string later.
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
tokens . push ( this . makeToken ( 'NEOSTRING' , strPart , offsetInChunk ) ) ;
str = str . slice ( strPart . length ) ;
offsetInChunk += strPart . length ;
2017-06-07 06:33:46 +00:00
if ( ! ( match = interpolators . exec ( str ) ) ) {
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
break ;
2012-04-10 18:57:45 +00:00
}
2017-06-07 06:33:46 +00:00
[ interpolator ] = match ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// To remove the `#` in `#{`.
2017-06-07 06:33:46 +00:00
interpolationOffset = interpolator . length - 1 ;
[ line , column ] = this . getLineAndColumnFromChunk ( offsetInChunk + interpolationOffset ) ;
rest = str . slice ( interpolationOffset ) ;
2017-04-06 17:06:45 +00:00
( {
tokens : nested ,
index
2017-06-07 06:33:46 +00:00
} = new Lexer ( ) . tokenize ( rest , {
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
line : line ,
column : column ,
untilBalanced : true
2017-04-06 17:06:45 +00:00
} ) ) ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Account for the `#` in `#{`
2017-06-07 06:33:46 +00:00
index += interpolationOffset ;
braceInterpolator = str [ index - 1 ] === '}' ;
if ( braceInterpolator ) {
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Turn the leading and trailing `{` and `}` into parentheses. Unnecessary
// parentheses will be removed later.
2018-01-16 03:24:21 +00:00
[ open ] = nested , [ close ] = slice . call ( nested , - 1 ) ;
2017-06-07 06:33:46 +00:00
open [ 0 ] = open [ 1 ] = '(' ;
close [ 0 ] = close [ 1 ] = ')' ;
close . origin = [ '' , 'end of interpolation' , close [ 2 ] ] ;
}
2017-04-06 17:06:45 +00:00
if ( ( ( ref = nested [ 1 ] ) != null ? ref [ 0 ] : void 0 ) === 'TERMINATOR' ) {
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Remove leading `'TERMINATOR'` (if any).
2015-01-12 19:10:54 +00:00
nested . splice ( 1 , 1 ) ;
2010-03-05 22:30:49 +00:00
}
2017-06-07 06:33:46 +00:00
if ( ! braceInterpolator ) {
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// We are not using `{` and `}`, so wrap the interpolated tokens instead.
2017-06-07 06:33:46 +00:00
open = this . makeToken ( '(' , '(' , offsetInChunk , 0 ) ;
close = this . makeToken ( ')' , ')' , offsetInChunk + index , 0 ) ;
nested = [ open , ... nested , close ] ;
}
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Push a fake `'TOKENS'` token, which will get turned into real tokens later.
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
tokens . push ( [ 'TOKENS' , nested ] ) ;
str = str . slice ( index ) ;
offsetInChunk += index ;
2010-09-25 22:06:14 +00:00
}
2017-06-07 06:33:46 +00:00
if ( str . slice ( 0 , closingDelimiter . length ) !== closingDelimiter ) {
this . error ( ` missing ${ closingDelimiter } ` , {
2015-02-06 09:52:02 +00:00
length : delimiter . length
} ) ;
2012-04-10 18:57:45 +00:00
}
2018-01-16 03:24:21 +00:00
[ firstToken ] = tokens , [ lastToken ] = slice . call ( tokens , - 1 ) ;
2015-02-03 17:55:38 +00:00
firstToken [ 2 ] . first _column -= delimiter . length ;
2016-10-23 07:41:46 +00:00
if ( lastToken [ 1 ] . substr ( - 1 ) === '\n' ) {
lastToken [ 2 ] . last _line += 1 ;
2017-06-07 06:33:46 +00:00
lastToken [ 2 ] . last _column = closingDelimiter . length - 1 ;
2016-10-23 07:41:46 +00:00
} else {
2017-06-07 06:33:46 +00:00
lastToken [ 2 ] . last _column += closingDelimiter . length ;
2016-10-23 07:41:46 +00:00
}
Fix #3597: Allow interpolations in object keys
The following is now allowed:
o =
a: 1
b: 2
"#{'c'}": 3
"#{'d'}": 4
e: 5
"#{'f'}": 6
g: 7
It compiles to:
o = (
obj = {
a: 1,
b: 2
},
obj["" + 'c'] = 3,
obj["" + 'd'] = 4,
obj.e = 5,
obj["" + 'f'] = 6,
obj.g = 7,
obj
);
- Closes #3039. Empty interpolations in object keys are now _supposed_ to be
allowed.
- Closes #1131. No need to improve error messages for attempted key
interpolation anymore.
- Implementing this required fixing the following bug: `("" + a): 1` used to
error out on the colon, saying "unexpected colon". But really, it is the
attempted object key that is unexpected. Now the error is on the opening
parenthesis instead.
- However, the above fix broke some error message tests for regexes. The easiest
way to fix this was to make a seemingly unrelated change: The error messages
for unexpected identifiers, numbers, strings and regexes now say for example
'unexpected string' instead of 'unexpected """some #{really long} string"""'.
In other words, the tag _name_ is used instead of the tag _value_.
This was way easier to implement, and is more helpful to the user. Using the
tag value is good for operators, reserved words and the like, but not for
tokens which can contain any text. For example, 'unexpected identifier' is
better than 'unexpected expected' (if a variable called 'expected' was used
erraneously).
- While writing tests for the above point I found a few minor bugs with string
locations which have been fixed.
2015-02-07 19:16:59 +00:00
if ( lastToken [ 1 ] . length === 0 ) {
lastToken [ 2 ] . last _column -= 1 ;
}
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
return {
2017-04-06 17:06:45 +00:00
tokens ,
2017-06-07 06:33:46 +00:00
index : offsetInChunk + closingDelimiter . length
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
} ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Merge the array `tokens` of the fake token types `'TOKENS'` and `'NEOSTRING'`
// (as returned by `matchWithInterpolations`) into the token stream. The value
// of `'NEOSTRING'`s are converted using `fn` and turned into strings using
// `options` first.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
mergeInterpolationTokens ( tokens , options , fn ) {
2017-08-24 06:34:59 +00:00
var converted , firstEmptyStringIndex , firstIndex , i , j , k , lastToken , len , len1 , locationToken , lparen , placeholderToken , plusToken , rparen , tag , token , tokensToPush , val , value ;
Fix #3597: Allow interpolations in object keys
The following is now allowed:
o =
a: 1
b: 2
"#{'c'}": 3
"#{'d'}": 4
e: 5
"#{'f'}": 6
g: 7
It compiles to:
o = (
obj = {
a: 1,
b: 2
},
obj["" + 'c'] = 3,
obj["" + 'd'] = 4,
obj.e = 5,
obj["" + 'f'] = 6,
obj.g = 7,
obj
);
- Closes #3039. Empty interpolations in object keys are now _supposed_ to be
allowed.
- Closes #1131. No need to improve error messages for attempted key
interpolation anymore.
- Implementing this required fixing the following bug: `("" + a): 1` used to
error out on the colon, saying "unexpected colon". But really, it is the
attempted object key that is unexpected. Now the error is on the opening
parenthesis instead.
- However, the above fix broke some error message tests for regexes. The easiest
way to fix this was to make a seemingly unrelated change: The error messages
for unexpected identifiers, numbers, strings and regexes now say for example
'unexpected string' instead of 'unexpected """some #{really long} string"""'.
In other words, the tag _name_ is used instead of the tag _value_.
This was way easier to implement, and is more helpful to the user. Using the
tag value is good for operators, reserved words and the like, but not for
tokens which can contain any text. For example, 'unexpected identifier' is
better than 'unexpected expected' (if a variable called 'expected' was used
erraneously).
- While writing tests for the above point I found a few minor bugs with string
locations which have been fixed.
2015-02-07 19:16:59 +00:00
if ( tokens . length > 1 ) {
lparen = this . token ( 'STRING_START' , '(' , 0 , 0 ) ;
2012-04-10 18:57:45 +00:00
}
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
firstIndex = this . tokens . length ;
2015-01-30 19:33:03 +00:00
for ( i = j = 0 , len = tokens . length ; j < len ; i = ++ j ) {
2012-11-17 00:09:56 +00:00
token = tokens [ i ] ;
2017-04-06 17:06:45 +00:00
[ tag , value ] = token ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
switch ( tag ) {
case 'TOKENS' :
2015-01-12 19:10:54 +00:00
if ( value . length === 2 ) {
2017-08-24 06:34:59 +00:00
if ( ! ( value [ 0 ] . comments || value [ 1 ] . comments ) ) {
// Optimize out empty interpolations (an empty pair of parentheses).
continue ;
}
// There are comments (and nothing else) in this interpolation.
if ( this . csxDepth === 0 ) {
// This is an interpolated string, not a CSX tag; and for whatever
// reason `` `a${/*test*/}b` `` is invalid JS. So compile to
// `` `a${/*test*/''}b` `` instead.
placeholderToken = this . makeToken ( 'STRING' , "''" ) ;
} else {
placeholderToken = this . makeToken ( 'JS' , '' ) ;
}
// Use the same location data as the first parenthesis.
placeholderToken [ 2 ] = value [ 0 ] [ 2 ] ;
for ( k = 0 , len1 = value . length ; k < len1 ; k ++ ) {
val = value [ k ] ;
if ( ! val . comments ) {
continue ;
}
if ( placeholderToken . comments == null ) {
placeholderToken . comments = [ ] ;
}
placeholderToken . comments . push ( ... val . comments ) ;
}
value . splice ( 1 , 0 , placeholderToken ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
}
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Push all the tokens in the fake `'TOKENS'` token. These already have
// sane location data.
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
locationToken = value [ 0 ] ;
tokensToPush = value ;
break ;
case 'NEOSTRING' :
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Convert `'NEOSTRING'` into `'STRING'`.
2017-04-20 06:03:06 +00:00
converted = fn . call ( this , token [ 1 ] , i ) ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Optimize out empty strings. We ensure that the tokens stream always
// starts with a string token, though, to make sure that the result
// really is a string.
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
if ( converted . length === 0 ) {
if ( i === 0 ) {
firstEmptyStringIndex = this . tokens . length ;
} else {
continue ;
}
}
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// However, there is one case where we can optimize away a starting
// empty string.
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
if ( i === 2 && ( firstEmptyStringIndex != null ) ) {
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
this . tokens . splice ( firstEmptyStringIndex , 2 ) ; // Remove empty string and the plus.
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
}
token [ 0 ] = 'STRING' ;
2015-02-05 16:23:03 +00:00
token [ 1 ] = this . makeDelimitedLiteral ( converted , options ) ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
locationToken = token ;
tokensToPush = [ token ] ;
}
if ( this . tokens . length > firstIndex ) {
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Create a 0-length "+" token.
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
plusToken = this . token ( '+' , '+' ) ;
2013-01-14 20:20:35 +00:00
plusToken [ 2 ] = {
first _line : locationToken [ 2 ] . first _line ,
first _column : locationToken [ 2 ] . first _column ,
last _line : locationToken [ 2 ] . first _line ,
last _column : locationToken [ 2 ] . first _column
2012-11-19 22:37:46 +00:00
} ;
2012-04-10 18:57:45 +00:00
}
Compile splats in arrays and function calls to ES2015 splats (#4353)
Rather than compiling splats to arrays built using `Array#concat`, splats
are now compiled directly to ES2015 splats, e.g.
f foo, arguments..., bar
[ foo, arguments..., bar ]
Which used to be compiled to:
f.apply(null, [foo].concat(slice.call(arguments), [bar]));
[foo].concat(slice.call(arguments), [bar]);
Is now compiled to:
f(foo, ...arguments, bar);
[ foo, ...arguments, bar ];
2016-11-06 16:30:04 +00:00
this . tokens . push ( ... tokensToPush ) ;
2010-03-05 22:30:49 +00:00
}
Fix #3597: Allow interpolations in object keys
The following is now allowed:
o =
a: 1
b: 2
"#{'c'}": 3
"#{'d'}": 4
e: 5
"#{'f'}": 6
g: 7
It compiles to:
o = (
obj = {
a: 1,
b: 2
},
obj["" + 'c'] = 3,
obj["" + 'd'] = 4,
obj.e = 5,
obj["" + 'f'] = 6,
obj.g = 7,
obj
);
- Closes #3039. Empty interpolations in object keys are now _supposed_ to be
allowed.
- Closes #1131. No need to improve error messages for attempted key
interpolation anymore.
- Implementing this required fixing the following bug: `("" + a): 1` used to
error out on the colon, saying "unexpected colon". But really, it is the
attempted object key that is unexpected. Now the error is on the opening
parenthesis instead.
- However, the above fix broke some error message tests for regexes. The easiest
way to fix this was to make a seemingly unrelated change: The error messages
for unexpected identifiers, numbers, strings and regexes now say for example
'unexpected string' instead of 'unexpected """some #{really long} string"""'.
In other words, the tag _name_ is used instead of the tag _value_.
This was way easier to implement, and is more helpful to the user. Using the
tag value is good for operators, reserved words and the like, but not for
tokens which can contain any text. For example, 'unexpected identifier' is
better than 'unexpected expected' (if a variable called 'expected' was used
erraneously).
- While writing tests for the above point I found a few minor bugs with string
locations which have been fixed.
2015-02-07 19:16:59 +00:00
if ( lparen ) {
2018-01-16 03:24:21 +00:00
[ lastToken ] = slice . call ( tokens , - 1 ) ;
Fix #3597: Allow interpolations in object keys
The following is now allowed:
o =
a: 1
b: 2
"#{'c'}": 3
"#{'d'}": 4
e: 5
"#{'f'}": 6
g: 7
It compiles to:
o = (
obj = {
a: 1,
b: 2
},
obj["" + 'c'] = 3,
obj["" + 'd'] = 4,
obj.e = 5,
obj["" + 'f'] = 6,
obj.g = 7,
obj
);
- Closes #3039. Empty interpolations in object keys are now _supposed_ to be
allowed.
- Closes #1131. No need to improve error messages for attempted key
interpolation anymore.
- Implementing this required fixing the following bug: `("" + a): 1` used to
error out on the colon, saying "unexpected colon". But really, it is the
attempted object key that is unexpected. Now the error is on the opening
parenthesis instead.
- However, the above fix broke some error message tests for regexes. The easiest
way to fix this was to make a seemingly unrelated change: The error messages
for unexpected identifiers, numbers, strings and regexes now say for example
'unexpected string' instead of 'unexpected """some #{really long} string"""'.
In other words, the tag _name_ is used instead of the tag _value_.
This was way easier to implement, and is more helpful to the user. Using the
tag value is good for operators, reserved words and the like, but not for
tokens which can contain any text. For example, 'unexpected identifier' is
better than 'unexpected expected' (if a variable called 'expected' was used
erraneously).
- While writing tests for the above point I found a few minor bugs with string
locations which have been fixed.
2015-02-07 19:16:59 +00:00
lparen . origin = [
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
'STRING' ,
null ,
{
Fix #3597: Allow interpolations in object keys
The following is now allowed:
o =
a: 1
b: 2
"#{'c'}": 3
"#{'d'}": 4
e: 5
"#{'f'}": 6
g: 7
It compiles to:
o = (
obj = {
a: 1,
b: 2
},
obj["" + 'c'] = 3,
obj["" + 'd'] = 4,
obj.e = 5,
obj["" + 'f'] = 6,
obj.g = 7,
obj
);
- Closes #3039. Empty interpolations in object keys are now _supposed_ to be
allowed.
- Closes #1131. No need to improve error messages for attempted key
interpolation anymore.
- Implementing this required fixing the following bug: `("" + a): 1` used to
error out on the colon, saying "unexpected colon". But really, it is the
attempted object key that is unexpected. Now the error is on the opening
parenthesis instead.
- However, the above fix broke some error message tests for regexes. The easiest
way to fix this was to make a seemingly unrelated change: The error messages
for unexpected identifiers, numbers, strings and regexes now say for example
'unexpected string' instead of 'unexpected """some #{really long} string"""'.
In other words, the tag _name_ is used instead of the tag _value_.
This was way easier to implement, and is more helpful to the user. Using the
tag value is good for operators, reserved words and the like, but not for
tokens which can contain any text. For example, 'unexpected identifier' is
better than 'unexpected expected' (if a variable called 'expected' was used
erraneously).
- While writing tests for the above point I found a few minor bugs with string
locations which have been fixed.
2015-02-07 19:16:59 +00:00
first _line : lparen [ 2 ] . first _line ,
first _column : lparen [ 2 ] . first _column ,
last _line : lastToken [ 2 ] . last _line ,
last _column : lastToken [ 2 ] . last _column
}
] ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
lparen [ 2 ] = lparen . origin [ 2 ] ;
Fix #3597: Allow interpolations in object keys
The following is now allowed:
o =
a: 1
b: 2
"#{'c'}": 3
"#{'d'}": 4
e: 5
"#{'f'}": 6
g: 7
It compiles to:
o = (
obj = {
a: 1,
b: 2
},
obj["" + 'c'] = 3,
obj["" + 'd'] = 4,
obj.e = 5,
obj["" + 'f'] = 6,
obj.g = 7,
obj
);
- Closes #3039. Empty interpolations in object keys are now _supposed_ to be
allowed.
- Closes #1131. No need to improve error messages for attempted key
interpolation anymore.
- Implementing this required fixing the following bug: `("" + a): 1` used to
error out on the colon, saying "unexpected colon". But really, it is the
attempted object key that is unexpected. Now the error is on the opening
parenthesis instead.
- However, the above fix broke some error message tests for regexes. The easiest
way to fix this was to make a seemingly unrelated change: The error messages
for unexpected identifiers, numbers, strings and regexes now say for example
'unexpected string' instead of 'unexpected """some #{really long} string"""'.
In other words, the tag _name_ is used instead of the tag _value_.
This was way easier to implement, and is more helpful to the user. Using the
tag value is good for operators, reserved words and the like, but not for
tokens which can contain any text. For example, 'unexpected identifier' is
better than 'unexpected expected' (if a variable called 'expected' was used
erraneously).
- While writing tests for the above point I found a few minor bugs with string
locations which have been fixed.
2015-02-07 19:16:59 +00:00
rparen = this . token ( 'STRING_END' , ')' ) ;
return rparen [ 2 ] = {
2015-02-04 10:06:35 +00:00
first _line : lastToken [ 2 ] . last _line ,
Fix #3597: Allow interpolations in object keys
The following is now allowed:
o =
a: 1
b: 2
"#{'c'}": 3
"#{'d'}": 4
e: 5
"#{'f'}": 6
g: 7
It compiles to:
o = (
obj = {
a: 1,
b: 2
},
obj["" + 'c'] = 3,
obj["" + 'd'] = 4,
obj.e = 5,
obj["" + 'f'] = 6,
obj.g = 7,
obj
);
- Closes #3039. Empty interpolations in object keys are now _supposed_ to be
allowed.
- Closes #1131. No need to improve error messages for attempted key
interpolation anymore.
- Implementing this required fixing the following bug: `("" + a): 1` used to
error out on the colon, saying "unexpected colon". But really, it is the
attempted object key that is unexpected. Now the error is on the opening
parenthesis instead.
- However, the above fix broke some error message tests for regexes. The easiest
way to fix this was to make a seemingly unrelated change: The error messages
for unexpected identifiers, numbers, strings and regexes now say for example
'unexpected string' instead of 'unexpected """some #{really long} string"""'.
In other words, the tag _name_ is used instead of the tag _value_.
This was way easier to implement, and is more helpful to the user. Using the
tag value is good for operators, reserved words and the like, but not for
tokens which can contain any text. For example, 'unexpected identifier' is
better than 'unexpected expected' (if a variable called 'expected' was used
erraneously).
- While writing tests for the above point I found a few minor bugs with string
locations which have been fixed.
2015-02-07 19:16:59 +00:00
first _column : lastToken [ 2 ] . last _column ,
2015-02-04 10:06:35 +00:00
last _line : lastToken [ 2 ] . last _line ,
Fix #3597: Allow interpolations in object keys
The following is now allowed:
o =
a: 1
b: 2
"#{'c'}": 3
"#{'d'}": 4
e: 5
"#{'f'}": 6
g: 7
It compiles to:
o = (
obj = {
a: 1,
b: 2
},
obj["" + 'c'] = 3,
obj["" + 'd'] = 4,
obj.e = 5,
obj["" + 'f'] = 6,
obj.g = 7,
obj
);
- Closes #3039. Empty interpolations in object keys are now _supposed_ to be
allowed.
- Closes #1131. No need to improve error messages for attempted key
interpolation anymore.
- Implementing this required fixing the following bug: `("" + a): 1` used to
error out on the colon, saying "unexpected colon". But really, it is the
attempted object key that is unexpected. Now the error is on the opening
parenthesis instead.
- However, the above fix broke some error message tests for regexes. The easiest
way to fix this was to make a seemingly unrelated change: The error messages
for unexpected identifiers, numbers, strings and regexes now say for example
'unexpected string' instead of 'unexpected """some #{really long} string"""'.
In other words, the tag _name_ is used instead of the tag _value_.
This was way easier to implement, and is more helpful to the user. Using the
tag value is good for operators, reserved words and the like, but not for
tokens which can contain any text. For example, 'unexpected identifier' is
better than 'unexpected expected' (if a variable called 'expected' was used
erraneously).
- While writing tests for the above point I found a few minor bugs with string
locations which have been fixed.
2015-02-07 19:16:59 +00:00
last _column : lastToken [ 2 ] . last _column
2015-02-04 10:06:35 +00:00
} ;
2012-04-10 18:57:45 +00:00
}
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Pairs up a closing token, ensuring that all listed pairs of tokens are
// correctly balanced throughout the course of the token stream.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
pair ( tag ) {
2017-04-06 17:06:45 +00:00
var lastIndent , prev , ref , ref1 , wanted ;
2018-01-16 03:24:21 +00:00
ref = this . ends , [ prev ] = slice . call ( ref , - 1 ) ;
2015-02-07 20:50:41 +00:00
if ( tag !== ( wanted = prev != null ? prev . tag : void 0 ) ) {
2012-04-10 18:57:45 +00:00
if ( 'OUTDENT' !== wanted ) {
2016-11-28 14:05:51 +00:00
this . error ( ` unmatched ${ tag } ` ) ;
2012-04-10 18:57:45 +00:00
}
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Auto-close `INDENT` to support syntax like this:
// el.click((event) ->
// el.hide())
2018-01-16 03:24:21 +00:00
ref1 = this . indents , [ lastIndent ] = slice . call ( ref1 , - 1 ) ;
2015-02-07 20:50:41 +00:00
this . outdentToken ( lastIndent , true ) ;
2011-09-16 23:26:04 +00:00
return this . pair ( tag ) ;
}
return this . ends . pop ( ) ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Helpers
// -------
// Returns the line and column number from an offset into the current chunk.
// `offset` is a number of characters into `@chunk`.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
getLineAndColumnFromChunk ( offset ) {
2017-04-06 17:06:45 +00:00
var column , lastLine , lineCount , ref , string ;
2012-11-17 00:09:56 +00:00
if ( offset === 0 ) {
return [ this . chunkLine , this . chunkColumn ] ;
}
if ( offset >= this . chunk . length ) {
string = this . chunk ;
} else {
string = this . chunk . slice ( 0 , + ( offset - 1 ) + 1 || 9e9 ) ;
}
lineCount = count ( string , '\n' ) ;
column = this . chunkColumn ;
if ( lineCount > 0 ) {
2018-01-16 03:24:21 +00:00
ref = string . split ( '\n' ) , [ lastLine ] = slice . call ( ref , - 1 ) ;
2015-02-07 20:50:41 +00:00
column = lastLine . length ;
2012-11-17 00:09:56 +00:00
} else {
column += string . length ;
}
return [ this . chunkLine + lineCount , column ] ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2012-11-17 00:09:56 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Same as `token`, except this just returns the token without adding it
// to the results.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
makeToken ( tag , value , offsetInChunk = 0 , length = value . length ) {
2017-04-06 17:06:45 +00:00
var lastCharacter , locationData , token ;
2012-11-17 00:09:56 +00:00
locationData = { } ;
2017-04-06 17:06:45 +00:00
[ locationData . first _line , locationData . first _column ] = this . getLineAndColumnFromChunk ( offsetInChunk ) ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Use length - 1 for the final offset - we're supplying the last_line and the last_column,
// so if last_column == first_column, then we're looking at a character of length 1.
Fix incorrect location data in OUTDENT nodes
In f609036beef3aa1529c210ffad3ced818ee94cce, a line was changed from
`if length > 0 then (length - 1) else 0` to `Math.max 0, length - 1`. However,
in some cases, the `length` variable can be `undefined`. The previous code would
correctly compute `lastCharacter` as 0, but the new code would compute it as
`NaN`. This would cause trouble later on: the end location would just be the end
of the current chunk, which would be incorrect.
Here's a specific case where the parser was behaving incorrectly:
```
a {
b: ->
return c d,
if e
f
}
g
```
The OUTDENT tokens after the `f` had an undefined length, so the `NaN` made it
so the end location was at the end of the file. That meant that various nodes in
the AST, like the `return` node, would incorrectly have an end location at the
end of the file.
To fix, I just reverted the change to that particular line.
2016-07-28 06:16:30 +00:00
lastCharacter = length > 0 ? length - 1 : 0 ;
2017-04-06 17:06:45 +00:00
[ locationData . last _line , locationData . last _column ] = this . getLineAndColumnFromChunk ( offsetInChunk + lastCharacter ) ;
2013-01-14 20:20:35 +00:00
token = [ tag , value , locationData ] ;
2012-11-17 00:09:56 +00:00
return token ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2012-11-17 00:09:56 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Add a token to the results.
// `offset` is the offset into the current `@chunk` where the token starts.
// `length` is the length of the token in the `@chunk`, after the offset. If
// not specified, the length of `value` will be used.
// Returns the new token.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
token ( tag , value , offsetInChunk , length , origin ) {
2012-11-17 00:09:56 +00:00
var token ;
token = this . makeToken ( tag , value , offsetInChunk , length ) ;
2014-01-22 02:44:50 +00:00
if ( origin ) {
token . origin = origin ;
}
2012-11-17 00:09:56 +00:00
this . tokens . push ( token ) ;
return token ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Peek at the last tag in the token stream.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
tag ( ) {
2017-04-06 17:06:45 +00:00
var ref , token ;
2018-01-16 03:24:21 +00:00
ref = this . tokens , [ token ] = slice . call ( ref , - 1 ) ;
2015-02-07 20:50:41 +00:00
return token != null ? token [ 0 ] : void 0 ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Peek at the last value in the token stream.
2017-10-12 18:47:12 +00:00
value ( useOrigin = false ) {
var ref , ref1 , token ;
2018-01-16 03:24:21 +00:00
ref = this . tokens , [ token ] = slice . call ( ref , - 1 ) ;
2017-10-12 18:47:12 +00:00
if ( useOrigin && ( ( token != null ? token . origin : void 0 ) != null ) ) {
return ( ref1 = token . origin ) != null ? ref1 [ 1 ] : void 0 ;
} else {
return token != null ? token [ 1 ] : void 0 ;
}
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Get the previous token in the token stream.
2017-04-09 04:59:09 +00:00
prev ( ) {
return this . tokens [ this . tokens . length - 1 ] ;
}
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Are we in the midst of an unfinished expression?
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
unfinished ( ) {
2017-04-06 17:06:45 +00:00
var ref ;
2017-07-09 17:28:50 +00:00
return LINE _CONTINUER . test ( this . chunk ) || ( ref = this . tag ( ) , indexOf . call ( UNFINISHED , ref ) >= 0 ) ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
2017-04-20 19:41:28 +00:00
formatString ( str , options ) {
2017-04-20 06:03:06 +00:00
return this . replaceUnicodeCodePointEscapes ( str . replace ( STRING _OMIT , '$1' ) , options ) ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
2017-04-25 17:15:08 +00:00
formatHeregex ( str , options ) {
return this . formatRegex ( str . replace ( HEREGEX _OMIT , '$1$2' ) , merge ( options , {
2017-04-20 06:03:06 +00:00
delimiter : '///'
2017-04-25 17:15:08 +00:00
} ) ) ;
2017-04-20 19:41:28 +00:00
}
2017-04-20 06:03:06 +00:00
2017-04-20 19:41:28 +00:00
formatRegex ( str , options ) {
2017-04-20 06:03:06 +00:00
return this . replaceUnicodeCodePointEscapes ( str , options ) ;
2017-04-20 19:41:28 +00:00
}
2017-04-20 06:03:06 +00:00
2017-04-20 19:41:28 +00:00
unicodeCodePointToUnicodeEscapes ( codePoint ) {
2017-04-20 06:03:06 +00:00
var high , low , toUnicodeEscape ;
toUnicodeEscape = function ( val ) {
var str ;
str = val . toString ( 16 ) ;
2017-04-20 19:41:28 +00:00
return ` \\ u ${ repeat ( '0' , 4 - str . length ) } ${ str } ` ;
2017-04-20 06:03:06 +00:00
} ;
if ( codePoint < 0x10000 ) {
return toUnicodeEscape ( codePoint ) ;
}
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// surrogate pair
2017-04-20 06:03:06 +00:00
high = Math . floor ( ( codePoint - 0x10000 ) / 0x400 ) + 0xD800 ;
low = ( codePoint - 0x10000 ) % 0x400 + 0xDC00 ;
2017-04-20 19:41:28 +00:00
return ` ${ toUnicodeEscape ( high ) } ${ toUnicodeEscape ( low ) } ` ;
}
2017-04-20 06:03:06 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Replace `\u{...}` with `\uxxxx[\uxxxx]` in regexes without `u` flag
2017-04-20 19:41:28 +00:00
replaceUnicodeCodePointEscapes ( str , options ) {
2017-04-25 17:15:08 +00:00
var shouldReplace ;
shouldReplace = ( options . flags != null ) && indexOf . call ( options . flags , 'u' ) < 0 ;
2017-04-20 19:41:28 +00:00
return str . replace ( UNICODE _CODE _POINT _ESCAPE , ( match , escapedBackslash , codePointHex , offset ) => {
var codePointDecimal ;
if ( escapedBackslash ) {
return escapedBackslash ;
}
codePointDecimal = parseInt ( codePointHex , 16 ) ;
if ( codePointDecimal > 0x10ffff ) {
this . error ( "unicode code point escapes greater than \\u{10ffff} are not allowed" , {
offset : offset + options . delimiter . length ,
length : codePointHex . length + 4
} ) ;
}
2017-04-25 17:15:08 +00:00
if ( ! shouldReplace ) {
return match ;
}
2017-04-20 19:41:28 +00:00
return this . unicodeCodePointToUnicodeEscapes ( codePointDecimal ) ;
} ) ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Validates escapes in strings and regexes.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
validateEscapes ( str , options = { } ) {
2017-04-20 19:41:28 +00:00
var before , hex , invalidEscape , invalidEscapeRegex , match , message , octal , ref , unicode , unicodeCodePoint ;
2017-04-09 09:53:43 +00:00
invalidEscapeRegex = options . isRegex ? REGEX _INVALID _ESCAPE : STRING _INVALID _ESCAPE ;
match = invalidEscapeRegex . exec ( str ) ;
2015-02-05 16:23:03 +00:00
if ( ! match ) {
return ;
}
2017-04-20 06:03:06 +00:00
match [ 0 ] , before = match [ 1 ] , octal = match [ 2 ] , hex = match [ 3 ] , unicodeCodePoint = match [ 4 ] , unicode = match [ 5 ] ;
2015-02-06 09:52:02 +00:00
message = octal ? "octal escape sequences are not allowed" : "invalid escape sequence" ;
2017-04-20 19:41:28 +00:00
invalidEscape = ` \\ ${ octal || hex || unicodeCodePoint || unicode } ` ;
2016-11-28 14:05:51 +00:00
return this . error ( ` ${ message } ${ invalidEscape } ` , {
2017-04-06 17:06:45 +00:00
offset : ( ( ref = options . offsetInChunk ) != null ? ref : 0 ) + match . index + before . length ,
2015-02-06 09:52:02 +00:00
length : invalidEscape . length
} ) ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2015-02-05 16:23:03 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Constructs a string or regex by escaping certain characters.
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
makeDelimitedLiteral ( body , options = { } ) {
2015-02-05 16:23:03 +00:00
var regex ;
if ( body === '' && options . delimiter === '/' ) {
body = '(?:)' ;
}
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
regex = RegExp ( ` ( \\ \\ \\ \\ )|( \\ \\ 0(?=[1-7]))| \\ \\ ?( ${ options . delimiter // Escaped backslash.
// Null character mistaken as octal escape.
// (Possibly escaped) delimiter.
// (Possibly escaped) newlines.
// Other escapes.
} ) | \ \ \ \ ? ( ? : ( \ \ n ) | ( \ \ r ) | ( \ \u2028 ) | ( \ \u2029 ) ) | ( \ \ \ \ . ) ` , "g");
2015-02-05 16:23:03 +00:00
body = body . replace ( regex , function ( match , backslash , nul , delimiter , lf , cr , ls , ps , other ) {
switch ( false ) {
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Ignore escaped backslashes.
2015-02-05 16:23:03 +00:00
case ! backslash :
if ( options . double ) {
return backslash + backslash ;
} else {
return backslash ;
}
case ! nul :
return '\\x00' ;
case ! delimiter :
2016-11-28 14:05:51 +00:00
return ` \\ ${ delimiter } ` ;
2015-02-05 16:23:03 +00:00
case ! lf :
return '\\n' ;
case ! cr :
return '\\r' ;
case ! ls :
return '\\u2028' ;
case ! ps :
return '\\u2029' ;
case ! other :
if ( options . double ) {
2016-11-28 14:05:51 +00:00
return ` \\ ${ other } ` ;
2015-02-05 16:23:03 +00:00
} else {
return other ;
}
2010-11-09 04:07:51 +00:00
}
2010-10-04 18:30:48 +00:00
} ) ;
2016-11-28 14:05:51 +00:00
return ` ${ options . delimiter } ${ body } ${ options . delimiter } ` ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-09-18 22:16:39 +00:00
2017-09-01 14:09:36 +00:00
suppressSemicolons ( ) {
var ref , ref1 , results ;
results = [ ] ;
while ( this . value ( ) === ';' ) {
this . tokens . pop ( ) ;
if ( ref = ( ref1 = this . prev ( ) ) != null ? ref1 [ 0 ] : void 0 , indexOf . call ( [ '=' , ... UNFINISHED ] , ref ) >= 0 ) {
results . push ( this . error ( 'unexpected ;' ) ) ;
} else {
results . push ( void 0 ) ;
}
}
return results ;
}
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Throws an error at either a given offset from the current chunk or at the
// location of a token (`token[2]`).
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
error ( message , options = { } ) {
2017-04-06 17:06:45 +00:00
var first _column , first _line , location , ref , ref1 ;
location = 'first_line' in options ? options : ( [ first _line , first _column ] = this . getLineAndColumnFromChunk ( ( ref = options . offset ) != null ? ref : 0 ) , {
first _line ,
first _column ,
last _column : first _column + ( ( ref1 = options . length ) != null ? ref1 : 1 ) - 1
2013-03-05 04:13:46 +00:00
} ) ;
2015-02-06 09:52:02 +00:00
return throwSyntaxError ( message , location ) ;
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
}
2011-12-14 15:39:20 +00:00
[CS2] Compile class constructors to ES2015 classes (#4354)
* Compile classes to ES2015 classes
Rather than compiling classes to named functions with prototype and
class assignments, they are now compiled to ES2015 class declarations.
Backwards compatibility has been maintained by compiling ES2015-
incompatible properties as prototype or class assignments. `super`
continues to be compiled as before.
Where possible, classes will be compiled "bare", without an enclosing
IIFE. This is possible when the class contains only ES2015 compatible
expressions (methods and static methods), and has no parent (this last
constraint is a result of the legacy `super` compilation, and could be
removed once ES2015 `super` is being used). Classes are still assigned
to variables to maintain compatibility for assigned class expressions.
There are a few changes to existing functionality that could break
backwards compatibility:
- Derived constructors that deliberately don't call `super` are no
longer possible. ES2015 derived classes can't use `this` unless the
parent constructor has been called, so it's now called implicitly when
not present.
- As a consequence of the above, derived constructors with @ parameters
or bound methods and explicit `super` calls are not allowed. The
implicit `super` must be used in these cases.
* Add tests to verify class interoperability with ES
* Refactor class nodes to separate executable body logic
Logic has been redistributed amongst the class nodes so that:
- `Class` contains the logic necessary to compile an ES class
declaration.
- `ExecutableClassBody` contains the logic necessary to compile CS'
class extensions that require an executable class body.
`Class` still necessarily contains logic to determine whether an
expression is valid in an ES class initializer or not. If any invalid
expressions are found then `Class` will wrap itself in an
`ExecutableClassBody` when compiling.
* Rename `Code#static` to `Code#isStatic`
This naming is more consistent with other `Code` flags.
* Output anonymous classes when possible
Anonymous classes can be output when:
- The class has no parent. The current super compilation needs a class
variable to reference. This condition will go away when ES2015 super
is in use.
- The class contains no bound static methods. Bound static methods have
their context set to the class name.
* Throw errors at compile time for async or generator constructors
* Improve handling of anonymous classes
Anonymous classes are now always anonymous. If a name is required (e.g.
for bound static methods or derived classes) then the class is compiled
in an `ExecutableClassBody` which will give the anonymous class a stable
reference.
* Add a `replaceInContext` method to `Node`
`replaceInContext` will traverse children looking for a node for which
`match` returns true. Once found, the matching node will be replaced by
the result of calling `replacement`.
* Separate `this` assignments from function parameters
This change has been made to simplify two future changes:
1. Outputting `@`-param assignments after a `super` call.
In this case it is necessary that non-`@` parameters are available
before `super` is called, so destructuring has to happen before
`this` assignment.
2. Compiling destructured assignment to ES6
In this case also destructuring has to happen before `this`,
as destructuring can happen in the arguments list, but `this`
assignment can not.
A bonus side-effect is that default values for `@` params are now output
as ES6 default parameters, e.g.
(@a = 1) ->
becomes
function a (a = 1) {
this.a = a;
}
* Change `super` handling in class constructors
Inside an ES derived constructor (a constructor for a class that extends
another class), it is impossible to access `this` until `super` has been
called. This conflicts with CoffeeScript's `@`-param and bound method
features, which compile to `this` references at the top of a function
body. For example:
class B extends A
constructor: (@param) -> super
method: =>
This would compile to something like:
class B extends A {
constructor (param) {
this.param = param;
this.method = bind(this.method, this);
super(...arguments);
}
}
This would break in an ES-compliant runtime as there are `this`
references before the call to `super`. Before this commit we were
dealing with this by injecting an implicit `super` call into derived
constructors that do not already have an explicit `super` call.
Furthermore, we would disallow explicit `super` calls in derived
constructors that used bound methods or `@`-params, meaning the above
example would need to be rewritten as:
class B extends A
constructor: (@param) ->
method: =>
This would result in a call to `super(...arguments)` being generated as
the first expression in `B#constructor`.
Whilst this approach seems to work pretty well, and is arguably more
convenient than having to manually call `super` when you don't
particularly care about the arguments, it does introduce some 'magic'
and separation from ES, and would likely be a pain point in a project
that made use of significant constructor overriding.
This commit introduces a mechanism through which `super` in constructors
is 'expanded' to include any generated `this` assignments, whilst
retaining the same semantics of a super call. The first example above
now compiles to something like:
class B extends A {
constructor (param) {
var ref
ref = super(...arguments), this.param = param, this.method = bind(this.method, this), ref;
}
}
* Improve `super` handling in constructors
Rather than functions expanding their `super` calls, the `SuperCall`
node can now be given a list of `thisAssignments` to apply when it is
compiled.
This allows us to use the normal compiler machinery to determine whether
the `super` result needs to be cached, whether it appears inline or not,
etc.
* Fix anonymous classes at the top level
Anonymous classes in ES are only valid within expressions. If an
anonymous class is at the top level it will now be wrapped in
parenthses to force it into an expression.
* Re-add Parens wrapper around executable class bodies
This was lost in the refactoring, but it necessary to ensure
`new class ...` works as expected when there's an executable body.
* Throw compiler errors for badly configured derived constructors
Rather than letting them become runtime errors, the following checks are
now performed when compiling a derived constructor:
- The constructor **must** include a call to `super`.
- The constructor **must not** reference `this` in the function body
before `super` has been called.
* Add some tests exercising new class behaviour
- async methods in classes
- `this` access after `super` in extended classes
- constructor super in arrow functions
- constructor functions can't be async
- constructor functions can't be generators
- derived constructors must call super
- derived constructors can't reference `this` before calling super
- generator methods in classes
- 'new' target
* Improve constructor `super` errors
Add a check for `super` in non-extended class constructors, and
explicitly mention derived constructors in the "can't reference this
before super" error.
* Fix compilation of multiple `super` paths in derived constructors
`super` can only be called once, but it can be called conditionally from
multiple locations. The chosen fix is to add the `this` assignments to
every super call.
* Additional class tests, added as a separate file to simplify testing and merging.
Some methods are commented out because they currently throw and I'm not sure how
to test for compilation errors like those.
There is also one test which I deliberately left without passing, `super` in an external prototype override.
This test should 'pass' but is really a variation on the failing `super only allowed in an instance method`
tests above it.
* Changes to the tests. Found bug in super in prototype method. fixed.
* Added failing test back in, dealing with bound functions in external prototype overrides.
* Located a bug in the compiler relating to assertions and escaped ES6 classes.
* Move tests from classes-additional.coffee into classes.coffee; comment out console.log
* Cleaned up tests and made changes based on feedback. Test at the end still has issues, but it's commented out for now.
* Make HoistTarget.expand recursive
It's possible that a hoisted node may itself contain hoisted nodes (e.g.
a class method inside a class method). For this to work the hoisted
fragments need to be expanded recursively.
* Uncomment final test in classes.coffee
The test case now compiles, however another issue is affecting the test
due to the error for `this` before `super` triggering based on source
order rather than execution order. These have been commented out for
now.
* Fixed last test TODOs in test/classes.coffee
Turns out an expression like `this.foo = super()` won't run in JS as it
attempts to lookup `this` before evaluating `super` (i.e. throws "this
is not defined").
* Added more tests for compatability checks, statics, prototypes and ES6 expectations. Cleaned test "nested classes with super".
* Changes to reflect feedback and to comment out issues that will be addressed seperately.
* Clean up test/classes.coffee
- Trim trailing whitespace.
- Rephrase a condition to be more idiomatic.
* Remove check for `super` in derived constructors
In order to be usable at runtime, an extended ES class must call `super`
OR return an alternative object. This check prevented the latter case,
and checking for an alternative return can't be completed statically
without control flow analysis.
* Disallow 'super' in constructor parameter defaults
There are many edge cases when combining 'super' in parameter defaults
with @-parameters and bound functions (and potentially property
initializers in the future).
Rather than attempting to resolve these edge cases, 'super' is now
explicitly disallowed in constructor parameter defaults.
* Disallow @-params in derived constructors without 'super'
@-parameters can't be assigned unless 'super' is called.
2017-01-13 05:55:30 +00:00
} ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Helper functions
// ----------------
[CS2] Output ES2015 arrow functions, default parameters, rest parameters (#4311)
* Eliminate wrapper around “bound” (arrow) functions; output `=>` for such functions
* Remove irrelevant (and breaking) tests
* Minor cleanup
* When a function parameter is a splat (i.e., it uses the ES2015 rest parameter syntax) output that parameter as ES2015
* Rearrange function parameters when one of the parameters is a splat and isn’t the last parameter (very WIP)
* Handle params like `@param`, adding assignment expressions for them when they appear; ensure splat parameter is last
* Add parameter names (not a text like `'\nValue IdentifierLiteral: a'`) to the scope, so that parameters can’t be deleted; move body-related lines together; more explanation of what’s going on
* For parameters with a default value, correctly add the parameter name to the function scope
* Handle expansions in function parameters: when an expansion is found, set the parameters to only be the original parameters left of the expansion, then an `...args` parameter; and in the function body define variables for the parameters to the right of the expansion, including setting default values
* Handle splat parameters the same way we handle expansions: if a splat parameter is found, it becomes the last parameter in the function definition, and all following parameters get declared in the function body. Fix the splat/rest parameter values after the post-splat parameters have been extracted from it. Clean up `Code.compileNode` so that we loop through the parameters only once, and we create all expressions using calls like `new IdentifierLiteral` rather than `@makeCode`.
* Fix parameter name when a parameter is a splat attached to `this` (e.g. `@param...`)
* Rather than assigning post-splat parameters based on index, use slice; passes test “Functions with splats being called with too few arguments”
* Dial back our w00t indentation
* Better parsing of parameter names (WIP)
* Refactor processing of splat/expansion parameters
* Fix assignment of default parameters for parameters that come after a splat
* Better check for whether a param is attached to `this`
* More understandable variable names
* For parameters after a splat or expansion, assign them similar to the 1.x destructuring method of using `arguments`, except only concern ourselves with the post-splat parameters instead of all parameters; and use the splat/expansion parameter name, since `arguments` in ES fat arrow functions refers to the parent function’s `arguments` rather than the fat arrow function’s arguments/parameters
* Don’t add unnamed parameters (like `[]` as a parameter) to the function scope
* Disallow multiple splat/expansion parameters in function definitions; disallow lone expansion parameters
* Fix `this` params not getting assigned if the parameter is after a splat parameter
* Allow names of function parameters attached to `this` to be reserved words
* Always add a statement to the function body defining a variable with its default value, if it has one, if the variable `== null`; this covers the case when ES doesn’t apply the default value when `null` is passed in as a value, but CoffeeScript expects `null` and `undefined` to act interchangeably
* Aftermath of having both `undefined` and `null` trigger the use of default values for parameters with default values
* More careful parsing of destructured parameters
* Fall back to processing destructured parameters in the function body, to account for `this` or default values within destructured objects
* Clean up comments
* Restore new bare function test, minus the arrow function part of it
* Test that bound/arrow functions aren’t overwriting the `arguments` object, which should refer to the parent scope’s `arguments` (like `this`)
* Follow ES2015 spec for parameter default values: `null` gets assigned as as `null`, not the default value
* Mimic ES default parameters behavior for parameters after a splat or expansion parameter
* Bound functions cannot be generators: remove no-longer-relevant test, add check to throw error if `yield` appears inside a bound (arrow) function
* Error for bound generator functions should underline the `yield`
2016-10-26 05:26:13 +00:00
isUnassignable = function ( name , displayName = name ) {
2016-03-05 19:59:39 +00:00
switch ( false ) {
Compile splats in arrays and function calls to ES2015 splats (#4353)
Rather than compiling splats to arrays built using `Array#concat`, splats
are now compiled directly to ES2015 splats, e.g.
f foo, arguments..., bar
[ foo, arguments..., bar ]
Which used to be compiled to:
f.apply(null, [foo].concat(slice.call(arguments), [bar]));
[foo].concat(slice.call(arguments), [bar]);
Is now compiled to:
f(foo, ...arguments, bar);
[ foo, ...arguments, bar ];
2016-11-06 16:30:04 +00:00
case indexOf . call ( [ ... JS _KEYWORDS , ... COFFEE _KEYWORDS ] , name ) < 0 :
2016-11-28 14:05:51 +00:00
return ` keyword ' ${ displayName } ' can't be assigned ` ;
2016-03-05 19:59:39 +00:00
case indexOf . call ( STRICT _PROSCRIBED , name ) < 0 :
2016-11-28 14:05:51 +00:00
return ` ' ${ displayName } ' can't be assigned ` ;
2016-03-05 19:59:39 +00:00
case indexOf . call ( RESERVED , name ) < 0 :
2016-11-28 14:05:51 +00:00
return ` reserved word ' ${ displayName } ' can't be assigned ` ;
2016-03-05 19:59:39 +00:00
default :
return false ;
}
} ;
exports . isUnassignable = isUnassignable ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// `from` isn’ t a CoffeeScript keyword, but it behaves like one in `import` and
// `export` statements (handled above) and in the declaration line of a `for`
// loop. Try to detect when `from` is a variable identifier and when it is this
// “sometimes” keyword.
2016-12-06 20:29:02 +00:00
isForFrom = function ( prev ) {
2017-04-06 17:06:45 +00:00
var ref ;
2016-12-06 20:29:02 +00:00
if ( prev [ 0 ] === 'IDENTIFIER' ) {
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// `for i from from`, `for from from iterable`
2016-12-06 20:29:02 +00:00
if ( prev [ 1 ] === 'from' ) {
prev [ 1 ] [ 0 ] = 'IDENTIFIER' ;
true ;
}
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// `for i from iterable`
2016-12-06 20:29:02 +00:00
return true ;
2018-01-31 14:33:17 +00:00
// `for from…`
} else if ( prev [ 0 ] === 'FOR' ) {
return false ;
// `for {from}…`, `for [from]…`, `for {a, from}…`, `for {a: from}…`
} else if ( ( ref = prev [ 1 ] ) === '{' || ref === '[' || ref === ',' || ref === ':' ) {
return false ;
2016-12-06 20:29:02 +00:00
} else {
2018-01-31 14:33:17 +00:00
return true ;
2016-12-06 20:29:02 +00:00
}
} ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Constants
// ---------
// Keywords that CoffeeScript shares in common with JavaScript.
2016-11-02 15:51:26 +00:00
JS _KEYWORDS = [ 'true' , 'false' , 'null' , 'this' , 'new' , 'delete' , 'typeof' , 'in' , 'instanceof' , 'return' , 'throw' , 'break' , 'continue' , 'debugger' , 'yield' , 'await' , 'if' , 'else' , 'switch' , 'for' , 'while' , 'do' , 'try' , 'catch' , 'finally' , 'class' , 'extends' , 'super' , 'import' , 'export' , 'default' ] ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// CoffeeScript-only keywords.
2016-03-05 20:32:20 +00:00
COFFEE _KEYWORDS = [ 'undefined' , 'Infinity' , 'NaN' , 'then' , 'unless' , 'until' , 'loop' , 'of' , 'by' , 'when' ] ;
2011-09-18 22:16:39 +00:00
2011-04-30 14:35:56 +00:00
COFFEE _ALIAS _MAP = {
2010-10-05 19:46:17 +00:00
and : '&&' ,
or : '||' ,
is : '==' ,
isnt : '!=' ,
2010-10-07 15:56:01 +00:00
not : '!' ,
2010-10-23 18:24:30 +00:00
yes : 'true' ,
no : 'false' ,
on : 'true' ,
off : 'false'
2011-04-30 14:35:56 +00:00
} ;
2011-09-18 22:16:39 +00:00
2011-04-30 14:35:56 +00:00
COFFEE _ALIASES = ( function ( ) {
2015-01-30 19:33:03 +00:00
var results ;
results = [ ] ;
2011-04-30 14:35:56 +00:00
for ( key in COFFEE _ALIAS _MAP ) {
2015-01-30 19:33:03 +00:00
results . push ( key ) ;
2011-04-30 14:35:56 +00:00
}
2015-01-30 19:33:03 +00:00
return results ;
2011-04-30 14:35:56 +00:00
} ) ( ) ;
2011-09-18 22:16:39 +00:00
2011-04-30 14:35:56 +00:00
COFFEE _KEYWORDS = COFFEE _KEYWORDS . concat ( COFFEE _ALIASES ) ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// The list of keywords that are reserved by JavaScript, but not used, or are
// used by CoffeeScript internally. We throw an error when these are encountered,
// to avoid having a JavaScript error at runtime.
Support import and export of ES2015 modules (#4300)
This pull request adds support for ES2015 modules, by recognizing `import` and `export` statements. The following syntaxes are supported, based on the MDN [import](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import) and [export](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/export) pages:
```js
import "module-name"
import defaultMember from "module-name"
import * as name from "module-name"
import { } from "module-name"
import { member } from "module-name"
import { member as alias } from "module-name"
import { member1, member2 as alias2, … } from "module-name"
import defaultMember, * as name from "module-name"
import defaultMember, { … } from "module-name"
export default expression
export class name
export { }
export { name }
export { name as exportedName }
export { name as default }
export { name1, name2 as exportedName2, name3 as default, … }
export * from "module-name"
export { … } from "module-name"
```
As a subsitute for ECMAScript’s `export var name = …` and `export function name {}`, CoffeeScript also supports:
```js
export name = …
```
CoffeeScript also supports optional commas within `{ … }`.
This PR converts the supported `import` and `export` statements into ES2015 `import` and `export` statements; it **does not resolve the modules**. So any CoffeeScript with `import` or `export` statements will be output as ES2015, and will need to be transpiled by another tool such as Babel before it can be used in a browser. We will need to add a warning to the documentation explaining this.
This should be fully backwards-compatible, as `import` and `export` were previously reserved keywords. No flags are used.
There are extensive tests included, though because no current JavaScript runtime supports `import` or `export`, the tests compare strings of what the compiled CoffeeScript output is against what the expected ES2015 should be. I also conducted two more elaborate tests:
* I forked the [ember-piqu](https://github.com/pauc/piqu-ember) project, which was an Ember CLI app that used ember-cli-coffeescript and [ember-cli-coffees6](https://github.com/alexspeller/ember-cli-coffees6) (which adds “support” for `import`/`export` by wrapping such statements in backticks before passing the result to the CoffeeScript compiler). I removed `ember-cli-coffees6` and replaced the CoffeeScript compiler used in the build chain with this code, and the app built without errors. [Demo here.](https://github.com/GeoffreyBooth/coffeescript-modules-test-piqu)
* I also forked the [CoffeeScript version of Meteor’s Todos example app](https://github.com/meteor/todos/tree/coffeescript), and replaced all of its `require` statements with the `import` and `export` statements from the original ES2015 version of the app on its `master` branch. I then updated the `coffeescript` Meteor package in the app to use this new code, and again the app builds without errors. [Demo here.](https://github.com/GeoffreyBooth/coffeescript-modules-test-meteor-todos)
The discussion history for this work started [here](https://github.com/jashkenas/coffeescript/pull/4160) and continued [here](https://github.com/GeoffreyBooth/coffeescript/pull/2). @lydell provided guidance, and @JimPanic and @rattrayalex contributed essential code.
2016-09-14 18:46:05 +00:00
RESERVED = [ 'case' , 'function' , 'var' , 'void' , 'with' , 'const' , 'let' , 'enum' , 'native' , 'implements' , 'interface' , 'package' , 'private' , 'protected' , 'public' , 'static' ] ;
2012-01-11 23:04:14 +00:00
2016-03-05 19:59:39 +00:00
STRICT _PROSCRIBED = [ 'arguments' , 'eval' ] ;
2012-01-11 23:04:14 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// The superset of both JavaScript keywords and reserved words, none of which may
// be used as identifiers or properties.
2016-03-05 19:59:39 +00:00
exports . JS _FORBIDDEN = JS _KEYWORDS . concat ( RESERVED ) . concat ( STRICT _PROSCRIBED ) ;
Refactor `Literal` into several subtypes
Previously, the parser created `Literal` nodes for many things. This resulted in
information loss. Instead of being able to check the node type, we had to use
regexes to tell the different types of `Literal`s apart. That was a bit like
parsing literals twice: Once in the lexer, and once (or more) in the compiler.
It also caused problems, such as `` `this` `` and `this` being indistinguishable
(fixes #2009).
Instead returning `new Literal` in the grammar, subtypes of it are now returned
instead, such as `NumberLiteral`, `StringLiteral` and `IdentifierLiteral`. `new
Literal` by itself is only used to represent code chunks that fit no category.
(While mentioning `NumberLiteral`, there's also `InfinityLiteral` now, which is
a subtype of `NumberLiteral`.)
`StringWithInterpolations` has been added as a subtype of `Parens`, and
`RegexWithInterpolations` as a subtype of `Call`. This makes it easier for other
programs to make use of CoffeeScript's "AST" (nodes). For example, it is now
possible to distinguish between `"a #{b} c"` and `"a " + b + " c"`. Fixes #4192.
`SuperCall` has been added as a subtype of `Call`.
Note, though, that some information is still lost, especially in the lexer. For
example, there is no way to distinguish a heredoc from a regular string, or a
heregex without interpolations from a regular regex. Binary and octal number
literals are indistinguishable from hexadecimal literals.
After the new subtypes were added, they were taken advantage of, removing most
regexes in nodes.coffee. `SIMPLENUM` (which matches non-hex integers) had to be
kept, though, because such numbers need special handling in JavaScript (for
example in `1..toString()`).
An especially nice hack to get rid of was using `new String()` for the token
value for reserved identifiers (to be able to set a property on them which could
survive through the parser). Now it's a good old regular string.
In range literals, slices, splices and for loop steps when number literals
are involved, CoffeeScript can do some optimizations, such as precomputing the
value of, say, `5 - 3` (outputting `2` instead of `5 - 3` literally). As a side
bonus, this now also works with hexadecimal number literals, such as `0x02`.
Finally, this also improves the output of `coffee --nodes`:
# Before:
$ bin/coffee -ne 'while true
"#{a}"
break'
Block
While
Value
Bool
Block
Value
Parens
Block
Op +
Value """"
Value
Parens
Block
Value "a" "break"
# After:
$ bin/coffee -ne 'while true
"#{a}"
break'
Block
While
Value BooleanLiteral: true
Block
Value
StringWithInterpolations
Block
Op +
Value StringLiteral: ""
Value
Parens
Block
Value IdentifierLiteral: a
StatementLiteral: break
2016-01-31 19:24:31 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// The character code of the nasty Microsoft madness otherwise known as the BOM.
2013-01-07 04:56:58 +00:00
BOM = 65279 ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Token matching regexes.
IDENTIFIER = /^(?!\d)((?:(?!\s)[$\w\x7f-\uffff])+)([^\n\S]*:(?!:))?/ ; // Is this a property name?
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
CSX _IDENTIFIER = /^(?![\d<])((?:(?!\s)[\.\-$\w\x7f-\uffff])+)/ ; // Must not start with `<`.
// Like `IDENTIFIER`, but includes `-`s and `.`s.
2017-06-07 06:33:46 +00:00
2017-12-09 06:41:03 +00:00
// Fragment: <></>
CSX _FRAGMENT _IDENTIFIER = /^()>/ ; // Ends immediately with `>`.
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
CSX _ATTRIBUTE = /^(?!\d)((?:(?!\s)[\-$\w\x7f-\uffff])+)([^\S]*=(?!=))?/ ; // Like `IDENTIFIER`, but includes `-`s.
// Is this an attribute with a value?
2017-06-07 06:33:46 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
NUMBER = /^0b[01]+|^0o[0-7]+|^0x[\da-f]+|^\d*\.?\d+(?:e[+-]?\d+)?/i ; // binary
// octal
// hex
// decimal
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
OPERATOR = /^(?:[-=]>|[-+*\/%<>&|^!?=]=|>>>=?|([-+:])\1|([&|<>*\/%])\2=?|\?(\.|::)|\.{2,3})/ ; // function
// compound assign / compare
// zero-fill right shift
// doubles
// logic / shift / power / floor division / modulo
// soak access
// range or splat
2011-09-18 22:16:39 +00:00
2010-11-09 13:25:48 +00:00
WHITESPACE = /^[^\n\S]+/ ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
COMMENT = /^\s*###([^#][\s\S]*?)(?:###[^\n\S]*|###$)|^(?:\s*#(?!##[^#]).*)+/ ;
2011-09-18 22:16:39 +00:00
2014-09-06 10:55:27 +00:00
CODE = /^[-=]>/ ;
2011-09-18 22:16:39 +00:00
2010-11-09 13:25:48 +00:00
MULTI _DENT = /^(?:\n[^\n\S]*)+/ ;
2011-09-18 22:16:39 +00:00
2016-11-19 19:13:30 +00:00
JSTOKEN = /^`(?!``)((?:[^`\\]|\\[\s\S])*)`/ ;
HERE _JSTOKEN = /^```((?:[^`\\]|\\[\s\S]|`(?!``))*)```/ ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// String-matching-regexes.
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
STRING _START = /^(?:'''|"""|'|")/ ;
STRING _SINGLE = /^(?:[^\\']|\\[\s\S])*/ ;
STRING _DOUBLE = /^(?:[^\\"#]|\\[\s\S]|\#(?!\{))*/ ;
HEREDOC _SINGLE = /^(?:[^\\']|\\[\s\S]|'(?!''))*/ ;
HEREDOC _DOUBLE = /^(?:[^\\"#]|\\[\s\S]|"(?!"")|\#(?!\{))*/ ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
INSIDE _CSX = /^(?:[^\{<])*/ ; // Start of CoffeeScript interpolation. // Similar to `HEREDOC_DOUBLE` but there is no escaping.
// Maybe CSX tag (`<` not allowed even if bare).
2017-06-07 06:33:46 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
CSX _INTERPOLATION = /^(?:\{|<(?!\/))/ ; // CoffeeScript interpolation.
// CSX opening tag.
2017-06-07 06:33:46 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
STRING _OMIT = /((?:\\\\)+)|\\[^\S\n]*\n\s*/g ; // Consume (and preserve) an even number of backslashes.
// Remove escaped newlines.
2015-02-05 16:23:03 +00:00
SIMPLE _STRING _OMIT = /\s*\n\s*/g ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
HEREDOC _INDENT = /\n+([^\n\S]*)(?=\S)/g ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Regex-matching-regexes.
REGEX = /^\/(?!\/)((?:[^[\/\n\\]|\\[^\n]|\[(?:\\[^\n]|[^\]\n\\])*\])*)(\/)?/ ; // Every other thing.
// Anything but newlines escaped.
// Character class.
2011-09-18 22:16:39 +00:00
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
REGEX _FLAGS = /^\w*/ ;
2017-04-20 06:03:06 +00:00
VALID _FLAGS = /^(?!.*(.).*\1)[imguy]*$/ ;
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
2017-12-03 19:55:26 +00:00
// Match any character, except those that need special handling below.
// Match `\` followed by any character.
// Match any `/` except `///`.
// Match `#` which is not part of interpolation, e.g. `#{}`.
// Comments consume everything until the end of the line, including `///`.
HEREGEX = /^(?:[^\\\/#\s]|\\[\s\S]|\/(?!\/\/)|\#(?!\{)|\s+(?:#(?!\{).*)?)*/ ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
HEREGEX _OMIT = /((?:\\\\)+)|\\(\s)|\s+(?:#.*)?/g ; // Consume (and preserve) an even number of backslashes.
// Preserve escaped whitespace.
// Remove whitespace and comments.
2011-09-18 22:16:39 +00:00
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
REGEX _ILLEGAL = /^(\/|\/{3}\s*)(\*)/ ;
2011-09-18 22:16:39 +00:00
2015-01-10 00:48:00 +00:00
POSSIBLY _DIVISION = /^\/=?\s/ ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Other regexes.
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
HERECOMMENT _ILLEGAL = /\*\// ;
2011-09-18 22:16:39 +00:00
2011-02-27 07:11:35 +00:00
LINE _CONTINUER = /^\s*(?:,|\??\.(?![.\d])|::)/ ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
STRING _INVALID _ESCAPE = /((?:^|[^\\])(?:\\\\)*)\\(?:(0[0-7]|[1-7])|(x(?![\da-fA-F]{2}).{0,2})|(u\{(?![\da-fA-F]{1,}\})[^}]*\}?)|(u(?!\{|[\da-fA-F]{4}).{0,4}))/ ; // Make sure the escape isn’ t escaped.
// octal escape
// hex escape
// unicode code point escape
// unicode escape
2017-04-20 06:03:06 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
REGEX _INVALID _ESCAPE = /((?:^|[^\\])(?:\\\\)*)\\(?:(0[0-7])|(x(?![\da-fA-F]{2}).{0,2})|(u\{(?![\da-fA-F]{1,}\})[^}]*\}?)|(u(?!\{|[\da-fA-F]{4}).{0,4}))/ ; // Make sure the escape isn’ t escaped.
// octal escape
// hex escape
// unicode code point escape
// unicode escape
2017-04-06 23:39:13 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
UNICODE _CODE _POINT _ESCAPE = /(\\\\)|\\u\{([\da-fA-F]+)\}/g ; // Make sure the escape isn’ t escaped.
Refactor interpolation (and string and regex) handling in lexer
- Fix #3394: Unclosed single-quoted strings (both regular ones and heredocs)
used to pass through the lexer, causing a parsing error later, while
double-quoted strings caused an error already in the lexing phase. Now both
single and double-quoted unclosed strings error out in the lexer (which is the
more logical option) with consistent error messages. This also fixes the last
comment by @satyr in #3301.
- Similar to the above, unclosed heregexes also used to pass through the lexer
and not error until in the parsing phase, which resulted in confusing error
messages. This has been fixed, too.
- Fix #3348, by adding passing tests.
- Fix #3529: If a string starts with an interpolation, an empty string is no
longer emitted before the interpolation (unless it is needed to coerce the
interpolation into a string).
- Block comments cannot contain `*/`. Now the error message also shows exactly
where the offending `*/`. This improvement might seem unrelated, but I had to
touch that code anyway to refactor string and regex related code, and the
change was very trivial. Moreover, it's consistent with the next two points.
- Regexes cannot start with `*`. Now the error message also shows exactly where
the offending `*` is. (It might actually not be exatly at the start in
heregexes.) It is a very minor improvement, but it was trivial to add.
- Octal escapes in strings are forbidden in CoffeeScript (just like in
JavaScript strict mode). However, this used to be the case only for regular
strings. Now they are also forbidden in heredocs. Moreover, the errors now
point at the offending octal escape.
- Invalid regex flags are no longer allowed. This includes repeated modifiers
and unknown ones. Moreover, invalid modifiers do not stop a heregex from
being matched, which results in better error messages.
- Fix #3621: `///a#{1}///` compiles to `RegExp("a" + 1)`. So does
`RegExp("a#{1}")`. Still, those two code snippets used to generate different
tokens, which is a bit weird, but more importantly causes problems for
coffeelint (see clutchski/coffeelint#340). This required lots of tests in
test/location.coffee to be updated. Note that some updates to those tests are
unrelated to this point; some have been updated to be more consistent (I
discovered this because the refactored code happened to be seemingly more
correct).
- Regular regex literals used to erraneously allow newlines to be escaped,
causing invalid JavaScript output. This has been fixed.
- Heregexes may now be completely empty (`//////`), instead of erroring out with
a confusing message.
- Fix #2388: Heredocs and heregexes used to be lexed simply, which meant that
you couldn't nest a heredoc within a heredoc (double-quoted, that is) or a
heregex inside a heregex.
- Fix #2321: If you used division inside interpolation and then a slash later in
the string containing that interpolation, the division slash and the latter
slash was erraneously matched as a regex. This has been fixed.
- Indentation inside interpolations in heredocs no longer affect how much
indentation is removed from each line of the heredoc (which is more
intuitive).
- Whitespace is now correctly trimmed from the start and end of strings in a few
edge cases.
- Last but not least, the lexing of interpolated strings now seems to be more
efficient. For a regular double-quoted string, we used to use a custom
function to find the end of it (taking interpolations and interpolations
within interpolations etc. into account). Then we used to re-find the
interpolations and recursively lex their contents. In effect, the same string
was processed twice, or even more in the case of deeper nesting of
interpolations. Now the same string is processed just once.
- Code duplication between regular strings, heredocs, regular regexes and
heregexes has been reduced.
- The above two points should result in more easily read code, too.
2015-01-03 22:40:43 +00:00
LEADING _BLANK _LINE = /^[^\n\S]*\n/ ;
TRAILING _BLANK _LINE = /\n[^\n\S]*$/ ;
2010-10-03 23:22:42 +00:00
TRAILING _SPACES = /\s+$/ ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Compound assignment tokens.
2013-03-25 06:19:05 +00:00
COMPOUND _ASSIGN = [ '-=' , '+=' , '/=' , '*=' , '%=' , '||=' , '&&=' , '?=' , '<<=' , '>>=' , '>>>=' , '&=' , '^=' , '|=' , '**=' , '//=' , '%%=' ] ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Unary tokens.
2014-09-06 11:53:21 +00:00
UNARY = [ 'NEW' , 'TYPEOF' , 'DELETE' , 'DO' ] ;
2011-09-18 22:16:39 +00:00
2013-03-25 03:05:04 +00:00
UNARY _MATH = [ '!' , '~' ] ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Bit-shifting tokens.
2010-08-12 02:24:43 +00:00
SHIFT = [ '<<' , '>>' , '>>>' ] ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Comparison tokens.
2010-10-23 18:24:30 +00:00
COMPARE = [ '==' , '!=' , '<' , '>' , '<=' , '>=' ] ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Mathematical tokens.
2013-03-25 06:19:05 +00:00
MATH = [ '*' , '/' , '%' , '//' , '%%' ] ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Relational tokens that are negatable with `not` prefix.
2010-10-05 15:43:44 +00:00
RELATION = [ 'IN' , 'OF' , 'INSTANCEOF' ] ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Boolean tokens.
2012-05-08 20:11:23 +00:00
BOOL = [ 'TRUE' , 'FALSE' ] ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Tokens which could legitimately be invoked or indexed. An opening
// parentheses or bracket following these tokens will be recorded as the start
// of a function invocation or indexing operation.
2016-03-05 16:41:15 +00:00
CALLABLE = [ 'IDENTIFIER' , 'PROPERTY' , ')' , ']' , '?' , '@' , 'THIS' , 'SUPER' ] ;
2011-09-18 22:16:39 +00:00
2016-03-05 20:32:20 +00:00
INDEXABLE = CALLABLE . concat ( [ 'NUMBER' , 'INFINITY' , 'NAN' , 'STRING' , 'STRING_END' , 'REGEX' , 'REGEX_END' , 'BOOL' , 'NULL' , 'UNDEFINED' , '}' , '::' ] ) ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Tokens which can be the left-hand side of a less-than comparison, i.e. `a<b`.
2017-06-07 06:33:46 +00:00
COMPARABLE _LEFT _SIDE = [ 'IDENTIFIER' , ')' , ']' , 'NUMBER' ] ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Tokens which a regular expression will never immediately follow (except spaced
// CALLABLEs in some cases), but which a division operator can.
// See: http://www-archive.mozilla.org/js/language/js20-2002-04/rationale/syntax.html#regular-expressions
2015-01-10 00:48:00 +00:00
NOT _REGEX = INDEXABLE . concat ( [ '++' , '--' ] ) ;
2011-09-18 22:16:39 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Tokens that, when immediately preceding a `WHEN`, indicate that the `WHEN`
// occurs at the start of a line. We disambiguate these from trailing whens to
// avoid an ambiguity in the grammar.
2010-03-20 04:58:25 +00:00
LINE _BREAK = [ 'INDENT' , 'OUTDENT' , 'TERMINATOR' ] ;
2011-12-14 15:39:20 +00:00
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Additional indent in front of these is ignored.
2014-01-22 18:30:13 +00:00
INDENTABLE _CLOSERS = [ ')' , '}' , ']' ] ;
[CS2] Comments (#4572)
* Make `addLocationDataFn` more DRY
* Style fixes
* Provide access to full parser inside our custom function running in parser.js; rename the function to lay the groundwork for adding data aside from location data
* Fix style.
* Fix style.
* Label test comments
* Update grammar to remove comment tokens; update DSL to call new helper function that preserves comments through parsing
* New implementation of compiling block comments: the lexer pulls them out of the token stream, attaching them as a property to a token; the rewriter moves the attachment around so it lives on a token that is destined to make it through to compilation (and in a good placement); and the nodes render the block comment. All tests but one pass (commented out).
* If a comment follows a class declaration, move the comment inside the class body
* Style
* Improve indentation of multiline comments
* Fix indentation for block comments, at least in the cases covered by the one failing test
* Don’t reverse the order of unshifted comments
* Simplify rewriter’s handling of comments, generalizing the special case
* Expand the list of tokens we need to avoid for passing comments through the parser; get some literal tokens to have nodes created for them so that the comments pass through
* Improve comments; fix multiline flag
* Prepare HereComments for processing line comments
* Line comments, first draft: the tests pass, but the line comments aren’t indented and sometimes trail previous lines when they shouldn’t; updated compiler output in following commit
* Updated compiler, now with line comments
* `process` doesn’t exist in the browser, so we should check for its existence first
* Update parser output
* Test that proves #4290 is fixed
* Indent line comments, first pass
* Compiled output with indented line comments
* Comments that start a new line shouldn’t trail; don’t skip comments attached to generated tokens; stop looking for indentation once we hit a newline
* Revised output
* Cleanup
* Split “multiline” line comment tokens, shifting them forward or back as appropriate
* Fix comments in module specifiers
* Abstract attaching comments to a node
* Line comments in interpolated strings
* Line comments can’t be multiline anymore
* Improve handling of blank lines and indentation of following comments that start a new line (i.e. don’t trail)
* Make comments compilation more object-oriented
* Remove lots of dead code that we don’t need anymore because a comment is never a node, only a fragment
* Improve eqJS helper
* Fix #4290 definitively, with improved output for arrays with interspersed block comments
* Add support for line comments output interspersed within arrays
* Fix mistake, don’t lose the variable we’re working on
* Remove redundant replacements
* Check for indentation only from the start of the string
* Indentations in generated JS are always multiples of two spaces (never tabs) so just look for 2+ spaces
* Update package versions; run Babel twice, once for each preset, temporarily until a Babili bug is fixed that prevents it from running with the env preset
* Don’t rely on `fragment.type`, which can break when the compiler is minified
* Updated generated docs and browser compiler
* Output block comments after function arguments
* Comments appear above scope `var` declarations; better tracking of generated `JS` tokens created only to shepherd comments through to the output
* Create new FuncGlyph node, to hold comments we want to output near the function parameters
* Block comments between `)` and `->`/`=>` get output between `)` and `{`.
* Fix indentation of comments that are the first line inside a bare mode block
* Updated output
* Full Flow example
* Updated browser compiler
* Abstract and organize comment fragment generation code; store more properties on the comment fragment objects; make `throw` behave like `return`
* Abstract token insertion code
* Add missing locationData to STRING_START token, giving it the locationData of the overall StringWithInterpolations token so that comments attached to STRING_START end up on the StringWithInterpolations node
* Allow `SUPER` tokens to carry comments
* Rescue comments from `Existence` nodes and `If` nodes’ conditions
* Rescue comments after `\` line continuation tokens
* Updated compiled output
* Updated browser compiler
* Output block comments in the same `compileFragments` method as line comments, except for inline block comments
* Comments before splice
* Updated browser compiler
* Track compiledComments as a property of Base, to ensure that it’s not a global variable
* Docs: split up the Usage section
* Docs for type annotations via Flow; updated docs output
* Update regular comments documentation
* Updated browser compiler
* Comments before soak
* Comments before static methods, and probably before `@variable =` (this) assignments generally
* Comments before ‘if exists?’, refactor comment before ‘if this.var’ to be more precise, improve helper methods
* Comments before a method that contains ‘super()’ should output above the method property, not above the ‘super.method()’ call
* Fix missing comments before `if not` (i.e. before a UNARY token)
* Fix comments before ‘for’; add test for comment before assignment if (fixed in earlier commit)
* Comments within heregexes
* Updated browser compiler
* Update description to reflect what’s now happening in compileCommentFragments
* Preserve blank lines between line comments; output “whitespace-only” line comments as blank lines, rather than `//` following by whitespace
* Better future-proof comments tests
* Comments before object destructuring; abstract method for setting comments aside before compilation
* Handle more cases of comments before or after `for` loop declaration lines
* Fix indentation of comments preceding `for` loops
* Fix comment before splat function parameter
* Catch another RegexWithInterpolations comment edge case
* Updated browser compiler
* Change heregex example to one that’s more readable; update output
* Remove a few last references to the defunct HERECOMMENT token
* Abstract location hash creation into a function
* Improved clarity per code review notes
* Updated browser compiler
2017-08-03 02:34:34 +00:00
// Tokens that, when appearing at the end of a line, suppress a following TERMINATOR/INDENT token
2017-11-16 05:38:05 +00:00
UNFINISHED = [ '\\' , '.' , '?.' , '?::' , 'UNARY' , 'MATH' , 'UNARY_MATH' , '+' , '-' , '**' , 'SHIFT' , 'RELATION' , 'COMPARE' , '&' , '^' , '|' , '&&' , '||' , 'BIN?' , 'EXTENDS' ] ;
2017-07-09 17:28:50 +00:00
2010-09-21 07:53:58 +00:00
} ) . call ( this ) ;