diff --git a/README.md b/README.md index 151fcd46..4bf06221 100644 --- a/README.md +++ b/README.md @@ -21,30 +21,6 @@ to do its magic. Mutant does not have really good "getting started" documentation currently so please refer to presentations and blog posts below. -Mutation-Operators: -------------------- - -Mutant supports a wide range of mutation operators. An exhaustive list can be found in the [mutant-meta](https://github.com/mbj/mutant/tree/master/meta). -The `mutant-meta` is arranged to the AST-Node-Types of parser. Refer to parsers [AST documentation](https://github.com/whitequark/parser/blob/master/doc/AST_FORMAT.md) in doubt. - -There is no easy and universal way to count the number of mutation operators a tool supports. - -Presentations -------------- - -There are some presentations about mutant in the wild: - -* [RailsConf 2014](http://railsconf.com/) / http://confreaks.com/videos/3333-railsconf-mutation-testing-with-mutant -* [Wrocloverb 2014](http://wrocloverb.com/) / https://www.youtube.com/watch?v=rz-lFKEioLk -* [eurucamp 2013](http://2013.eurucamp.org/) / FrOSCon-2013 http://slid.es/markusschirp/mutation-testing -* [Cologne.rb](http://www.colognerb.de/topics/mutation-testing-mit-mutant) / https://github.com/DonSchado/colognerb-on-mutant/blob/master/mutation_testing_slides.pdf - -Blog-Posts ----------- - -* http://www.sitepoint.com/mutation-testing-mutant/ -* http://solnic.eu/2013/01/23/mutation-testing-with-mutant.html - Installation ------------ @@ -57,34 +33,6 @@ gem install mutant-rspec The minitest integration is still in the [works](https://github.com/mbj/mutant/pull/330). -The Crash / Stuck Problem (MRI) -------------------------------- - -Mutations generated by mutant can cause MRI to enter VM states its not prepared for. -All MRI versions > 1.9 and < 2.2.1 are affected by this depending on your compiler flags, -compiler version, and OS scheduling behavior. - -This can have the following unintended effects: - -* MRI crashes with a segfault. Mutant kills each mutation in a dedicated fork to isolate - the mutations side effects when this fork terminates abnormally (segfault) mutant - counts the mutation as killed. - -* MRI crashes with a segfault and gets stuck when handling the segfault. - Depending on the number of active kill jobs mutant might appear to continue normally until - all workers are stuck into this state when it begins to hang. - Currently mutant must assume that your test suite simply not terminated yet as from the outside - (parent process) the difference between a long running test and a stuck MRI is not observable. - Its planned to implement a timeout enforced from the parent process, but ideally MRI simply gets fixed. - -References: - -* [MRI fix](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636) -* [MRI backport to 2.2.1](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636) -* [Mutant issue](https://github.com/mbj/mutant/issues/265) -* [Upstream bug redmine](https://bugs.ruby-lang.org/issues/10460) -* [Upstream bug github](https://github.com/ruby/ruby/pull/822) - Examples -------- @@ -100,6 +48,31 @@ mutant --include lib --require virtus --use rspec Virtus::Attribute.build mutant --include lib --require virtus --use rspec Virtus::Attribute#type ``` +Rails +------- + +Assuming you are using rspec, you can mutation test Rails models by adding the following lines to your Gemfile: + +```ruby +group :test do + gem 'mutant-rspec' +end +``` + +Next, run bundle and comment out ```require 'rspec/autorun'``` from your spec_helper.rb file. Having done so you should be able to use commands like the following: + +```sh +RAILS_ENV=test bundle exec mutant -r ./config/environment --use rspec User +``` + +Mutation-Operators: +------------------- + +Mutant supports a wide range of mutation operators. An exhaustive list can be found in the [mutant-meta](https://github.com/mbj/mutant/tree/master/meta). +The `mutant-meta` is arranged to the AST-Node-Types of parser. Refer to parsers [AST documentation](https://github.com/whitequark/parser/blob/master/doc/AST_FORMAT.md) in doubt. + +There is no easy and universal way to count the number of mutation operators a tool supports. + Subjects -------- @@ -191,6 +164,50 @@ Mutation output is grouped by selection groups. Each group contains three sectio ----------------------- ``` +Presentations +------------- + +There are some presentations about mutant in the wild: + +* [RailsConf 2014](http://railsconf.com/) / http://confreaks.com/videos/3333-railsconf-mutation-testing-with-mutant +* [Wrocloverb 2014](http://wrocloverb.com/) / https://www.youtube.com/watch?v=rz-lFKEioLk +* [eurucamp 2013](http://2013.eurucamp.org/) / FrOSCon-2013 http://slid.es/markusschirp/mutation-testing +* [Cologne.rb](http://www.colognerb.de/topics/mutation-testing-mit-mutant) / https://github.com/DonSchado/colognerb-on-mutant/blob/master/mutation_testing_slides.pdf + +Blog-Posts +---------- + +* http://www.sitepoint.com/mutation-testing-mutant/ +* http://solnic.eu/2013/01/23/mutation-testing-with-mutant.html + +The Crash / Stuck Problem (MRI) +------------------------------- + +Mutations generated by mutant can cause MRI to enter VM states its not prepared for. +All MRI versions > 1.9 and < 2.2.1 are affected by this depending on your compiler flags, +compiler version, and OS scheduling behavior. + +This can have the following unintended effects: + +* MRI crashes with a segfault. Mutant kills each mutation in a dedicated fork to isolate + the mutations side effects when this fork terminates abnormally (segfault) mutant + counts the mutation as killed. + +* MRI crashes with a segfault and gets stuck when handling the segfault. + Depending on the number of active kill jobs mutant might appear to continue normally until + all workers are stuck into this state when it begins to hang. + Currently mutant must assume that your test suite simply not terminated yet as from the outside + (parent process) the difference between a long running test and a stuck MRI is not observable. + Its planned to implement a timeout enforced from the parent process, but ideally MRI simply gets fixed. + +References: + +* [MRI fix](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636) +* [MRI backport to 2.2.1](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636) +* [Mutant issue](https://github.com/mbj/mutant/issues/265) +* [Upstream bug redmine](https://bugs.ruby-lang.org/issues/10460) +* [Upstream bug github](https://github.com/ruby/ruby/pull/822) + Planning a presentation? ------------------------ @@ -210,23 +227,6 @@ assumptions based on the absence of docs, use the tool authors brain to fill the Hint, same applies to papers. -Rails -------- - -Assuming you are using rspec, you can mutation test Rails models by adding the following lines to your Gemfile: - -```ruby -group :test do - gem 'mutant-rspec' -end -``` - -Next, run bundle and comment out ```require 'rspec/autorun'``` from your spec_helper.rb file. Having done so you should be able to use commands like the following: - -```sh -RAILS_ENV=test bundle exec mutant -r ./config/environment --use rspec User -``` - Support -------