kotovalexarian/free_mutant
1
0
Fork 0
Code Releases Activity

Change location of known problems docs

Browse source
This commit is contained in:
Markus Schirp 2018-11-27 13:01:32 +00:00
parent 90c1c64974
commit 71652cc677
2 changed files with 45 additions and 46 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

47
README.md
View file

@ -22,6 +22,7 @@ Topics
* [Nomenclature](/docs/nomenclature.md)
* [Reading Reports](/docs/reading-reports.md)
* [Known Problems](/docs/known-problems.md)
Sponsoring
----------
@ -194,51 +195,6 @@ Running mutant for the first time on an existing codebase can be a rather dishea
bundle exec mutant --include lib --require virtus --since master --use rspec Virtus::Attribute#type
```
Known Problems
==============
Mutations with Infinite Runtimes
---------------------------------
Occasionally mutant will produce a mutation with an infinite runtime. When this happens
mutant will look like it is running indefinitely without killing a remaining mutation. To
avoid mutations like this, consider adding a timeout around your tests. For example, in
RSpec you can add the following to your `spec_helper`:
```ruby
config.around(:each) do |example|
Timeout.timeout(5, &example)
end
```
which will fail specs which run for longer than 5 seconds.
The Crash / Stuck Problem (MRI)
-------------------------------
Mutations generated by mutant can cause MRI to enter VM states its not prepared for.
All MRI versions > 1.9 and < 2.2.1 are affected by this depending on your compiler flags,
compiler version, and OS scheduling behavior.
This can have the following unintended effects:
* MRI crashes with a segfault. Mutant kills each mutation in a dedicated fork to isolate
the mutations side effects when this fork terminates abnormally (segfault) mutant
counts the mutation as killed.
* MRI crashes with a segfault and gets stuck when handling the segfault.
Depending on the number of active kill jobs mutant might appear to continue normally until
all workers are stuck into this state when it begins to hang.
Currently mutant must assume that your test suite simply not terminated yet as from the outside
(parent process) the difference between a long running test and a stuck MRI is not observable.
Its planned to implement a timeout enforced from the parent process, but ideally MRI simply gets fixed.
References:
* [MRI fix](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636)
* [MRI backport to 2.2.1](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636)
* [Mutant issue](https://github.com/mbj/mutant/issues/265)
* [Upstream bug redmine](https://bugs.ruby-lang.org/issues/10460)
* [Upstream bug github](https://github.com/ruby/ruby/pull/822)
Presentations
-------------
@ -249,7 +205,6 @@ There are some presentations about mutant in the wild:
* [eurucamp 2013](http://2013.eurucamp.org/) / FrOSCon-2013 http://slid.es/markusschirp/mutation-testing
* [Cologne.rb](http://www.colognerb.de/topics/mutation-testing-mit-mutant) / https://github.com/DonSchado/colognerb-on-mutant/blob/master/mutation_testing_slides.pdf
Planning a presentation?
------------------------

44
docs/known-problems.md Normal file
View file

@ -0,0 +1,44 @@
Known Problems
==============
Mutations with Infinite Runtimes
---------------------------------
Occasionally mutant will produce a mutation with an infinite runtime. When this happens
mutant will look like it is running indefinitely without killing a remaining mutation. To
avoid mutations like this, consider adding a timeout around your tests. For example, in
RSpec you can add the following to your `spec_helper`:
```ruby
config.around(:each) do |example|
Timeout.timeout(5, &example)
end
```
which will fail specs which run for longer than 5 seconds.
The Crash / Stuck Problem (MRI)
-------------------------------
Mutations generated by mutant can cause MRI to enter VM states its not prepared for.
All MRI versions > 1.9 and < 2.2.1 are affected by this depending on your compiler flags,
compiler version, and OS scheduling behavior.
This can have the following unintended effects:
* MRI crashes with a segfault. Mutant kills each mutation in a dedicated fork to isolate
the mutations side effects when this fork terminates abnormally (segfault) mutant
counts the mutation as killed.
* MRI crashes with a segfault and gets stuck when handling the segfault.
Depending on the number of active kill jobs mutant might appear to continue normally until
all workers are stuck into this state when it begins to hang.
Currently mutant must assume that your test suite simply not terminated yet as from the outside
(parent process) the difference between a long running test and a stuck MRI is not observable.
Its planned to implement a timeout enforced from the parent process, but ideally MRI simply gets fixed.
References:
* [MRI fix](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636)
* [MRI backport to 2.2.1](https://github.com/ruby/ruby/commit/8fe95fea9d238a6deb70c8953ceb3a28a67f4636)
* [Mutant issue](https://github.com/mbj/mutant/issues/265)
* [Upstream bug redmine](https://bugs.ruby-lang.org/issues/10460)
* [Upstream bug github](https://github.com/ruby/ruby/pull/822)