mirror of
https://github.com/ruby/ruby.git
synced 2022-11-09 12:17:21 -05:00
9d24ddbb53
`make benchmark ARGS=../benchmark/erb_render.yml` does not work.
```
% make benchmark ARGS=../benchmark/erb_render.yml
/Users/kamipo/.rbenv/shims/ruby --disable=gems -rrubygems -I./benchmark/lib ./benchmark/benchmark-driver/exe/benchmark-driver \
--executables="compare-ruby::/Users/kamipo/.rbenv/shims/ruby --disable=gems -I.ext/common --disable-gem" \
--executables="built-ruby::./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems --disable-gem" \
../benchmark/erb_render.yml
Traceback (most recent call last):
6: from ./benchmark/benchmark-driver/exe/benchmark-driver:112:in `<main>'
5: from ./benchmark/benchmark-driver/exe/benchmark-driver:112:in `flat_map'
4: from ./benchmark/benchmark-driver/exe/benchmark-driver:112:in `each'
3: from ./benchmark/benchmark-driver/exe/benchmark-driver:122:in `block in <main>'
2: from /Users/kamipo/.rbenv/versions/2.6.6/lib/ruby/2.6.0/psych.rb:577:in `load_file'
1: from /Users/kamipo/.rbenv/versions/2.6.6/lib/ruby/2.6.0/psych.rb:577:in `open'
/Users/kamipo/.rbenv/versions/2.6.6/lib/ruby/2.6.0/psych.rb:577:in `initialize': No such file or directory @ rb_sysopen - ../benchmark/erb_render.yml (Errno::ENOENT)
make: *** [benchmark] Error 1
% make benchmark ARGS=benchmark/erb_render.yml
/Users/kamipo/.rbenv/shims/ruby --disable=gems -rrubygems -I./benchmark/lib ./benchmark/benchmark-driver/exe/benchmark-driver \
--executables="compare-ruby::/Users/kamipo/.rbenv/shims/ruby --disable=gems -I.ext/common --disable-gem" \
--executables="built-ruby::./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems --disable-gem" \
benchmark/erb_render.yml
Calculating -------------------------------------
compare-ruby built-ruby
erb_render 825.454k 783.664k i/s - 1.500M times in 1.817181s 1.914086s
Comparison:
erb_render
compare-ruby: 825454.4 i/s
built-ruby: 783663.8 i/s - 1.05x slower
```
72 lines
2.8 KiB
Markdown
72 lines
2.8 KiB
Markdown
# ruby/benchmark
|
|
|
|
This directory has benchmark definitions to be run with
|
|
[benchmark\_driver.gem](https://github.com/benchmark-driver/benchmark-driver).
|
|
|
|
## Normal usage
|
|
|
|
Execute `gem install benchmark_driver` and run a command like:
|
|
|
|
```bash
|
|
# Run a benchmark script with the ruby in the $PATH
|
|
benchmark-driver benchmark/app_fib.rb
|
|
|
|
# Run benchmark scripts with multiple Ruby executables or options
|
|
benchmark-driver benchmark/*.rb -e /path/to/ruby -e '/path/to/ruby --jit'
|
|
|
|
# Or compare Ruby versions managed by rbenv
|
|
benchmark-driver benchmark/*.rb --rbenv '2.5.1;2.6.0-preview2 --jit'
|
|
|
|
# You can collect many metrics in many ways
|
|
benchmark-driver benchmark/*.rb --runner memory --output markdown
|
|
|
|
# Some are defined with YAML for complex setup or accurate measurement
|
|
benchmark-driver benchmark/*.yml
|
|
```
|
|
|
|
See also:
|
|
|
|
```console
|
|
Usage: benchmark-driver [options] RUBY|YAML...
|
|
-r, --runner TYPE Specify runner type: ips, time, memory, once (default: ips)
|
|
-o, --output TYPE Specify output type: compare, simple, markdown, record (default: compare)
|
|
-e, --executables EXECS Ruby executables (e1::path1 arg1; e2::path2 arg2;...)
|
|
--rbenv VERSIONS Ruby executables in rbenv (x.x.x arg1;y.y.y arg2;...)
|
|
--repeat-count NUM Try benchmark NUM times and use the fastest result or the worst memory usage
|
|
--repeat-result TYPE Yield "best", "average" or "worst" result with --repeat-count (default: best)
|
|
--bundler Install and use gems specified in Gemfile
|
|
--filter REGEXP Filter out benchmarks with given regexp
|
|
--run-duration SECONDS Warmup estimates loop_count to run for this duration (default: 3)
|
|
-v, --verbose Verbose mode. Multiple -v options increase visilibity (max: 2)
|
|
```
|
|
|
|
## make benchmark
|
|
|
|
Using `make benchmark`, `make update-benchmark-driver` automatically downloads
|
|
the supported version of benchmark\_driver, and it runs benchmarks with the downloaded
|
|
benchmark\_driver.
|
|
|
|
```bash
|
|
# Run all benchmarks with the ruby in the $PATH and the built ruby
|
|
make benchmark
|
|
|
|
# Or compare with specific ruby binary
|
|
make benchmark COMPARE_RUBY="/path/to/ruby --jit"
|
|
|
|
# Run vm benchmarks
|
|
make benchmark ITEM=vm
|
|
|
|
# Run some limited benchmarks in ITEM-matched files
|
|
make benchmark ITEM=vm OPTS=--filter=block
|
|
|
|
# You can specify the benchmark by an exact filename instead of using the default argument:
|
|
# ARGS = $$(find $(srcdir)/benchmark -maxdepth 1 -name '*$(ITEM)*.yml' -o -name '*$(ITEM)*.rb')
|
|
make benchmark ARGS=benchmark/erb_render.yml
|
|
|
|
# You can specify any option via $OPTS
|
|
make benchmark OPTS="--help"
|
|
|
|
# With `make benchmark`, some special runner plugins are available:
|
|
# -r peak, -r size, -r total, -r utime, -r stime, -r cutime, -r cstime
|
|
make benchmark ITEM=vm_bigarray OPTS="-r peak"
|
|
```
|