1
0
Fork 0
mirror of https://github.com/ruby/ruby.git synced 2022-11-09 12:17:21 -05:00
ruby--ruby/benchmark
Alan Wu 68ffc8db08 Set allocator on class creation
Allocating an instance of a class uses the allocator for the class. When
the class has no allocator set, Ruby looks for it in the super class
(see rb_get_alloc_func()).

It's uncommon for classes created from Ruby code to ever have an
allocator set, so it's common during the allocation process to search
all the way to BasicObject from the class with which the allocation is
being performed. This makes creating instances of classes that have
long ancestry chains more expensive than creating instances of classes
have that shorter ancestry chains.

Setting the allocator at class creation time removes the need to perform
a search for the alloctor during allocation.

This is a breaking change for C-extensions that assume that classes
created from Ruby code have no allocator set. Libraries that setup a
class hierarchy in Ruby code and then set the allocator on some parent
class, for example, can experience breakage. This seems like an unusual
use case and hopefully it is rare or non-existent in practice.

Rails has many classes that have upwards of 60 elements in the ancestry
chain and benchmark shows a significant improvement for allocating with
a class that includes 64 modules.

```
pre: ruby 3.0.0dev (2020-11-12T14:39:27Z master 6325866421)
post: ruby 3.0.0dev (2020-11-12T20:15:30Z cut-allocator-lookup)

Comparison:
                  allocate_8_deep
                post:  10336985.6 i/s
                 pre:   8691873.1 i/s - 1.19x  slower

                 allocate_32_deep
                post:  10423181.2 i/s
                 pre:   6264879.1 i/s - 1.66x  slower

                 allocate_64_deep
                post:  10541851.2 i/s
                 pre:   4936321.5 i/s - 2.14x  slower

                allocate_128_deep
                post:  10451505.0 i/s
                 pre:   3031313.5 i/s - 3.45x  slower
```
2020-11-16 17:41:40 -05:00
..
gc
lib Share warmup logic across MJIT benchmarks 2020-06-22 00:54:27 -07:00
other-lang
app_answer.rb
app_aobench.rb
app_erb.yml
app_factorial.rb
app_fib.rb
app_lc_fizzbuzz.rb
app_mandelbrot.rb
app_pentomino.rb
app_raise.rb
app_strconcat.rb
app_tak.rb
app_tarai.rb
app_uri.rb
array_flatten.yml
array_intersection.yml
array_max_float.yml Optimize Array#max (#3325) 2020-07-18 23:45:00 +09:00
array_max_int.yml Optimize Array#max (#3325) 2020-07-18 23:45:00 +09:00
array_max_str.yml Optimize Array#max (#3325) 2020-07-18 23:45:00 +09:00
array_min.yml Optimize Array#min (#3324) 2020-07-18 23:45:25 +09:00
array_sample_100k_10.rb
array_sample_100k_11.rb
array_sample_100k__1k.rb
array_sample_100k__6k.rb
array_sample_100k__100.rb
array_sample_100k___10k.rb
array_sample_100k___50k.rb
array_shift.rb
array_small_and.rb
array_small_diff.rb
array_small_or.rb
array_sort_block.rb
array_sort_float.rb
array_values_at_int.rb
array_values_at_range.rb
bighash.rb
cgi_escape_html.yml
complex_float_add.yml
complex_float_div.yml
complex_float_mul.yml
complex_float_new.yml
complex_float_power.yml
complex_float_sub.yml
dir_empty_p.rb
enum_lazy_flat_map.yml Improved Enumerable::Lazy#flat_map 2020-07-23 16:57:26 +09:00
enum_lazy_grep_v_20.rb
enum_lazy_grep_v_50.rb
enum_lazy_grep_v_100.rb
enum_lazy_uniq_20.rb
enum_lazy_uniq_50.rb
enum_lazy_uniq_100.rb
enum_lazy_zip.yml Improved Enumerable::Lazy#zip 2020-07-23 16:57:26 +09:00
erb_render.yml
fiber_chain.yml
fiber_locals.yml
file_chmod.rb
file_rename.rb
hash_aref_dsym.rb
hash_aref_dsym_long.rb
hash_aref_fix.rb
hash_aref_flo.rb
hash_aref_miss.rb
hash_aref_str.rb
hash_aref_sym.rb
hash_aref_sym_long.rb
hash_defaults.yml
hash_dup.yml
hash_flatten.rb
hash_ident_flo.rb
hash_ident_num.rb
hash_ident_obj.rb
hash_ident_str.rb
hash_ident_sym.rb
hash_keys.rb
hash_literal_small2.rb
hash_literal_small4.rb
hash_literal_small8.rb
hash_long.rb
hash_shift.rb
hash_shift_u16.rb
hash_shift_u24.rb
hash_shift_u32.rb
hash_small2.rb
hash_small4.rb
hash_small8.rb
hash_to_proc.rb
hash_values.rb
int_quo.rb
io_copy_stream_write.rb
io_copy_stream_write_socket.rb
io_file_create.rb
io_file_read.rb
io_file_write.rb
io_nonblock_noex.rb
io_nonblock_noex2.rb
io_pipe_rw.rb
io_select.rb
io_select2.rb
io_select3.rb
irb_color.yml
irb_exec.yml
kernel_clone.yml
kernel_float.yml support builtin for Kernel#Float 2020-04-22 09:49:13 +09:00
kernel_tap.yml Rewrite Kernel#tap with Ruby (#3281) 2020-07-03 09:52:35 -07:00
kernel_then.yml Make Kernel#then, #yield_self, #frozen? builtin (#3283) 2020-07-03 18:02:43 -07:00
keyword_arguments.yml
loop_for.rb
loop_generator.rb
loop_times.rb
loop_whileloop.rb
loop_whileloop2.rb
marshal_dump_flo.rb
marshal_dump_load_geniv.rb
marshal_dump_load_time.rb
match_gt4.rb
match_small.rb
mjit_exec_jt2jt.yml The RUBYOPT= comment is no longer needed 2020-06-22 00:20:30 -07:00
mjit_exec_vm2jt.yml The RUBYOPT= comment is no longer needed 2020-06-22 00:20:30 -07:00
mjit_exec_vm2vm.yml The RUBYOPT= comment is no longer needed 2020-06-22 00:20:30 -07:00
mjit_exivar.yml Share warmup logic across MJIT benchmarks 2020-06-22 00:54:27 -07:00
mjit_integer.yml Mark some Integer methods as inline (#3264) 2020-06-27 10:07:47 -07:00
mjit_kernel.yml Make Kernel#then, #yield_self, #frozen? builtin (#3283) 2020-07-03 18:02:43 -07:00
mjit_leave.yml Share warmup logic across MJIT benchmarks 2020-06-22 00:54:27 -07:00
mjit_opt_cc_insns.yml Decide JIT-ed insn based on cached cfunc 2020-06-25 23:33:08 -07:00
mjit_struct_aref.yml Inline builtin struct aref 2020-07-06 00:14:00 -07:00
nil_p.yml
num_zero_p.yml Make Integer#zero? a separated method and builtin (#3226) 2020-06-20 14:55:09 -07:00
object_allocate.yml Set allocator on class creation 2020-11-16 17:41:40 -05:00
objspace_dump_all.yml Optimize ObjectSpace.dump_all 2020-09-09 11:11:36 -07:00
pm_array.yml Add pattern matching with arrays benchmark 2020-06-27 13:51:03 +09:00
range_last.yml
README.md Fix make benchmark example 2020-06-07 10:33:14 +09:00
realpath.yml
require.yml
require_thread.yml
securerandom.rb
so_ackermann.rb
so_array.rb
so_binary_trees.rb
so_concatenate.rb
so_count_words.yml
so_exception.rb
so_fannkuch.rb
so_fasta.rb
so_k_nucleotide.yml
so_lists.rb
so_mandelbrot.rb
so_matrix.rb
so_meteor_contest.rb
so_nbody.rb
so_nested_loop.rb
so_nsieve.rb
so_nsieve_bits.rb
so_object.rb
so_partial_sums.rb
so_pidigits.rb
so_random.rb
so_reverse_complement.yml
so_sieve.rb
so_spectralnorm.rb
string_capitalize.yml
string_casecmp.yml
string_casecmp_p.yml
string_downcase.yml
string_index.rb
string_scan_re.rb
string_scan_str.rb
string_slice.yml
string_split.yml Added String#split benchmark for empty regexp 2020-05-12 22:59:58 +09:00
string_swapcase.yml
string_upcase.yml
time_strptime.yml
time_subsec.rb
vm_array.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_attr_ivar.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_attr_ivar_set.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_backtrace.rb Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_bigarray.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_bighash.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_block.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_block_handler.yml add benchmark for different block handlers 2020-06-03 16:13:47 +09:00
vm_blockparam.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_blockparam_call.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_blockparam_pass.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_blockparam_yield.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_case.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_case_lit.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_clearmethodcache.rb Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_const.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_defined_method.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_dstr.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_ensure.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_eval.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_fiber_allocate.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_fiber_count.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_fiber_reuse.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_fiber_reuse_gc.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_fiber_switch.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_float_simple.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_freezestring.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_gc.rb Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_gc_old_full.rb Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_gc_old_immediate.rb Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_gc_old_lazy.rb Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_gc_short_lived.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_gc_short_with_complex_long.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_gc_short_with_long.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_gc_short_with_symbol.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_gc_wb_ary.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_gc_wb_ary_promoted.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_gc_wb_obj.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_gc_wb_obj_promoted.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_iclass_super.yml Improve the performance of super 2020-09-23 11:52:36 -07:00
vm_ivar.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_ivar_init.yml eagerly initialize ivar table when index is small enough 2020-11-09 09:44:16 -08:00
vm_ivar_set.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_ivar_set_subclass.yml Add a benchmark for polymorphic ivar setting 2020-11-09 14:05:41 -08:00
vm_length.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_lvar_init.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_lvar_set.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_method.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_method_missing.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_method_with_block.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_module_ann_const_set.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_module_const_set.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_mutex.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_neq.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_newlambda.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_not.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_poly_method.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_poly_method_ov.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_poly_same_method.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_poly_singleton.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_proc.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_raise1.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_raise2.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_regexp.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_rescue.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_send.yml Added benchmark of vm_send by variable [ci skip] 2020-10-28 09:47:46 +09:00
vm_send_cfunc.yml
vm_simplereturn.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_string_literal.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_struct_big_aref_hi.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_struct_big_aref_lo.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_struct_big_aset.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_struct_big_href_hi.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_struct_big_href_lo.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_struct_big_hset.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_struct_small_aref.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_struct_small_aset.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_struct_small_href.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_struct_small_hset.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_super.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_swap.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_symbol_block_pass.rb
vm_thread_alive_check.yml
vm_thread_close.rb
vm_thread_condvar1.rb
vm_thread_condvar2.rb
vm_thread_create_join.rb
vm_thread_mutex1.rb
vm_thread_mutex2.rb
vm_thread_mutex3.rb
vm_thread_pass.rb
vm_thread_pass_flood.rb
vm_thread_pipe.rb
vm_thread_queue.rb
vm_thread_sized_queue.rb
vm_thread_sized_queue2.rb
vm_thread_sized_queue3.rb
vm_thread_sized_queue4.rb
vm_thread_sleep.yml
vm_unif1.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_yield.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00
vm_zsuper.yml Unify vm benchmark prefixes to vm_ (#3028) 2020-04-13 21:37:42 -07:00

ruby/benchmark

This directory has benchmark definitions to be run with benchmark_driver.gem.

Normal usage

Execute gem install benchmark_driver and run a command like:

# Run a benchmark script with the ruby in the $PATH
benchmark-driver benchmark/app_fib.rb

# Run benchmark scripts with multiple Ruby executables or options
benchmark-driver benchmark/*.rb -e /path/to/ruby -e '/path/to/ruby --jit'

# Or compare Ruby versions managed by rbenv
benchmark-driver benchmark/*.rb --rbenv '2.5.1;2.6.0-preview2 --jit'

# You can collect many metrics in many ways
benchmark-driver benchmark/*.rb --runner memory --output markdown

# Some are defined with YAML for complex setup or accurate measurement
benchmark-driver benchmark/*.yml

See also:

Usage: benchmark-driver [options] RUBY|YAML...
    -r, --runner TYPE                Specify runner type: ips, time, memory, once (default: ips)
    -o, --output TYPE                Specify output type: compare, simple, markdown, record (default: compare)
    -e, --executables EXECS          Ruby executables (e1::path1 arg1; e2::path2 arg2;...)
        --rbenv VERSIONS             Ruby executables in rbenv (x.x.x arg1;y.y.y arg2;...)
        --repeat-count NUM           Try benchmark NUM times and use the fastest result or the worst memory usage
        --repeat-result TYPE         Yield "best", "average" or "worst" result with --repeat-count (default: best)
        --bundler                    Install and use gems specified in Gemfile
        --filter REGEXP              Filter out benchmarks with given regexp
        --run-duration SECONDS       Warmup estimates loop_count to run for this duration (default: 3)
    -v, --verbose                    Verbose mode. Multiple -v options increase visilibity (max: 2)

make benchmark

Using make benchmark, make update-benchmark-driver automatically downloads the supported version of benchmark_driver, and it runs benchmarks with the downloaded benchmark_driver.

# Run all benchmarks with the ruby in the $PATH and the built ruby
make benchmark

# Or compare with specific ruby binary
make benchmark COMPARE_RUBY="/path/to/ruby --jit"

# Run vm benchmarks
make benchmark ITEM=vm

# Run some limited benchmarks in ITEM-matched files
make benchmark ITEM=vm OPTS=--filter=block

# You can specify the benchmark by an exact filename instead of using the default argument:
# ARGS = $$(find $(srcdir)/benchmark -maxdepth 1 -name '*$(ITEM)*.yml' -o -name '*$(ITEM)*.rb')
make benchmark ARGS=benchmark/erb_render.yml

# You can specify any option via $OPTS
make benchmark OPTS="--help"

# With `make benchmark`, some special runner plugins are available:
#   -r peak, -r size, -r total, -r utime, -r stime, -r cutime, -r cstime
make benchmark ITEM=vm_bigarray OPTS="-r peak"