1
0
Fork 0
mirror of https://github.com/ruby/ruby.git synced 2022-11-09 12:17:21 -05:00
ruby--ruby/yjit.rb
Alan Wu f90549cd38 Rust YJIT
In December 2021, we opened an [issue] to solicit feedback regarding the
porting of the YJIT codebase from C99 to Rust. There were some
reservations, but this project was given the go ahead by Ruby core
developers and Matz. Since then, we have successfully completed the port
of YJIT to Rust.

The new Rust version of YJIT has reached parity with the C version, in
that it passes all the CRuby tests, is able to run all of the YJIT
benchmarks, and performs similarly to the C version (because it works
the same way and largely generates the same machine code). We've even
incorporated some design improvements, such as a more fine-grained
constant invalidation mechanism which we expect will make a big
difference in Ruby on Rails applications.

Because we want to be careful, YJIT is guarded behind a configure
option:

```shell
./configure --enable-yjit # Build YJIT in release mode
./configure --enable-yjit=dev # Build YJIT in dev/debug mode
```

By default, YJIT does not get compiled and cargo/rustc is not required.
If YJIT is built in dev mode, then `cargo` is used to fetch development
dependencies, but when building in release, `cargo` is not required,
only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer.

The YJIT command-line options remain mostly unchanged, and more details
about the build process are documented in `doc/yjit/yjit.md`.

The CI tests have been updated and do not take any more resources than
before.

The development history of the Rust port is available at the following
commit for interested parties:
1fd9573d8b

Our hope is that Rust YJIT will be compiled and included as a part of
system packages and compiled binaries of the Ruby 3.2 release. We do not
anticipate any major problems as Rust is well supported on every
platform which YJIT supports, but to make sure that this process works
smoothly, we would like to reach out to those who take care of building
systems packages before the 3.2 release is shipped and resolve any
issues that may come up.

[issue]: https://bugs.ruby-lang.org/issues/18481

Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Co-authored-by: Noah Gibbs <the.codefolio.guy@gmail.com>
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
2022-04-27 11:00:22 -04:00

182 lines
6.9 KiB
Ruby

# frozen_string_literal: true
# This module allows for introspection of YJIT, CRuby's experimental in-process
# just-in-time compiler. This module exists only to help develop YJIT, as such,
# everything in the module is highly implementation specific and comes with no
# API stability guarantee whatsoever.
#
# This module may not exist if YJIT does not support the particular platform
# for which CRuby is built. There is also no API stability guarantee as to in
# what situations this module is defined.
module RubyVM::YJIT
# Check if YJIT is enabled
def self.enabled?
Primitive.cexpr! 'RBOOL(rb_yjit_enabled_p())'
end
def self.stats_enabled?
Primitive.rb_yjit_stats_enabled_p
end
# Discard statistics collected for --yjit-stats.
def self.reset_stats!
Primitive.rb_yjit_reset_stats_bang
end
# Return a hash for statistics generated for the --yjit-stats command line option.
# Return nil when option is not passed or unavailable.
def self.runtime_stats
Primitive.rb_yjit_get_stats
end
# Produce disassembly for an iseq
def self.disasm(iseq)
# If a method or proc is passed in, get its iseq
iseq = RubyVM::InstructionSequence.of(iseq)
if self.enabled?
# Produce the disassembly string
# Include the YARV iseq disasm in the string for additional context
iseq.disasm + "\n" + Primitive.rb_yjit_disasm_iseq(iseq)
else
iseq.disasm
end
end
# Produce a list of instructions compiled by YJIT for an iseq
def self.insns_compiled(iseq)
# If a method or proc is passed in, get its iseq
iseq = RubyVM::InstructionSequence.of(iseq)
if self.enabled?
Primitive.rb_yjit_insns_compiled(iseq)
else
Qnil
end
end
def self.simulate_oom!
Primitive.rb_yjit_simulate_oom_bang
end
# Avoid calling a method here to not interfere with compilation tests
if Primitive.rb_yjit_stats_enabled_p
at_exit { _print_stats }
end
class << self
private
# Format and print out counters
def _print_stats
stats = runtime_stats
return unless stats
$stderr.puts("***YJIT: Printing YJIT statistics on exit***")
print_counters(stats, prefix: 'send_', prompt: 'method call exit reasons: ')
print_counters(stats, prefix: 'invokesuper_', prompt: 'invokesuper exit reasons: ')
print_counters(stats, prefix: 'leave_', prompt: 'leave exit reasons: ')
print_counters(stats, prefix: 'gbpp_', prompt: 'getblockparamproxy exit reasons: ')
print_counters(stats, prefix: 'getivar_', prompt: 'getinstancevariable exit reasons:')
print_counters(stats, prefix: 'setivar_', prompt: 'setinstancevariable exit reasons:')
print_counters(stats, prefix: 'oaref_', prompt: 'opt_aref exit reasons: ')
print_counters(stats, prefix: 'expandarray_', prompt: 'expandarray exit reasons: ')
print_counters(stats, prefix: 'opt_getinlinecache_', prompt: 'opt_getinlinecache exit reasons: ')
print_counters(stats, prefix: 'invalidate_', prompt: 'invalidation reasons: ')
side_exits = total_exit_count(stats)
total_exits = side_exits + stats[:leave_interp_return]
# Number of instructions that finish executing in YJIT.
# See :count-placement: about the subtraction.
retired_in_yjit = stats[:exec_instruction] - side_exits
# Average length of instruction sequences executed by YJIT
avg_len_in_yjit = retired_in_yjit.to_f / total_exits
# Proportion of instructions that retire in YJIT
total_insns_count = retired_in_yjit + stats[:vm_insns_count]
yjit_ratio_pct = 100.0 * retired_in_yjit.to_f / total_insns_count
# Number of failed compiler invocations
compilation_failure = stats[:compilation_failure]
$stderr.puts "bindings_allocations: " + ("%10d" % stats[:binding_allocations])
$stderr.puts "bindings_set: " + ("%10d" % stats[:binding_set])
$stderr.puts "compilation_failure: " + ("%10d" % compilation_failure) if compilation_failure != 0
$stderr.puts "compiled_iseq_count: " + ("%10d" % stats[:compiled_iseq_count])
$stderr.puts "compiled_block_count: " + ("%10d" % stats[:compiled_block_count])
$stderr.puts "invalidation_count: " + ("%10d" % stats[:invalidation_count])
$stderr.puts "constant_state_bumps: " + ("%10d" % stats[:constant_state_bumps])
$stderr.puts "inline_code_size: " + ("%10d" % stats[:inline_code_size])
$stderr.puts "outlined_code_size: " + ("%10d" % stats[:outlined_code_size])
$stderr.puts "total_exit_count: " + ("%10d" % total_exits)
$stderr.puts "total_insns_count: " + ("%10d" % total_insns_count)
$stderr.puts "vm_insns_count: " + ("%10d" % stats[:vm_insns_count])
$stderr.puts "yjit_insns_count: " + ("%10d" % stats[:exec_instruction])
$stderr.puts "ratio_in_yjit: " + ("%9.1f" % yjit_ratio_pct) + "%"
$stderr.puts "avg_len_in_yjit: " + ("%10.1f" % avg_len_in_yjit)
print_sorted_exit_counts(stats, prefix: "exit_")
end
def print_sorted_exit_counts(stats, prefix:, how_many: 20, left_pad: 4)
exits = []
stats.each do |k, v|
if k.start_with?(prefix)
exits.push [k.to_s.delete_prefix(prefix), v]
end
end
exits = exits.sort_by { |name, count| -count }[0...how_many]
total_exits = total_exit_count(stats)
top_n_total = exits.map { |name, count| count }.sum
top_n_exit_pct = 100.0 * top_n_total / total_exits
$stderr.puts "Top-#{how_many} most frequent exit ops (#{"%.1f" % top_n_exit_pct}% of exits):"
longest_insn_name_len = exits.map { |name, count| name.length }.max
exits.each do |name, count|
padding = longest_insn_name_len + left_pad
padded_name = "%#{padding}s" % name
padded_count = "%10d" % count
percent = 100.0 * count / total_exits
formatted_percent = "%.1f" % percent
$stderr.puts("#{padded_name}: #{padded_count} (#{formatted_percent}%)" )
end
end
def total_exit_count(stats, prefix: "exit_")
total = 0
stats.each do |k,v|
total += v if k.start_with?(prefix)
end
total
end
def print_counters(counters, prefix:, prompt:)
$stderr.puts(prompt)
counters = counters.filter { |key, _| key.start_with?(prefix) }
counters.filter! { |_, value| value != 0 }
counters.transform_keys! { |key| key.to_s.delete_prefix(prefix) }
if counters.empty?
$stderr.puts(" (all relevant counters are zero)")
return
end
counters = counters.to_a
counters.sort_by! { |(_, counter_value)| counter_value }
longest_name_length = counters.max_by { |(name, _)| name.length }.first.length
total = counters.sum { |(_, counter_value)| counter_value }
counters.reverse_each do |(name, value)|
percentage = value.fdiv(total) * 100
$stderr.printf(" %*s %10d (%4.1f%%)\n", longest_name_length, name, value, percentage);
end
end
end
end