For garbage-concious users who use the `outbuf' argument of
`readpartial' to supply a destination buffer, this provides
a drastic reduction in garbage when inflating large inputs
in a streaming fashion.
This results in a anonymous RSS reduction in the reader
similar to the reduction in the writer from r61631.
Results using the test script from r61631
<https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=61631>
Before:
writer 7.359999 0.000000 7.359999 ( 7.360639)
writer RssAnon: 4040 kB
reader 6.346667 0.070000 6.416667 ( 7.387654)
reader RssAnon: 98272 kB
After:
writer 7.309999 0.000000 7.309999 ( 7.310651)
writer RssAnon: 4048 kB
reader 6.146666 0.003333 6.149999 ( 7.334868)
reader RssAnon: 4300 kB
* ext/zlib/zlib.c (struct read_raw_arg): new struct
(gzfile_read_raw_partial): use read_raw_arg
(gzfile_read_raw_rescue): ditto
(gzfile_read_raw): accept outbuf, use read_raw_arg
(gzfile_read_raw_ensure): accept outbuf
(gzfile_read_header): ditto
(gzfile_check_footer): ditto
(gzfile_read_more): ditto
(gzfile_read_raw_until_zero): adjust for changes
(gzfile_fill): ditto
(gzfile_readpartial): ditto
(gzfile_read_all): ditto
(gzfile_getc): ditto
(gzfile_reader_end_run): ditto
(gzfile_reader_get_unused): ditto
(rb_gzreader_initialize): ditto
(gzreader_skip_linebreaks): ditto
(gzreader_gets): ditto
(zlib_gunzip_run): ditto
[ruby-core:84660] [Feature #14319]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61665 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Zlib::GzipWriter generated large amounts of garbage from
(struct zstream).input. Reuse the .input field when it is
hidden, and recycle it when its lifetime is over. This change
alone reduced memory usage of the writer from 90MB to 4.5MB.
For the detached buffer of compressed data used by
gzfile_write_raw, we can only clear the string (not recycle it)
since user code may hold references to it (but the data would be
clobbered, anyways). This reduced memory usage slightly by
around 0.5MB (because it's smaller compressed data).
Combined, these changes reduce the anonymous RSS memory of a
dedicated writer process from over 90MB to under 4MB.
before:
# user system total real
writer 7.823332 0.053333 7.876665 ( 7.881464)
writer RssAnon: 92944 kB
reader 6.969999 0.076666 7.046665 ( 7.906377)
reader RssAnon: 109820 kB
after:
writer 7.359999 0.000000 7.359999 ( 7.360639)
writer RssAnon: 4040 kB
reader 6.346667 0.070000 6.416667 ( 7.387654)
reader RssAnon: 98272 kB
Script used:
-------
require 'zlib'
require 'benchmark'
nr = 16384 * 2
def stats(pfx, bm)
str = "#{bm}#{File.readlines("/proc/#$$/status").grep(/^RssAnon:/)[0]}"
puts str.gsub!(/^/m, pfx)
end
rd, wr = IO.pipe
pid = fork do
buf = ((0..255).map(&:chr).join * 128).freeze
rd.close
gzip = Zlib::GzipWriter.new(wr)
bm = Benchmark.measure do
nr.times { gzip.write(buf) }
gzip.close
wr.close
end
stats('writer ', bm)
end
wr.close
buf = ''
gunzip = Zlib::GzipReader.new(rd)
n = 0
bm = Benchmark.measure do
begin
gunzip.readpartial(16384, buf)
n += buf.size
rescue EOFError
break
end while true
end
stats('reader ', bm)
Process.waitall
-------
* ext/zlib/zlib.c (zstream_discard_input): reuse or recycle hidden input
(zstream_reset_input): clear hidden input
(zstream_run): detach input and recycle after use
(gzfile_write_raw): clear buffer after write
[ruby-core:84638] [Feature #14315]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61631 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
No need to reveal strings freshly created with rb_str_new.
* ext/zlib/zlib.c (zstream_detach_input): remove redundant call
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61612 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
NODE_PRELUDE contains a `BEGIN` node, a main node, and compile_option.
This node is assumed that it must be located immediately under the root
NODE_SCOPE, but this strange assumption is not so good, IMO.
This change removes the assumtion; it integrates the former two nodes by
block_append, and moves compile_option into rb_ast_body_t.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61610 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
New IMEMO types were introduced, this just fixes the function that
converts the type to support the new types.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61377 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It version changed fallback option to keywoad argument
on `Yaml.load` method. It break backword compatiblity.
see detailed discuttion: https://github.com/ruby/psych/issues/340
From: SHIBATA Hiroshi <hsbt@ruby-lang.org>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61336 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
But we couldn't ship strscan (and stringio) gem. Because rubygems.org
still masked them.
https://github.com/rubygems/rubygems.org/pull/1696
I'm going to ship them after it deployed.
From: SHIBATA Hiroshi <hsbt@ruby-lang.org>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61333 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
It needs to support header file installation and uninstallation.
But current RubyGems didn't support it feature yet.
From: SHIBATA Hiroshi <hsbt@ruby-lang.org>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61332 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* tool/gperf.sed: comment out arguments part only, to keep the
following declarations static. [Feature #13883]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61282 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/ripper/lib/ripper/lexer.rb (Ripper::Lexer): added allbits?,
anybits? and nobits? methods, as well as Integer. a patch by
aycabta. [Feature #14170]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61205 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* tool/gperf.sed: extracted sed commands to a script. ANSI-C code
produced by gperf 3.1 declares length arguments as `size_t`. it
causes conflict with existing declarations, and needs casts for
a local variable and return statements.
[Feature #13883]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61076 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This change makes method coverage result have not only first lineno of
method defintion, but also code range (i.e. first lineno, first column,
last lineno, and last column).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61026 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This change makes each ISeq keep NODE's code range. This information is
needed for method coverage.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61025 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Traditionally, method coverage measurement was implemented by inserting
`trace2` instruction to the head of method iseq. So, it just measured
methods defined by `def` keyword.
This commit drastically changes the measuring mechanism of method
coverage; at `RUBY_EVENT_CALL`, it keeps a hash from rb_method_entry_t*
to runs (i.e., it counts the runs per method entry), and at
`Coverage.result`, it creates the result hash by enumerating all
`rb_method_entry_t*` objects (by `ObjectSpace.each_object`).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61023 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/socket/ifaddr.c (ifaddr_memsize): do not count the whole
rb_ifaddr_t array for each elements. the header size is
included in the first element for the time being.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61006 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/socket/ifaddr.c (struct rb_ifaddr_tag): removed set but
unused member root.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61001 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/strscan/strscan.c: added `size`, `captures` and `values_at`
to StringScanner, shorthands of accessing the matched data.
based on the patch by apeiros (Stefan Rusterholz) at
[ruby-core:20412]. [Feature #836]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60929 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* ext/psych/psych_parser.c (parse): fix declarations after
statement, which cause compile error on mswin.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60919 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Import Ruby/OpenSSL 2.1.0.beta2. The full commit log since commit
e72d960db262 which was imported by r60013 can be found at:
https://github.com/ruby/openssl/compare/e72d960db262...v2.1.0.beta2
----------------------------------------------------------------
Kazuki Yamaguchi (26):
bn: use ALLOCV() macro instead of xmalloc()
appveyor.yml: remove 'openssl version' line
test/test_ssl_session: skip tests for session_remove_cb
x509ext: implement X509::Extension#==
x509attr: implement X509::Attribute#==
x509cert: implement X509::Certificate#==
x509revoked: add missing X509::Revoked#to_der
x509crl, x509revoked: implement X509::{CRL,Revoked}#==
x509req: implement X509::Request#==
ssl: extract rb_intern("call")
cipher: disallow setting AAD for non-AEAD ciphers
test/test_cipher: fix test_non_aead_cipher_set_auth_data failure
ssl: fix conflict of options in SSLContext#set_params
buffering: let #write accept multiple arguments
pkey: make pkey_check_public_key() non-static
x509cert, x509crl, x509req, ns_spki: check sanity of public key
test/envutil: port assert_warning from Ruby trunk
test/utils: remove a pointless .public_key call in issue_cert
ssl: add SSLContext#add_certificate
test/test_ssl: fix test_security_level
Drop support for LibreSSL 2.4
kdf: add HKDF support
test/test_x509cert: fix flaky test
test/test_x509crl: fix random failure
History.md: fix a typo
Ruby/OpenSSL 2.1.0.beta2
Mark Wright (1):
Fix build failure against OpenSSL 1.1 built with no-deprecated Thanks rhenium for the code review and fixes.
Peter Karman (1):
Add RSA sign_pss() and verify_pss() methods
aeris (1):
TLS Fallback Signaling Cipher Suite Value
kazu (1):
Use caller with length to reduce unused strings
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60907 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
And adapt a net/http test to their old behavior. [ruby-core:83491]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60906 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
