1
0
Fork 0
mirror of https://github.com/ruby/ruby.git synced 2022-11-09 12:17:21 -05:00

st.c: fix crashes on huge hash tables

From: Vladimir Makarov <vmakarov@redhat.com>

By Vladimir's estimation, this requires at least 64 GB of memory
to reproduce this bug due to the hash sizes required.  So there
is no new test case (and I am unable to test it, myself).

* st.c (get_bins_num): avoid out-of-bounds on shift by using correct type
  [ruby-core:78139] [Bug #12939]
* st.c (get_allocated_entries): ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56793 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This commit is contained in:
normal 2016-11-15 00:27:20 +00:00
parent 30dcac0494
commit f46a43fc58

4
st.c
View file

@ -417,7 +417,7 @@ get_size_ind(const st_table *tab) {
/* Return the number of allocated bins of table TAB. */
static inline st_index_t
get_bins_num(const st_table *tab) {
return 1<<tab->bin_power;
return ((st_index_t) 1)<<tab->bin_power;
}
/* Return mask for a bin index in table TAB. */
@ -436,7 +436,7 @@ hash_bin(st_hash_t hash_value, st_table *tab) {
/* Return the number of allocated entries of table TAB. */
static inline st_index_t
get_allocated_entries(const st_table *tab) {
return 1<<tab->entry_power;
return ((st_index_t) 1)<<tab->entry_power;
}
/* Return size of the allocated bins of table TAB. */