kotovalexarian-likes-github/ruby--ruby
1
0
Fork 0
mirror of https://github.com/ruby/ruby.git synced 2022-11-09 12:17:21 -05:00
Code Releases Activity

~(unsigned char) is not unsigned char

Browse source 
The unary ~ operator excercises integer promotion of the operand
_before_ actually applying bitwise complement (cf: ISO/IEC 9899:1990
section 6.3.3.3).  Which means `~buf[i]` is in fact
`(int)~(int)buf[i]`.

The problem is, when buf[i] is 0xFF:

      buf[i]        0xFF
 (int)buf[i] 0x0000_00FF
~(int)buf[i] 0xFFFF_FF00 This is -256, out of unsigned char range.

The proposed fix is to change the char signed.  By doing so,

                   buf[i]        0xFF
      (signed char)buf[i]        0xFF
 (int)(signed char)buf[i] 0xFFFF_FFFF
~(int)(signed char)buf[i] 0x0000_0000 This is 0, does not overflow.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65675 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This commit is contained in:
shyouhei 2018-11-12 01:08:35 +00:00
parent 623ecdace6
commit 199c5cc185
1 changed files with 6 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
bignum.c
View file

@ -616,8 +616,12 @@ static int
bytes_2comp(unsigned char *buf, size_t len)
{
size_t i;
for (i = 0; i < len; i++)
buf[i] = ~buf[i];
for (i = 0; i < len; i++) {
signed char c = buf[i];
signed int d = ~c;
unsigned int e = d & 0xFF;
buf[i] = e;
}
for (i = 0; i < len; i++) {
buf[i]++;
if (buf[i] != 0)