JRuby

Float format string rounding tie breaker strategy does not match MRI

Details

  • Type: Bug Bug
  • Status: Open Open
  • Priority: Minor Minor
  • Resolution: Unresolved
  • Affects Version/s: JRuby 1.6, JRuby 1.7.0.pre2
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None
  • Number of attachments :
    0

Description

In MRI (several versions, including at least 1.8.7p358 and 1.9.3p194) the rounding tie breaker strategy used for floating point format strings is "round half to even", whereas in JRuby it is "round half away from zero", as can be seen in the following example code. This is different from Float#round in both implementations, which appears to adopt the "round half away from zero" strategy.

http://en.wikipedia.org/wiki/Rounding#Round_half_to_even

http://en.wikipedia.org/wiki/Rounding#Round_half_away_from_zero

MRI:

(-3...3).map

{|i|f=i+0.5; [f,'%1.0f' % f,f.round]}
=> [[-2.5, "-2", -3], [-1.5, "-2", -2], [-0.5, "-0", -1], [0.5, "0", 1], [1.5, "2", 2], [2.5, "2", 3]]

JRuby:

(-3...3).map{|i|f=i+0.5; [f,'%1.0f' % f,f.round]}

=> [[-2.5, "-3", -3], [-1.5, "-2", -2], [-0.5, "-1", -1], [0.5, "1", 1], [1.5, "2", 2], [2.5, "3", 3]]

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Charles Oliver Nutter added a comment -

As a quick sanity check, I confirmed this applies to all "half" rounding, regardless of scale or precision:

system ~/projects/jruby $ jruby -e "p '%1.1f' % -1.15"
"-1.2"

system ~/projects/jruby $ ruby-1.8.7-p358 -e "p '%1.1f' % -1.15"
"-1.1"

system ~/projects/jruby $ jruby -e "p '%1.1f' % 1.15"
"1.2"

system ~/projects/jruby $ ruby-1.8.7-p358 -e "p '%1.1f' % 1.15"
"1.1"
Show
Charles Oliver Nutter added a comment - As a quick sanity check, I confirmed this applies to all "half" rounding, regardless of scale or precision: system ~/projects/jruby $ jruby -e "p '%1.1f' % -1.15" "-1.2" system ~/projects/jruby $ ruby-1.8.7-p358 -e "p '%1.1f' % -1.15" "-1.1" system ~/projects/jruby $ jruby -e "p '%1.1f' % 1.15" "1.2" system ~/projects/jruby $ ruby-1.8.7-p358 -e "p '%1.1f' % 1.15" "1.1"
Hide
Permalink
Charles Oliver Nutter added a comment -

Ok, I'm thoroughly confused by MRI's behavior.

It seems that MRI does round half to even, but only if the next less-significant digit is nonzero.

system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % 0.05'
"0.1"

system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % 0.15'
"0.1"

system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % 0.25'
"0.2"

system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % -0.05'
"-0.1"

system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % -0.15'
"-0.1"

system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % -0.25'
"-0.2"

system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % 1.05'
"1.1"

system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % 1.15'
"1.1"

system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % 1.25'
"1.2"

I have been able to find no information about IEEE 754 rounding that indicates this is correct, but I'm still looking.

Show
Charles Oliver Nutter added a comment - Ok, I'm thoroughly confused by MRI's behavior. It seems that MRI does round half to even, but only if the next less-significant digit is nonzero. system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % 0.05' "0.1" system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % 0.15' "0.1" system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % 0.25' "0.2" system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % -0.05' "-0.1" system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % -0.15' "-0.1" system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % -0.25' "-0.2" system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % 1.05' "1.1" system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % 1.15' "1.1" system ~/projects/jruby $ ruby-1.9.3 -e 'p "%1.1f" % 1.25' "1.2" I have been able to find no information about IEEE 754 rounding that indicates this is correct, but I'm still looking.
Hide
Permalink
Charles Oliver Nutter added a comment -

Here's Java's BigDecimal doing "half even" for various values:

irb(main):029:0> java.math.BigDecimal.new('1.05').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s
=> "1.0"
irb(main):030:0> java.math.BigDecimal.new('1.15').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s
=> "1.2"
irb(main):031:0> java.math.BigDecimal.new('1.25').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s
=> "1.2"
irb(main):032:0> java.math.BigDecimal.new('1.35').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s
=> "1.4"

MRI appears to be wrong here, since it is definitely rounding to even, but not for xx05 or xx15 rounded to xxx.

irb(main):001:0> "%1.1f" % 1.05
=> "1.1"
irb(main):002:0> "%1.1f" % 1.15
=> "1.1"
irb(main):003:0> "%1.1f" % 1.25
=> "1.2"
irb(main):004:0> "%1.1f" % 1.35
=> "1.4"
irb(main):005:0> "%1.1f" % 1.45
=> "1.4"
irb(main):006:0> "%1.1f" % 1.55
=> "1.6"

JRuby is clearly wrong in all cases, if we assume half even is the proper mode, since we always round away from zero:

irb(main):034:0> "%1.1f" % 1.05
=> "1.1"
irb(main):035:0> "%1.1f" % 1.15
=> "1.2"
irb(main):036:0> "%1.1f" % 1.25
=> "1.3"
irb(main):037:0> "%1.1f" % 1.35
=> "1.4"

I'm going to file a bug with MRI to figure out what the proper handling is supposed to be.

Show
Charles Oliver Nutter added a comment - Here's Java's BigDecimal doing "half even" for various values: irb(main):029:0> java.math.BigDecimal.new('1.05').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s => "1.0" irb(main):030:0> java.math.BigDecimal.new('1.15').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s => "1.2" irb(main):031:0> java.math.BigDecimal.new('1.25').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s => "1.2" irb(main):032:0> java.math.BigDecimal.new('1.35').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s => "1.4" MRI appears to be wrong here, since it is definitely rounding to even, but not for xx05 or xx15 rounded to xxx. irb(main):001:0> "%1.1f" % 1.05 => "1.1" irb(main):002:0> "%1.1f" % 1.15 => "1.1" irb(main):003:0> "%1.1f" % 1.25 => "1.2" irb(main):004:0> "%1.1f" % 1.35 => "1.4" irb(main):005:0> "%1.1f" % 1.45 => "1.4" irb(main):006:0> "%1.1f" % 1.55 => "1.6" JRuby is clearly wrong in all cases, if we assume half even is the proper mode, since we always round away from zero: irb(main):034:0> "%1.1f" % 1.05 => "1.1" irb(main):035:0> "%1.1f" % 1.15 => "1.2" irb(main):036:0> "%1.1f" % 1.25 => "1.3" irb(main):037:0> "%1.1f" % 1.35 => "1.4" I'm going to file a bug with MRI to figure out what the proper handling is supposed to be.
Hide
Permalink
Charles Oliver Nutter added a comment -

I have filed https://bugs.ruby-lang.org/issues/7037.

Show
Charles Oliver Nutter added a comment - I have filed https://bugs.ruby-lang.org/issues/7037 .
Hide
Permalink
Charles Oliver Nutter added a comment -

The MRI folks have gotten back to me about the apparent inconsistency in the above results, pointing out (correctly) that 1.05 is not representable accurately in an IEEE 754 64-bit floating point value. 1.05 is actually represented as slightly above 1.05, so there's no tiebreaking in this case. The only cases where tiebreaking actually applies are xx25 and xx75, which round to xx2 and xx8, respectively, according to the half-even strategy. All other half representations are not accurate, but just happen to line up properly with what looks like half-even strategy. Here's the full set of halves rounded in MRI:

irb(main):002:0> "%1.1f" % 1.05
=> "1.1"
irb(main):003:0> "%1.1f" % 1.15
=> "1.1"
irb(main):004:0> "%1.1f" % 1.25
=> "1.2"
irb(main):005:0> "%1.1f" % 1.35
=> "1.4"
irb(main):006:0> "%1.1f" % 1.45
=> "1.4"
irb(main):007:0> "%1.1f" % 1.55
=> "1.6"
irb(main):008:0> "%1.1f" % 1.65
=> "1.6"
irb(main):009:0> "%1.1f" % 1.75
=> "1.8"
irb(main):010:0> "%1.1f" % 1.85
=> "1.9"
irb(main):011:0> "%1.1f" % 1.95
=> "1.9"

Note the same unexpected behavior occurs for 1.85 because it is again represented internally as slightly more than 1.85. The other inaccurate cases are either high or low depending on the results you see above... 1.15 is slightly less, 1.35 is slightly more, 1.45 is slightly less, and so on, so it looks like it's applying half-away-from-zero strategy when it's actually not tiebreaking at all.

I wrote a longer explanation in the bug, and I'm waiting for clarification from MRI folks about what specified behavior should be.

Show
Charles Oliver Nutter added a comment - The MRI folks have gotten back to me about the apparent inconsistency in the above results, pointing out (correctly) that 1.05 is not representable accurately in an IEEE 754 64-bit floating point value. 1.05 is actually represented as slightly above 1.05, so there's no tiebreaking in this case. The only cases where tiebreaking actually applies are xx25 and xx75, which round to xx2 and xx8, respectively, according to the half-even strategy. All other half representations are not accurate, but just happen to line up properly with what looks like half-even strategy. Here's the full set of halves rounded in MRI: irb(main):002:0> "%1.1f" % 1.05 => "1.1" irb(main):003:0> "%1.1f" % 1.15 => "1.1" irb(main):004:0> "%1.1f" % 1.25 => "1.2" irb(main):005:0> "%1.1f" % 1.35 => "1.4" irb(main):006:0> "%1.1f" % 1.45 => "1.4" irb(main):007:0> "%1.1f" % 1.55 => "1.6" irb(main):008:0> "%1.1f" % 1.65 => "1.6" irb(main):009:0> "%1.1f" % 1.75 => "1.8" irb(main):010:0> "%1.1f" % 1.85 => "1.9" irb(main):011:0> "%1.1f" % 1.95 => "1.9" Note the same unexpected behavior occurs for 1.85 because it is again represented internally as slightly more than 1.85. The other inaccurate cases are either high or low depending on the results you see above... 1.15 is slightly less, 1.35 is slightly more, 1.45 is slightly less, and so on, so it looks like it's applying half-away-from-zero strategy when it's actually not tiebreaking at all. I wrote a longer explanation in the bug, and I'm waiting for clarification from MRI folks about what specified behavior should be.

People

  • Assignee:
    Thomas E Enebo
    Reporter:
    Ben Armstrong
Vote (0)
Watch (3)

Dates

  • Created:
    Updated: