Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Bug Rounding Floats? (9.245 * 100).round => 924? S/B 925!

Reply
Thread Tools

Bug Rounding Floats? (9.245 * 100).round => 924? S/B 925!

 
 
ddoherty03
Guest
Posts: n/a
 
      07-22-2009
All,

Would someone try the below unit test and explain why ruby does not
pass. It came up while I was trying to write a generalized rounding
method for floats that takes the number of place to round to as a
parameter. It gave wrong answers, but only on rare inputs.

For example, (9.245 * 100).round should be 925, but my ruby gives 924.

=================================
#! /usr/bin/env ruby
#
require 'test/unit'

class TestFloatRounding < Test::Unit::TestCase
def test_round
assert_equal(925, 924.5.round, "Rounding Error: 924.5.round")
assert_equal(9.25, (9.245 * 10.0**2).round / 10.0**2, "Rounding
Error: (9.245 * 10.0**2).round / 10.0**2")
assert_equal(925, (9.245 * 10.0**2).round, "Rounding Error9.245
* 10.0**2).round")
assert_equal(924.5, 9.245 * 10.0**2, "Rounding Error: 9.245 *
10.0**2")
assert_equal(925, 924.5.round, "Rounding Error: 924.5.round")
assert_equal(924.5, 9.245 * 10 * 10, "Rounding Error: 9.245 * 10 *
10")
assert_equal(925, (9.245 * 10 * 10).round, "Rounding Error: (9.245
* 10 * 10).round")
assert_equal(925.0, 9.245 * 10 * 10 + 0.5, "Rounding Error: 9.245
* 10 * 10 + 0.5")
assert_equal(925, (9.245 * 10 * 10 + 0.5).floor, "Rounding Error:
(9.245 * 10 * 10 + 0.5).floor")
end
end
===================================
 
Reply With Quote
 
 
 
 
Thomas Preymesser
Guest
Posts: n/a
 
      07-22-2009
[Note: parts of this message were removed to make it a legal post.]

2009/7/22 ddoherty03 <(E-Mail Removed)>


> For example, (9.245 * 100).round should be 925, but my ruby gives 924.
>


try what your term really is:

printf("%.50f",9.245 * 100)

-Thomas

--
Thomas Preymesser
http://www.velocityreviews.com/forums/(E-Mail Removed)
http://thopre.googlepages.com/
http://thopre.wordpress.com/

Mike Ditka <http://www.brainyquote.com/quotes/authors/m/mike_ditka.html> -
"If God had wanted man to play soccer, he wouldn't have given us arms."

 
Reply With Quote
 
 
 
 
ddoherty03
Guest
Posts: n/a
 
      07-22-2009
On Jul 22, 1:13*pm, Glenn Jackman <(E-Mail Removed)> wrote:
> A workaround, convert the expression to a string and back to a number:
>
> * * num = Float("%.1f" % (9.245 * 100)).round
>
> * * puts "yippee" if num == 925
>


Glenn,

Thanks for the explanation. If I could pick your brain for a second
on how to generalize your workaround, I would greatly appreciate it.

Here is how I tried to implement the "nround" method.

######################################
class Float
def nround(n = 0)
(self * 10.0 ** n).round / 10.0 ** n
end
end
######################################

My issue is getting the workaround to deal with the parameter n
properly.

Regards,
 
Reply With Quote
 
Bil Kleb
Guest
Posts: n/a
 
      07-23-2009
ddoherty03 wrote:
>
> Thanks for the explanation. If I could pick your brain for a second
> on how to generalize your workaround, I would greatly appreciate it.


If performance isn't a problem, you could use 'bigdecimal':

http://ruby-doc.org/stdlib/libdoc/bigdecimal/rdoc/

See for example,

http://rubyquiz.com/quiz46.html

Regards,
--
Bil Kleb
http://fun3d.larc.nasa.gov
http://twitter.com/bil_kleb
 
Reply With Quote
 
ddoherty03
Guest
Posts: n/a
 
      07-23-2009
On Jul 23, 9:37*am, Glenn Jackman <(E-Mail Removed)> wrote:
> At 2009-07-22 03:22PM, "ddoherty03" wrote:
>
>
>
> > *On Jul 22, 1:13*pm, Glenn Jackman <(E-Mail Removed)> wrote:
> > > A workaround, convert the expression to a string and back to a number:
> > > * * num = Float("%.1f" % (9.245 * 100)).round
> > > * * puts "yippee" if num == 925

>
> > *Glenn,

>
> > *Thanks for the explanation. *If I could pick your brain for a second
> > *on how to generalize your workaround, I would greatly appreciate it.

>
> > *Here is how I tried to implement the "nround" method.

>
> > *######################################
> > *class Float
> > * *def nround(n = 0)
> > * * *(self * 10.0 ** n).round / 10.0 ** n
> > * *end
> > *end
> > *######################################

>
> > *My issue is getting the workaround to deal with the parameter n
> > *properly.

>
> Taking Bil Kleb's advice,
>
> * * require 'bigdecimal'
> * * require 'bigdecimal/util'
>
> * * class Float
> * * * def nround(n=0)
> * * * * (self.to_d * (10.0**n).to_d).round.to_f / 10.0**n
> * * * end
> * * end
>
> * * p 9.245.nround(2) *# => 9.25
>
> --
> Glenn Jackman
> * * Write a wise saying and your name will live forever. -- Anonymous


Bil & Glenn & Thomas,

Thanks. Performance is not a problem, so this works for me.

Much appreciated.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
*bug* *bug* *bug* David Raleigh Arnold Firefox 12 04-02-2007 03:13 AM
Formatting a number without rounding Thor W Hammer ASP .Net 2 11-22-2005 06:51 PM
rounding to integer valentin tihomirov VHDL 2 02-16-2004 10:07 AM
will Synposys Design Compiler support division by two's power and integer rounding? walala VHDL 12 09-14-2003 03:49 PM
Rounding Numbers C ASP .Net 2 08-25-2003 03:24 PM



Advertisments