On Fri, 31 Mar 2006, Ryan Allen defenestrated me:
> Hi Guys (and Gals!)
>
> This is my first post on the list, so Hello!
>
> Something very strange is happening (as far as I can tell) with a
> float comparison; here is the code in question:
>
> ("55.59".to_f / 100) == 0.5559 # returns false
>
>
> It doesn't seem to like .59's... What did .59 ever do?
>
> I picked this up while writing a unit test, I've run this code in irb
> and I'm not sure what's going on...
>
> Can anyone shed some light on this?
Looking at some IEEE spec will explain this in agonizing detail, but
simply stated numbers in a computer are stored in a base 2 format. The
math you are doing is base 10. Not all base 10 values can be perfectly
represented in base 2. This can cause calculations to not quite come
out right.
A book on numerical analysis can also be a good intro on this stuff.
-Tom
--
+
http://www.tc.umn.edu/~enebo +---- private.php?do=newpm&u= ----+
| Thomas E Enebo, Protagonist | "Luck favors the prepared |
| | mind." -Louis Pasteur |