Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > floating point problem... floating indeed :(

Reply
Thread Tools

floating point problem... floating indeed :(

 
 
teeshift
Guest
Posts: n/a
 
      12-01-2006
I recently wrote a code and haven't been found the bug and start
suspecting that it was
the floating computation that causing the problem. I did a little
experiement and found the
following phenomena:

I am using

Linux source 2.6.15-27-386 #1 PREEMPT Sat Sep 16 01:51:59 UTC 2006 i686
GNU/Linux
Ubuntu is the OS.

#####################
#!/usr/bin/ruby1.9

a = 1.0

a.to_f

while(a < 15.0)
a = a + 0.1
print a,"\t",a * a,"\n"
end
#####################

and the output is...(as following) quite odd. Anyone can explain it?
or a way to avoid it (while
keeping the precision)


1.1 1.21
1.2 1.44
1.3 1.69
1.4 1.96
1.5 2.25
1.6 2.56
1.7 2.89
1.8 3.24
1.9 3.61
2.0 4.0
2.1 4.41
2.2 4.84
2.3 5.29000000000001
2.4 5.76000000000001
2.5 6.25000000000001
2.6 6.76000000000001
2.7 7.29000000000001
2.8 7.84000000000001
2.9 8.41000000000001
3.0 9.00000000000001
3.1 9.61000000000001
3.2 10.24
3.3 10.89
3.4 11.56
3.5 12.25
3.6 12.96
3.7 13.69
3.8 14.44
3.9 15.21
4.0 16.0
4.1 16.81
4.2 17.64
4.3 18.49
4.4 19.36
4.5 20.25
4.6 21.16
4.7 22.09
4.8 23.04
4.9 24.01
5.0 25.0
5.1 26.01
5.2 27.04
5.3 28.09
5.4 29.16
5.5 30.25
5.6 31.36
5.7 32.49
5.8 33.64
5.9 34.81
6.0 35.9999999999999
6.1 37.2099999999999
6.19999999999999 38.4399999999999
6.29999999999999 39.6899999999999
6.39999999999999 40.9599999999999
6.49999999999999 42.2499999999999
6.59999999999999 43.5599999999999

 
Reply With Quote
 
 
 
 
Wilson Bilkovich
Guest
Posts: n/a
 
      12-01-2006
On 11/30/06, teeshift <(E-Mail Removed)> wrote:
> I recently wrote a code and haven't been found the bug and start
> suspecting that it was
> the floating computation that causing the problem. I did a little
> experiement and found the
> following phenomena:
>
> I am using
>
> Linux source 2.6.15-27-386 #1 PREEMPT Sat Sep 16 01:51:59 UTC 2006 i686
> GNU/Linux
> Ubuntu is the OS.


See this writeup for some details on why this happens.
http://en.wikipedia.org/wiki/IEEE_754

 
Reply With Quote
 
 
 
 
Chris Pearl
Guest
Posts: n/a
 
      12-01-2006
Someone else might wish to give a more concrete explanation, but as a
rule, floating-point calculations - using the usual Float datatype -
are not accurate, and prone to rounding errors such as the above.

If you're dealing with limited precision (i.e. X digits after the
decimal point) you should consider using the Decimal datatype instead.

On 12/1/06, teeshift <(E-Mail Removed)> wrote:
> I recently wrote a code and haven't been found the bug and start
> suspecting that it was
> the floating computation that causing the problem. I did a little
> experiement and found the
> following phenomena:
>
> I am using
>
> Linux source 2.6.15-27-386 #1 PREEMPT Sat Sep 16 01:51:59 UTC 2006 i686
> GNU/Linux
> Ubuntu is the OS.
>
> #####################
> #!/usr/bin/ruby1.9
>
> a = 1.0
>
> a.to_f
>
> while(a < 15.0)
> a = a + 0.1
> print a,"\t",a * a,"\n"
> end
> #####################
>
> and the output is...(as following) quite odd. Anyone can explain it?
> or a way to avoid it (while
> keeping the precision)
>
>
> 1.1 1.21
> 1.2 1.44
> 1.3 1.69
> 1.4 1.96
> 1.5 2.25
> 1.6 2.56
> 1.7 2.89
> 1.8 3.24
> 1.9 3.61
> 2.0 4.0
> 2.1 4.41
> 2.2 4.84
> 2.3 5.29000000000001
> 2.4 5.76000000000001
> 2.5 6.25000000000001
> 2.6 6.76000000000001
> 2.7 7.29000000000001
> 2.8 7.84000000000001
> 2.9 8.41000000000001
> 3.0 9.00000000000001
> 3.1 9.61000000000001
> 3.2 10.24
> 3.3 10.89
> 3.4 11.56
> 3.5 12.25
> 3.6 12.96
> 3.7 13.69
> 3.8 14.44
> 3.9 15.21
> 4.0 16.0
> 4.1 16.81
> 4.2 17.64
> 4.3 18.49
> 4.4 19.36
> 4.5 20.25
> 4.6 21.16
> 4.7 22.09
> 4.8 23.04
> 4.9 24.01
> 5.0 25.0
> 5.1 26.01
> 5.2 27.04
> 5.3 28.09
> 5.4 29.16
> 5.5 30.25
> 5.6 31.36
> 5.7 32.49
> 5.8 33.64
> 5.9 34.81
> 6.0 35.9999999999999
> 6.1 37.2099999999999
> 6.19999999999999 38.4399999999999
> 6.29999999999999 39.6899999999999
> 6.39999999999999 40.9599999999999
> 6.49999999999999 42.2499999999999
> 6.59999999999999 43.5599999999999
>
>
>


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Share-Point-2010 ,Share-Point -2010 Training , Share-point-2010Hyderabad , Share-point-2010 Institute Saraswati lakki ASP .Net 0 01-06-2012 06:39 AM
converting floating point to fixed point H aka N VHDL 15 03-02-2006 02:26 PM
floating point to fixed point conversion riya1012@gmail.com C Programming 4 02-22-2006 05:56 PM
Fixed-point format for floating-point numbers Motaz Saad Java 7 11-05-2005 05:33 PM
Duct tape is -indeed- your friend Silverstrand The Lounge 4 09-16-2005 02:30 PM



Advertisments