Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > number accuracy

Reply
Thread Tools

number accuracy

 
 
kencavagnolo@gmail.com
Guest
Posts: n/a
 
      12-13-2006
I was digging through some code this morning after encountering a
program error and I came across the following...

For this bit of code:
#-------------------------------------------------------#
$eta = 0.5;
$etacrit = 0.97;
while ($eta <= $etacrit) {
($eta < 0.80) ? ($etastep = 0.1) :
($eta < 0.95) ? ($etastep = 0.05) :
($etastep = 0.01);
$eta += $etastep;
}
#-------------------------------------------------------#

Which *should* output:
0.6
0.7
0.8
0.85
0.9
0.95
0.96
0.97

It instead was outputing:
0.6
0.7
0.8
0.9
0.95
0.96
0.97

So I cut the code out of the main program and had it print to the
floating point accuracy at the beginning of the loop and found this:
0.100000000000000005551115123126
0.200000000000000011102230246252
0.300000000000000044408920985006
0.400000000000000022204460492503
0.500000000000000000000000000000
0.599999999999999977795539507497
0.699999999999999955591079014994
0.799999999999999933386618522491
0.899999999999999911182158029987
0.949999999999999955591079014994
0.959999999999999964472863211995
0.969999999999999973354647408996

So I now see the reason why the < 0.8 comparison failed because the
value of $eta is actually 0.7999999 and not 0.8. I know there are fixes
for this such as making the comparions with 0.79, using a sprintf,
rounding, truncating, et cetera; but my officemate checked this code on
his machine and we got the same result.

My question is... WHY? Why is the addition below 0.5 of a number which
is slightly above 0.1, but then above 0.5 the addition becomes
something which is slightly less than 0.1? And, how does one fix (if
that's possible) this problem?

Thanks.

 
Reply With Quote
 
 
 
 
Paul Lalli
Guest
Posts: n/a
 
      12-13-2006
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:
> My question is... WHY? Why is the addition below 0.5 of a number which
> is slightly above 0.1, but then above 0.5 the addition becomes
> something which is slightly less than 0.1? And, how does one fix (if
> that's possible) this problem?


This is not specific to Perl, but is a problem with computers in
general. Please read:
perldoc -q 999
perldoc perlnumber

Paul Lalli

 
Reply With Quote
 
 
 
 
usenet@DavidFilmer.com
Guest
Posts: n/a
 
      12-13-2006
(E-Mail Removed) wrote:

> So I now see the reason why the < 0.8 comparison failed because the
> value of $eta is actually 0.7999999 and not 0.8.


perldoc -q 9999

--
The best way to get a good answer is to ask a good question.
David Filmer (http://DavidFilmer.com)

 
Reply With Quote
 
Charlton Wilbur
Guest
Posts: n/a
 
      12-13-2006
>>>>> "k" == kencavagnolo <(E-Mail Removed)> writes:

k> My question is... WHY? Why is the addition below 0.5 of a
k> number which is slightly above 0.1, but then above 0.5 the
k> addition becomes something which is slightly less than 0.1?

perldoc -q 'long decimal'
http://docs.sun.com/source/806-3568/ncg_goldberg.html

k> And, how does one fix (if that's possible) this problem?

Start by understanding why and how it happens.

Charlton


--
Charlton Wilbur
(E-Mail Removed)
 
Reply With Quote
 
Ken
Guest
Posts: n/a
 
      12-13-2006
Thank you all very much.

The fix is in and everything works fine.

 
Reply With Quote
 
Tad McClellan
Guest
Posts: n/a
 
      12-14-2006
(E-Mail Removed) <(E-Mail Removed)> wrote:
> I was digging through some code this morning after encountering a
> program error and I came across the following...
>
> For this bit of code:
> #-------------------------------------------------------#
> $eta = 0.5;
> $etacrit = 0.97;
> while ($eta <= $etacrit) {
> ($eta < 0.80) ? ($etastep = 0.1) :
> ($eta < 0.95) ? ($etastep = 0.05) :
> ($etastep = 0.01);
> $eta += $etastep;
> }
> #-------------------------------------------------------#
>
> Which *should* output:



No it shouldn't.

The code contains no statements that should make output,
so it should output nothing.




--
Tad McClellan SGML consulting
(E-Mail Removed) Perl programming
Fort Worth, Texas
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Accuracy and CSS =?Utf-8?B?Q2hhcmxlc0E=?= ASP .Net 5 01-17-2006 12:47 PM
[Newbie] Type conversion - Loss of accuracy PercyP Java 5 10-20-2004 07:04 PM
Measuring the colour accuracy of digital cameras andrew29@littlepinkcloud.invalid Digital Photography 4 06-18-2004 10:24 AM
Canon digital Rebel focus accuracy and Mirror shock test jriegle Digital Photography 1 02-06-2004 11:03 PM
Dimage Xt viewfinder accuracy? Matthew Halfant Digital Photography 2 08-19-2003 07:52 PM



Advertisments