Velocity Reviews > Perl > number accuracy

# number accuracy

kencavagnolo@gmail.com
Guest
Posts: n/a

 12-13-2006
I was digging through some code this morning after encountering a
program error and I came across the following...

For this bit of code:
#-------------------------------------------------------#
\$eta = 0.5;
\$etacrit = 0.97;
while (\$eta <= \$etacrit) {
(\$eta < 0.80) ? (\$etastep = 0.1) :
(\$eta < 0.95) ? (\$etastep = 0.05) :
(\$etastep = 0.01);
\$eta += \$etastep;
}
#-------------------------------------------------------#

Which *should* output:
0.6
0.7
0.8
0.85
0.9
0.95
0.96
0.97

0.6
0.7
0.8
0.9
0.95
0.96
0.97

So I cut the code out of the main program and had it print to the
floating point accuracy at the beginning of the loop and found this:
0.100000000000000005551115123126
0.200000000000000011102230246252
0.300000000000000044408920985006
0.400000000000000022204460492503
0.500000000000000000000000000000
0.599999999999999977795539507497
0.699999999999999955591079014994
0.799999999999999933386618522491
0.899999999999999911182158029987
0.949999999999999955591079014994
0.959999999999999964472863211995
0.969999999999999973354647408996

So I now see the reason why the < 0.8 comparison failed because the
value of \$eta is actually 0.7999999 and not 0.8. I know there are fixes
for this such as making the comparions with 0.79, using a sprintf,
rounding, truncating, et cetera; but my officemate checked this code on
his machine and we got the same result.

My question is... WHY? Why is the addition below 0.5 of a number which
is slightly above 0.1, but then above 0.5 the addition becomes
something which is slightly less than 0.1? And, how does one fix (if
that's possible) this problem?

Thanks.

Paul Lalli
Guest
Posts: n/a

 12-13-2006
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:
> My question is... WHY? Why is the addition below 0.5 of a number which
> is slightly above 0.1, but then above 0.5 the addition becomes
> something which is slightly less than 0.1? And, how does one fix (if
> that's possible) this problem?

This is not specific to Perl, but is a problem with computers in
perldoc -q 999
perldoc perlnumber

Paul Lalli

usenet@DavidFilmer.com
Guest
Posts: n/a

 12-13-2006
(E-Mail Removed) wrote:

> So I now see the reason why the < 0.8 comparison failed because the
> value of \$eta is actually 0.7999999 and not 0.8.

perldoc -q 9999

--
The best way to get a good answer is to ask a good question.
David Filmer (http://DavidFilmer.com)

Charlton Wilbur
Guest
Posts: n/a

 12-13-2006
>>>>> "k" == kencavagnolo <(E-Mail Removed)> writes:

k> My question is... WHY? Why is the addition below 0.5 of a
k> number which is slightly above 0.1, but then above 0.5 the
k> addition becomes something which is slightly less than 0.1?

perldoc -q 'long decimal'
http://docs.sun.com/source/806-3568/ncg_goldberg.html

k> And, how does one fix (if that's possible) this problem?

Start by understanding why and how it happens.

Charlton

--
Charlton Wilbur
(E-Mail Removed)

Ken
Guest
Posts: n/a

 12-13-2006
Thank you all very much.

The fix is in and everything works fine.

Guest
Posts: n/a

 12-14-2006
(E-Mail Removed) <(E-Mail Removed)> wrote:
> I was digging through some code this morning after encountering a
> program error and I came across the following...
>
> For this bit of code:
> #-------------------------------------------------------#
> \$eta = 0.5;
> \$etacrit = 0.97;
> while (\$eta <= \$etacrit) {
> (\$eta < 0.80) ? (\$etastep = 0.1) :
> (\$eta < 0.95) ? (\$etastep = 0.05) :
> (\$etastep = 0.01);
> \$eta += \$etastep;
> }
> #-------------------------------------------------------#
>
> Which *should* output:

No it shouldn't.

The code contains no statements that should make output,
so it should output nothing.

--
(E-Mail Removed) Perl programming
Fort Worth, Texas