Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C++ > small numerical differences in floating point result between wintel and Sun/SPARC

Reply
Thread Tools

small numerical differences in floating point result between wintel and Sun/SPARC

 
 
Herman D. Knoble
Guest
Posts: n/a
 
      12-14-2004
First, we assuming that the program in question here is not displaying
or writing results past the precision of the corresponding variables.

How many "places" (mantissa bits) are you talking about here?
If the results differ by only the last mantissa bit or two I would not
worry about this. If results differ by more than the least significant
bit or two, there could be reason to suspect compiler options, like /OP
(or, for example, XLF under AIX the compiler option -qfloat=nomaf).

For the case where results differ by more that a mantissa bit or two,
it could also be possible that the code itself is not completely
numerically stable. One of the ways to determine if a model is stable
is in fact to perturb the input (or platform) slightly and
compare results.

Skip

On Mon, 13 Dec 2004 22:00:03 -0500, JS <(E-Mail Removed)> wrote:

-|We have the same floating point intensive C++ program that runs on
-|Windows on Intel chip and on Sun Solaris on SPARC chips. The program
-|reads the exactly the same input files on the two platforms. However,
-|they generate slightly different results for floating point numbers.
-|
-|Are they really supposed to generate exactly the same results? I
-|guess so because both platforms are supposed to be IEEE floating point
-|standard (754?) compliant. I have turned on the Visual C++ compile
-|flags which will make sure the Windows produce standard compliant code
-|(the /Op flags). However, they still produce different results. I
-|suspect that this may be due to a commerical mathematical library that
-|we use which can't be compiled using /Op option. If I had recompiled
-|everything using /Op option, the two should have produced the same
-|results.
-|
-|Am I right?
-|
-|Thanks a lot.


Herman D. (Skip) Knoble, Research Associate
(a computing professional for 38 years)
Email: SkipKnobleLESS at SPAMpsu dot edu
Web: http://www.personal.psu.edu/hdk
Penn State Information Technology Services
Academic Services and Emerging Technologies
Graduate Education and Research Services
Penn State University
214C Computer Building
University Park, PA 16802-21013
Phone:+1 814 865-0818 Fax:+1 814 863-7049
 
Reply With Quote
 
 
 
 
websnarf@gmail.com
Guest
Posts: n/a
 
      12-14-2004
Sun and Intel should produce identical results for
negation,+,-,*,/,sqrt() operations, because this is required for IEEE
754 compliance, and both implement double and float on the same sized
floating point values (older x86 compilers used to use 80bit for all FP
temporaries which could definately be a source of differences, but VC++
doesn't do this anymore). You should also expect identical output for
things like fprem(), ceil(), floor(), modf(), and so on. All of these
operations have well known and finite ways of being calculated to the
best and closest possible result, which is what the IEEE 754 specifies.
Results can be different if you put the processors into different
rounding modes -- I thought that the ANSI C specification was supposed
to specify a consistent rounding mode, so there shouldn't be any
differences because of that, but I could be wrong.

The problem is everything else. sin(), cos(), log(), exp(), atanh()
etc, ... these guys are not even consistent between Athlons and
Pentiums. The reason is that there is no known finite algorithm for
computing these guys to the exactly corrected rounded result. There
are plenty of ways to compute them within an accuracy of "1 ulp"
(i.e., off by at most one unit in the last siginificant bit.)

But I am not the foremost expert on this stuff (and it looks like the
other posters here know even less than me.) This question is more
appropriate for a group like comp.arch.arithmetic where there are
plenty of posters there who are expert on this.
--
Paul Hsieh
http://www.pobox.com/~qed/
http://bstring.sf.net/

 
Reply With Quote
 
 
 
 
Gerry Thomas
Guest
Posts: n/a
 
      12-15-2004

"JS" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> We have the same floating point intensive C++ program that runs on
> Windows on Intel chip and on Sun Solaris on SPARC chips. The program
> reads the exactly the same input files on the two platforms. However,
> they generate slightly different results for floating point numbers.
>
> Are they really supposed to generate exactly the same results?


For fp intensive programs, whatever the language, no.

> I
> guess so because both platforms are supposed to be IEEE floating point
> standard (754?) compliant. I have turned on the Visual C++ compile
> flags which will make sure the Windows produce standard compliant code
> (the /Op flags). However, they still produce different results. I
> suspect that this may be due to a commerical mathematical library that
> we use which can't be compiled using /Op option. If I had recompiled
> everything using /Op option, the two should have produced the same
> results.
>
> Am I right?
>


In all probability, no, .

For an update to the over-hackneyed Goldberg variations, see Parker,
Pierce, and Eggert , "Monte Carlo Arithmetic: exploiting randomness in
floating-point arithmetic," Comp. Sci. & Eng., pp. 58-68, July 2000.
..> Thanks a lot.

--
You're Welcome,
Gerry T.
______
"Competent engineers rightly distrust all numerical computations and seek
corroboration from alternative numerical methods, from scale models, from
prototypes, from experience... ." -- William V. Kahan.



 
Reply With Quote
 
JS
Guest
Posts: n/a
 
      12-15-2004


On Tue, 14 Dec 2004 08:20:41 -0500, Herman D. Knoble
<(E-Mail Removed)> wrote:

>First, we assuming that the program in question here is not displaying
>or writing results past the precision of the corresponding variables.
>
>How many "places" (mantissa bits) are you talking about here?


All the calculations are done in double. The output result has
precision (in the format of 999999999.666666) that is well within the
precision limit of double. So this is not due not incorrect precision
used for displaying or outputting the result.

>If the results differ by only the last mantissa bit or two I would not
>worry about this. If results differ by more than the least significant
>bit or two, there could be reason to suspect compiler options, like /OP
>(or, for example, XLF under AIX the compiler option -qfloat=nomaf).
>


Some posters don't seem to know about /Op option. From Microsoft doc:

"By default, the compiler uses the coprocessor’s 80-bit registers to
hold the intermediate results of floating-point calculations. This
increases program speed and decreases program size. However, because
the calculation involves floating-point data types that are
represented in memory by less than 80 bits, carrying the extra bits of
precision (80 bits minus the number of bits in a smaller
floating-point type) through a lengthy calculation can produce
inconsistent results.

With /Op, the compiler loads data from memory prior to each
floating-point operation and, if assignment occurs, writes the results
back to memory upon completion. Loading the data prior to each
operation guarantees that the data does not retain any significance
greater than the capacity of its type."

Thus the difference I got doesn't seem to come from the 80-bit vs 64
bit issue mentioned by some posters.

>For the case where results differ by more that a mantissa bit or two,
>it could also be possible that the code itself is not completely
>numerically stable. One of the ways to determine if a model is stable
>is in fact to perturb the input (or platform) slightly and
>compare results.
>
>Skip
>
>On Mon, 13 Dec 2004 22:00:03 -0500, JS <(E-Mail Removed)> wrote:
>
>-|We have the same floating point intensive C++ program that runs on
>-|Windows on Intel chip and on Sun Solaris on SPARC chips. The program
>-|reads the exactly the same input files on the two platforms. However,
>-|they generate slightly different results for floating point numbers.
>-|
>-|Are they really supposed to generate exactly the same results? I
>-|guess so because both platforms are supposed to be IEEE floating point
>-|standard (754?) compliant. I have turned on the Visual C++ compile
>-|flags which will make sure the Windows produce standard compliant code
>-|(the /Op flags). However, they still produce different results. I
>-|suspect that this may be due to a commerical mathematical library that
>-|we use which can't be compiled using /Op option. If I had recompiled
>-|everything using /Op option, the two should have produced the same
>-|results.
>-|
>-|Am I right?
>-|
>-|Thanks a lot.
>
>
> Herman D. (Skip) Knoble, Research Associate
> (a computing professional for 38 years)
> Email: SkipKnobleLESS at SPAMpsu dot edu
> Web: http://www.personal.psu.edu/hdk
> Penn State Information Technology Services
> Academic Services and Emerging Technologies
> Graduate Education and Research Services
> Penn State University
> 214C Computer Building
> University Park, PA 16802-21013
> Phone:+1 814 865-0818 Fax:+1 814 863-7049


 
Reply With Quote
 
Gerry Thomas
Guest
Posts: n/a
 
      12-15-2004

"JS" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...

[Stuff from the ever-helpful Skip Knoble elided without prejudice]

>
> Some posters don't seem to know about /Op option.


Not me, and a bunch besides!

FYI, most clf's posters are rabidly anti Microsoft and could give a toss
what it has to say about anything, least of all in respect to it's own
products, .

> From Microsoft doc: (sic DOC!)


> "By default, the compiler uses the coprocessor's 80-bit registers to
> hold the intermediate results of floating-point calculations. This
> increases program speed and decreases program size. However, because
> the calculation involves floating-point data types that are
> represented in memory by less than 80 bits, carrying the extra bits of
> precision (80 bits minus the number of bits in a smaller
> floating-point type) through a lengthy calculation can produce
> inconsistent results.
>
> With /Op, the compiler loads data from memory prior to each
> floating-point operation and, if assignment occurs, writes the results
> back to memory upon completion. Loading the data prior to each
> operation guarantees that the data does not retain any significance
> greater than the capacity of its type."
>


IIRC (and I'm sure I do) this is a verbatim quote from Microsoft's Fortran
PowerStation documentation (sic DOC!). This product commands the least of
respect in these environs. (Word to the wise, look into VC++ ~2 for what
you seek!)

> Thus the difference I got doesn't seem to come from the 80-bit vs 64
> bit issue mentioned by some posters.


How come? I've no idea why you're beginning to remind me of Ozmandias but
I'm wondering if it's that Alexander the Wimp movie I endured on the
weekend.

--
You're Welcome,
Gerry T.
______
"Things are not what they seem; or, to be more accurate, they are not only
what they seem, but very much else besides." -- Aldous Huxley.



 
Reply With Quote
 
Richard Maine
Guest
Posts: n/a
 
      12-15-2004
http://www.velocityreviews.com/forums/(E-Mail Removed) (Gordon Burditt) writes:

> I don't believe the == operator applied to calculated floating-point
> results is ever required to return 1. Nor does it have to be consistent
> about it. An implementation like that probably won't sell well, but
> ANSI C allows it.


I'd argue that the same thing applies to Fortran (ok, except that
I'd talk about == returning .true. instead of 1). There are probably
people who would disagree with me, but that would be my interpretation
of the standard.

The comment about such a compiler not selling well would also
apply. I'd likely not get too much argument on that part.

--
Richard Maine
email: my last name at domain
domain: summertriangle dot net
 
Reply With Quote
 
James Giles
Guest
Posts: n/a
 
      12-15-2004
Richard Maine wrote:
> (E-Mail Removed) (Gordon Burditt) writes:
>
>> I don't believe the == operator applied to calculated floating-point
>> results is ever required to return 1. Nor does it have to be consistent
>> about it. An implementation like that probably won't sell well, but
>> ANSI C allows it.

>
> I'd argue that the same thing applies to Fortran (ok, except that
> I'd talk about == returning .true. instead of 1). There are probably
> people who would disagree with me, but that would be my interpretation
> of the standard.


I'd argue that any language that claims IEEE compliance must
get a true result under some predictable circumstances. Not as
many such circumstances as originally intended by the IEEE
committee! But, *some* such circumstances exist. For example,
I think that 0.1 == 0.1 must be true, since the IEEE standard
places minimum requirements on the precision of decimal
to binary conversion. Of course, whether the value of a literal
constitutes a "calculated" result is maybe a matter of opinion.

--
J. Giles

"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies." -- C. A. R. Hoare


 
Reply With Quote
 
Gordon Burditt
Guest
Posts: n/a
 
      12-15-2004
>>> I don't believe the == operator applied to calculated floating-point
>>> results is ever required to return 1. Nor does it have to be consistent
>>> about it. An implementation like that probably won't sell well, but
>>> ANSI C allows it.

>>
>> I'd argue that the same thing applies to Fortran (ok, except that
>> I'd talk about == returning .true. instead of 1). There are probably
>> people who would disagree with me, but that would be my interpretation
>> of the standard.

>
>I'd argue that any language that claims IEEE compliance must
>get a true result under some predictable circumstances. Not as
>many such circumstances as originally intended by the IEEE
>committee! But, *some* such circumstances exist. For example,


For example, d == d where d is not a NaN.

>I think that 0.1 == 0.1 must be true, since the IEEE standard
>places minimum requirements on the precision of decimal
>to binary conversion.


But does it place MAXIMUM requirements? 0.1 is an infinite repeating
decimal in binary. Any extra precision on one side but not the other
is likely to cause a mismatch.

>Of course, whether the value of a literal
>constitutes a "calculated" result is maybe a matter of opinion.


Ok, I didn't define "calculated result", but the intent was something
like "the result of a floating-point binary operator such as +, -, *, /".
Although multiplication by 0.0 should come out exact.

0.1 == 0.1 has a much better chance than, say, (0.1 + 0.0) == (0.1 + 0.0) .

Gordon L. Burditt
 
Reply With Quote
 
James Giles
Guest
Posts: n/a
 
      12-15-2004
Gordon Burditt wrote:
....
>> I'd argue that any language that claims IEEE compliance must
>> get a true result under some predictable circumstances. Not as
>> many such circumstances as originally intended by the IEEE
>> committee! But, *some* such circumstances exist. For example,

....
>> I think that 0.1 == 0.1 must be true, since the IEEE standard
>> places minimum requirements on the precision of decimal
>> to binary conversion.

>
> But does it place MAXIMUM requirements? 0.1 is an infinite repeating
> decimal in binary. Any extra precision on one side but not the other
> is likely to cause a mismatch.


The IEEE standard requires that, for numbers in this range, the
result must correctly rounded to the target precision. Since they're
both the same type and KIND (hence, the same target precision),
and since there is but one value that's closest to one tenth that's
representable as an IEEE single precision number, they must
give the same result.

--
J. Giles

"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies." -- C. A. R. Hoare


 
Reply With Quote
 
Gerry Thomas
Guest
Posts: n/a
 
      12-15-2004

"James Giles" <(E-Mail Removed)> wrote in message
news:24Pvd.1095905$(E-Mail Removed)...

[...]
>
> The IEEE standard requires that, for numbers in this range, the
> result must correctly rounded to the target precision. Since they're
> both the same type and KIND (hence, the same target precision),
> and since there is but one value that's closest to one tenth that's
> representable as an IEEE single precision number, they must
> give the same result.
>


How profound!

--
You're Welcome,
Gerry T.
______
"Facts are meaningless. You could use facts to prove anything that's even
remotely true." -- Homer Simpson.


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Share-Point-2010 ,Share-Point -2010 Training , Share-point-2010Hyderabad , Share-point-2010 Institute Saraswati lakki ASP .Net 0 01-06-2012 06:39 AM
DEVELOP THE WINNING EDGE, SMALL DIFFERENCES IN YOUR PERFORMANCE CANLEAD TO LARGE DIFFERENCES IN YOUR RESULTS Home_Job_opportunity C Programming 0 01-14-2009 03:51 PM
DEVELOP THE WINNING EDGE, SMALL DIFFERENCES IN YOUR PERFORMANCE CANLEAD TO LARGE DIFFERENCES IN YOUR RESULTS Home_Job_opportunity C Programming 0 01-08-2009 04:31 PM
floating point problem... floating indeed :( teeshift Ruby 2 12-01-2006 01:16 AM
small numerical differences in floating point result between wintel and Sun/SPARC JS C Programming 30 12-20-2004 08:55 PM



Advertisments