Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C++ > small numerical differences in floating point result between wintel and Sun/SPARC

Reply
Thread Tools

small numerical differences in floating point result between wintel and Sun/SPARC

 
 
Christian Bau
Guest
Posts: n/a
 
      12-15-2004
In article <(E-Mail Removed)>,
JS <(E-Mail Removed)> wrote:

> On Tue, 14 Dec 2004 08:20:41 -0500, Herman D. Knoble
> <(E-Mail Removed)> wrote:
>
> >First, we assuming that the program in question here is not displaying
> >or writing results past the precision of the corresponding variables.
> >
> >How many "places" (mantissa bits) are you talking about here?

>
> All the calculations are done in double. The output result has
> precision (in the format of 999999999.666666) that is well within the
> precision limit of double. So this is not due not incorrect precision
> used for displaying or outputting the result.
>
> >If the results differ by only the last mantissa bit or two I would not
> >worry about this. If results differ by more than the least significant
> >bit or two, there could be reason to suspect compiler options, like /OP
> >(or, for example, XLF under AIX the compiler option -qfloat=nomaf).
> >

>
> Some posters don't seem to know about /Op option. From Microsoft doc:
>
> "By default, the compiler uses the coprocessoršs 80-bit registers to
> hold the intermediate results of floating-point calculations. This
> increases program speed and decreases program size. However, because
> the calculation involves floating-point data types that are
> represented in memory by less than 80 bits, carrying the extra bits of
> precision (80 bits minus the number of bits in a smaller
> floating-point type) through a lengthy calculation can produce
> inconsistent results.
>
> With /Op, the compiler loads data from memory prior to each
> floating-point operation and, if assignment occurs, writes the results
> back to memory upon completion. Loading the data prior to each
> operation guarantees that the data does not retain any significance
> greater than the capacity of its type."


That isn't enough. First, the x86 FPU must be switched to 64 bit mode,
otherwise you will occasionally get different result due to double
rounding (about 1 in 2000 chance for a single random operation). The
load/store will make sure that overflows and underflows are reproduced
(let x = 1e300. (x * x) / x should be Inf / x = Inf. Without storing and
reloading the result of x * x, x * x = 1e600, (x * x) / x = 1e300. You
still run into trouble because underflow in a multiplication and
division can give different results due to double rounding. You can
always switch to Java + strict FP mode to get reproducible results.
(Which will be damned slow but it will get everything right, that is why
nonstrict FP mode had to be introduced. It is slow and gives sometimes
different results).
 
Reply With Quote
 
 
 
 
Tim Prince
Guest
Posts: n/a
 
      12-15-2004

"Christian Bau" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> In article <(E-Mail Removed)>,
> JS <(E-Mail Removed)> wrote:
>


> > Some posters don't seem to know about /Op option. From Microsoft doc:
> >
> > "By default, the compiler uses the coprocessoršs 80-bit registers to
> > hold the intermediate results of floating-point calculations. This
> > increases program speed and decreases program size. However, because
> > the calculation involves floating-point data types that are
> > represented in memory by less than 80 bits, carrying the extra bits of
> > precision (80 bits minus the number of bits in a smaller
> > floating-point type) through a lengthy calculation can produce
> > inconsistent results.
> >
> > With /Op, the compiler loads data from memory prior to each
> > floating-point operation and, if assignment occurs, writes the results
> > back to memory upon completion. Loading the data prior to each
> > operation guarantees that the data does not retain any significance
> > greater than the capacity of its type."

>
> That isn't enough. First, the x86 FPU must be switched to 64 bit mode,
> otherwise you will occasionally get different result due to double
> rounding (about 1 in 2000 chance for a single random operation). The
> load/store will make sure that overflows and underflows are reproduced
> (let x = 1e300. (x * x) / x should be Inf / x = Inf. Without storing and
> reloading the result of x * x, x * x = 1e600, (x * x) / x = 1e300. You
> still run into trouble because underflow in a multiplication and
> division can give different results due to double rounding.

That Microsoft doc is misleading, given that they always (at least since
VS6)
set the FPU to 53-bit precision, which I assume is what you mean by
"64-bit mode." I couldn't find any reference in the thread as to whether
SSE
code generation was invoked (no such option in VS6, but presumably preferred
in later versions).


 
Reply With Quote
 
 
 
 
Lawrence Kirby
Guest
Posts: n/a
 
      12-15-2004
On Tue, 14 Dec 2004 20:34:14 -0500, JS wrote:

>
>
> On Tue, 14 Dec 2004 08:20:41 -0500, Herman D. Knoble
> <(E-Mail Removed)> wrote:
>
>>First, we assuming that the program in question here is not displaying
>>or writing results past the precision of the corresponding variables.
>>
>>How many "places" (mantissa bits) are you talking about here?

>
> All the calculations are done in double. The output result has
> precision (in the format of 999999999.666666) that is well within the
> precision limit of double. So this is not due not incorrect precision
> used for displaying or outputting the result.


Can you create a small program that demonstrates the problem? Perhaps a
little summation loop or something. With a particular example it would be
possible to pin down the source of any inconsistency.

....

> Some posters don't seem to know about /Op option. From Microsoft doc:
>
> "By default, the compiler uses the coprocessor's 80-bit registers to
> hold the intermediate results of floating-point calculations. This
> increases program speed and decreases program size. However, because
> the calculation involves floating-point data types that are
> represented in memory by less than 80 bits, carrying the extra bits of
> precision (80 bits minus the number of bits in a smaller
> floating-point type) through a lengthy calculation can produce
> inconsistent results.


I think somebody else pointed out that recent Microsoft compilers set the
FPU to 53 bits precision anyway. Another thing to consider is rounding
mode. They may well be the same but this needs to be checked.

Also I have an inherent mistrust of options like /Op. gcc certainly
doesn't get everything right with its version of that option, maybe other
compilers do but it can be hard on x86 especially and I wouldn't trust
it without some verification.

Lawrence
 
Reply With Quote
 
Dick Hendrickson
Guest
Posts: n/a
 
      12-15-2004


James Giles wrote:
> Gordon Burditt wrote:
> ...
>
>>>I'd argue that any language that claims IEEE compliance must
>>>get a true result under some predictable circumstances. Not as
>>>many such circumstances as originally intended by the IEEE
>>>committee! But, *some* such circumstances exist. For example,

>
> ...
>
>>>I think that 0.1 == 0.1 must be true, since the IEEE standard
>>>places minimum requirements on the precision of decimal
>>>to binary conversion.

>>
>>But does it place MAXIMUM requirements? 0.1 is an infinite repeating
>>decimal in binary. Any extra precision on one side but not the other
>>is likely to cause a mismatch.

>
>
> The IEEE standard requires that, for numbers in this range, the
> result must correctly rounded to the target precision. Since they're
> both the same type and KIND (hence, the same target precision),
> and since there is but one value that's closest to one tenth that's
> representable as an IEEE single precision number, they must
> give the same result.
>

Fortran is a little more restrictive in this case. It
requires that all constants written in the same form
have the same value. So 0.1 always has the same value
within a program. The non-IEEE parts of Fortran don't
specify how accurate the representation must be.

Dick Hendrickson

 
Reply With Quote
 
Dik T. Winter
Guest
Posts: n/a
 
      12-16-2004
In article <(E-Mail Removed)> Lawrence Kirby <(E-Mail Removed)> writes:
....
> Also I have an inherent mistrust of options like /Op. gcc certainly
> doesn't get everything right with its version of that option, maybe other
> compilers do but it can be hard on x86 especially and I wouldn't trust
> it without some verification.


Indeed. Is an expression like a * b + c indeed calculated with 53 bits
mantissa only?
--
dik t. winter, cwi, kruislaan 413, 1098 sj amsterdam, nederland, +31205924131
home: bovenover 215, 1025 jn amsterdam, nederland; http://www.cwi.nl/~dik/
 
Reply With Quote
 
JS
Guest
Posts: n/a
 
      12-16-2004
How can you switch FPU to 64 bit mode? Thanks.

On Wed, 15 Dec 2004 08:54:36 +0000, Christian Bau
<(E-Mail Removed)> wrote:


>That isn't enough. First, the x86 FPU must be switched to 64 bit mode,
>otherwise you will occasionally get different result due to double
>rounding (about 1 in 2000 chance for a single random operation). The
>load/store will make sure that overflows and underflows are reproduced
>(let x = 1e300. (x * x) / x should be Inf / x = Inf. Without storing and
>reloading the result of x * x, x * x = 1e600, (x * x) / x = 1e300. You
>still run into trouble because underflow in a multiplication and
>division can give different results due to double rounding. You can
>always switch to Java + strict FP mode to get reproducible results.
>(Which will be damned slow but it will get everything right, that is why
>nonstrict FP mode had to be introduced. It is slow and gives sometimes
>different results).


 
Reply With Quote
 
JS
Guest
Posts: n/a
 
      12-16-2004
Sorry, I hasn't been able to create a small program that demonstrates
the problem.

>
>Can you create a small program that demonstrates the problem? Perhaps a
>little summation loop or something. With a particular example it would be
>possible to pin down the source of any inconsistency.
>
>...
>


 
Reply With Quote
 
Tim Prince
Guest
Posts: n/a
 
      12-16-2004

"Dik T. Winter" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> In article <(E-Mail Removed)> Lawrence Kirby

<(E-Mail Removed)> writes:
> ...
> > Also I have an inherent mistrust of options like /Op. gcc certainly
> > doesn't get everything right with its version of that option, maybe

other
> > compilers do but it can be hard on x86 especially and I wouldn't trust
> > it without some verification.

>
> Indeed. Is an expression like a * b + c indeed calculated with 53 bits
> mantissa only?

Yes, because 53-bit precision mode is set, not on account of /Op. floats
also will be done with effective promotion of intermediates to double,
unless SSE is selected.


 
Reply With Quote
 
Merrill & Michele
Guest
Posts: n/a
 
      12-20-2004

"JS" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Sorry, I hasn't been able to create a small program that demonstrates
> the problem.
>
> >
> >Can you create a small program that demonstrates the problem? Perhaps a
> >little summation loop or something. With a particular example it would be
> >possible to pin down the source of any inconsistency.


Does the Standard have anything to say about machine epsilon? MPJ


 
Reply With Quote
 
Victor Bazarov
Guest
Posts: n/a
 
      12-20-2004
Merrill & Michele wrote:
> "JS" <(E-Mail Removed)> wrote in message
> news:(E-Mail Removed)...
>
>>Sorry, I hasn't been able to create a small program that demonstrates
>>the problem.
>>
>>
>>>Can you create a small program that demonstrates the problem? Perhaps a
>>>little summation loop or something. With a particular example it would be
>>>possible to pin down the source of any inconsistency.

>
>
> Does the Standard have anything to say about machine epsilon? MPJ


Which Standard do you mean to ask about? You cross-posted to three
different language newsgroups. The C++ Standard says that the member
'epsilon' of 'std::numeric_limits' returns the difference between 1
and the least value greater than 1 that is representable. It does not
say specify the actual value returned from 'epsilon'. The C Standard
does give the maximum acceptable values of 'FLT_EPSILON', 'DBL_EPSILON'
and 'LDBL_EPSILON', but your implementation is free to provide its own,
smaller values. And I have no idea about the Fortran Standard.

V
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Share-Point-2010 ,Share-Point -2010 Training , Share-point-2010Hyderabad , Share-point-2010 Institute Saraswati lakki ASP .Net 0 01-06-2012 06:39 AM
DEVELOP THE WINNING EDGE, SMALL DIFFERENCES IN YOUR PERFORMANCE CANLEAD TO LARGE DIFFERENCES IN YOUR RESULTS Home_Job_opportunity C Programming 0 01-14-2009 03:51 PM
DEVELOP THE WINNING EDGE, SMALL DIFFERENCES IN YOUR PERFORMANCE CANLEAD TO LARGE DIFFERENCES IN YOUR RESULTS Home_Job_opportunity C Programming 0 01-08-2009 04:31 PM
floating point problem... floating indeed :( teeshift Ruby 2 12-01-2006 01:16 AM
small numerical differences in floating point result between wintel and Sun/SPARC JS C Programming 30 12-20-2004 08:55 PM



Advertisments