Velocity Reviews > Re: Python fails on math

# Re: Python fails on math

Grant Edwards
Guest
Posts: n/a

 02-22-2011
On 2011-02-22, christian schulze <(E-Mail Removed)> wrote:
> Hey guys,
>
> I just found out, how much Python fails on simple math.

Python doesn't do math.

It does floating point operations.

They're different. Seriously.

On all of the platforms I know of, it's IEEE 754 (base-2) floating
point.

> I checked a simple equation for a friend.
>
>
Code:
```
>>>> from math import e as e
>>>> from math import sqrt as sqrt
>>>> 2*e*sqrt(3) - 2*e == 2*e*(sqrt(3) - 1)
> False
>```
>
> So WTF?

Python doesn't do equations. Python does floating point operations.

[And it does them in _base_2_ -- which is important, because it makes
things even more difficult.]

> The equation is definitive equivalent. (See http://mathbin.net/59158)

But, the two floating point expressions you provided are not
equivalent.

Remember, you're not doing math with Python.

You're doing binary floating point operations.

> #1:
>>>> 2.0 * e * sqrt(3.0) - 2.0 * e

> 3.9798408154464964
>
> #2:
>>>> 2.0 * e * (sqrt(3.0) -1.0)

> 3.979840815446496
>
> I was wondering what exactly is failing here. The math module?
> Python, or the IEEE specifications?

I'm afraid it's the user that's failing. Unfortunately, in many
situations using floating point is neither intuitive nor easy to get
right.

http://docs.python.org/tutorial/floatingpoint.html
http://en.wikipedia.org/wiki/Floating_point

--
Grant Edwards grant.b.edwards Yow! I like your SNOOPY
at POSTER!!
gmail.com

Roy Smith
Guest
Posts: n/a

 02-22-2011
In article <ik0rmr\$ck4\$(E-Mail Removed)>,
Grant Edwards <(E-Mail Removed)> wrote:

> Python doesn't do equations. Python does floating point operations.

More generally, all general-purpose programming languages have the same
problem. You'll see the same issues in Fortran, C, Java, Ruby, Pascal,
etc, etc. You'll see the same problem if you punch the numbers into a
hand calculator. It's just the nature of how digital computers do
floating point calculations.

If you really want something that understands that:

>>>> 2*e*sqrt(3) - 2*e == 2*e*(sqrt(3) - 1)

you need to be looking at specialized math packages like Mathematica and
things of that ilk.

Grant Edwards
Guest
Posts: n/a

 02-22-2011
On 2011-02-22, Roy Smith <(E-Mail Removed)> wrote:
> In article <ik0rmr\$ck4\$(E-Mail Removed)>,
> Grant Edwards <(E-Mail Removed)> wrote:
>
>> Python doesn't do equations. Python does floating point operations.

>
> More generally, all general-purpose programming languages have the same
> problem. You'll see the same issues in Fortran, C, Java, Ruby, Pascal,
> etc, etc. You'll see the same problem if you punch the numbers into a
> hand calculator.

Some hand calculators use base-10 (BCD) floating point, so the
problems aren't exactly the same, but they're very similar.

--
Grant Edwards grant.b.edwards Yow! YOU PICKED KARL
at MALDEN'S NOSE!!
gmail.com

John Nagle
Guest
Posts: n/a

 02-23-2011
On 2/22/2011 9:59 AM, Roy Smith wrote:
> In article<ik0rmr\$ck4\$(E-Mail Removed)>,
> Grant Edwards<(E-Mail Removed)> wrote:
>
>> Python doesn't do equations. Python does floating point operations.

>
> More generally, all general-purpose programming languages have the same
> problem. You'll see the same issues in Fortran, C, Java, Ruby, Pascal,
> etc, etc.

Not quite. CPython has the problem that it "boxes" its floating
point numbers. After each operation, the value is stored back into
a 64-bit space.

The IEEE 754 compliant FPU on most machines today, though, has
an 80-bit internal representation. If you do a sequence of
operations that retain all the intermediate results in the FPU
registers, you get 16 more bits of precision than if you store
after each operation. Rounding occurs when the 80-bit value is
forced back to 64 bits.

So it's quite possible that this would look like an equality
in C, or ShedSkin, or maybe PyPy (which has some unboxing
optimizations) but not in CPython.

(That's not the problem here, of course. The problem is that
the user doesn't understand floating point. The issues I'm talking
about are subtle, and affect few people. Those of us who've had
of simulation systems, where cumulative error can be a problem.
In the 1990s, I had to put a lot of work into this for collision
detection algorithms for a physics engine. As two objects settle
into contact, issues with tiny differences between large numbers
start to dominate. It takes careful handling to prevent that from
causing high frequency simulated vibration in the simulation,
chewing up CPU time at best and causing simulations to fly apart
at worst. The problems are understood now, but they weren't in
the mid-1990s. The licensed Jurassic Park game "Trespasser" was a flop
for that reason.)

John Nagle

Steven D'Aprano
Guest
Posts: n/a

 02-24-2011
On Wed, 23 Feb 2011 13:26:05 -0800, John Nagle wrote:

> The IEEE 754 compliant FPU on most machines today, though, has an 80-bit
> internal representation. If you do a sequence of operations that retain
> all the intermediate results in the FPU registers, you get 16 more bits
> of precision than if you store after each operation.

That's a big if though. Which languages support such a thing? C doubles
are 64 bit, same as Python.

--
Steven

Ethan Furman
Guest
Posts: n/a

 02-24-2011
Steven D'Aprano wrote:
> On Wed, 23 Feb 2011 13:26:05 -0800, John Nagle wrote:
>
>> The IEEE 754 compliant FPU on most machines today, though, has an 80-bit
>> internal representation. If you do a sequence of operations that retain
>> all the intermediate results in the FPU registers, you get 16 more bits
>> of precision than if you store after each operation.

>
> That's a big if though. Which languages support such a thing? C doubles
> are 64 bit, same as Python.

Assembly!

~Ethan~

D'Arcy J.M. Cain
Guest
Posts: n/a

 02-24-2011
On Thu, 24 Feb 2011 04:56:46 -0800
Ethan Furman <(E-Mail Removed)> wrote:
> >> The IEEE 754 compliant FPU on most machines today, though, has an 80-bit
> >> internal representation. If you do a sequence of operations that retain
> >> all the intermediate results in the FPU registers, you get 16 more bits
> >> of precision than if you store after each operation.

> >
> > That's a big if though. Which languages support such a thing? C doubles
> > are 64 bit, same as Python.

>
> Assembly!

Really? Why would you need that level of precision just to gather all
the students into the auditorium?

--
D'Arcy J.M. Cain <(E-Mail Removed)> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

Mel
Guest
Posts: n/a

 02-24-2011
D'Arcy J.M. Cain wrote:
> On Thu, 24 Feb 2011 04:56:46 -0800
> Ethan Furman <(E-Mail Removed)> wrote:

>> > That's a big if though. Which languages support such a thing? C doubles
>> > are 64 bit, same as Python.

>>
>> Assembly!

>
> Really? Why would you need that level of precision just to gather all
> the students into the auditorium?

You would think so, but darned if some of them don't wind up in a
*different* *auditorium*!

Mel.

Robert Kern
Guest
Posts: n/a

 02-24-2011
On 2/24/11 5:55 AM, Steven D'Aprano wrote:
> On Wed, 23 Feb 2011 13:26:05 -0800, John Nagle wrote:
>
>> The IEEE 754 compliant FPU on most machines today, though, has an 80-bit
>> internal representation. If you do a sequence of operations that retain
>> all the intermediate results in the FPU registers, you get 16 more bits
>> of precision than if you store after each operation.

>
> That's a big if though. Which languages support such a thing? C doubles
> are 64 bit, same as Python.

C double *variables* are, but as John suggests, C compilers are allowed (to my
knowledge) to keep intermediate results of an expression in the larger-precision
FPU registers. The final result does get shoved back into a 64-bit double when
it is at last assigned back to a variable or passed to a function that takes a
double.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
an underlying truth."
-- Umberto Eco

Steven D'Aprano
Guest
Posts: n/a

 02-25-2011
On Thu, 24 Feb 2011 10:40:45 -0600, Robert Kern wrote:

> On 2/24/11 5:55 AM, Steven D'Aprano wrote:
>> On Wed, 23 Feb 2011 13:26:05 -0800, John Nagle wrote:
>>
>>> The IEEE 754 compliant FPU on most machines today, though, has an
>>> 80-bit internal representation. If you do a sequence of operations
>>> that retain all the intermediate results in the FPU registers, you get
>>> 16 more bits of precision than if you store after each operation.

>>
>> That's a big if though. Which languages support such a thing? C doubles
>> are 64 bit, same as Python.

>
> C double *variables* are, but as John suggests, C compilers are allowed
> (to my knowledge) to keep intermediate results of an expression in the
> larger-precision FPU registers. The final result does get shoved back
> into a 64-bit double when it is at last assigned back to a variable or
> passed to a function that takes a double.

So...

(1) you can't rely on it, because it's only "allowed" and not mandatory;

(2) you may or may not have any control over whether or not it happens;

(3) it only works for calculations that are simple enough to fit in a
single expression; and

(4) we could say the same thing about Python -- there's no prohibition on
Python using extended precision when performing intermediate results, so
it too could be said to be "allowed".

It seems rather unfair to me to single Python out as somehow lacking
(compared to which other languages?) and to gloss over the difficulties
in "If you do a sequence of operations that retain all the intermediate
results..." Yes, *if* you do so, you get more precision, but *how* do you
do so? Such a thing will be language or even implementation dependent,
and the implication that it just automatically happens without any effort
seems dubious to me.

But I could be wrong, of course. It may be that Python, alone of all
modern high-level languages, fails to take advantage of 80-bit registers
in FPUs *wink*

--
Steven