Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > NaN handling

Reply
Thread Tools

NaN handling

 
 
Andy McDonagh
Guest
Posts: n/a
 
      05-03-2006
Dear python experts,

I am new to python and this site, so I apologize if this is off topic (i.e. is it a SciPy question?). I will try to demonstrate my problem below:
--------------------------------------------------------
#!/usr/local/bin/python

from scipy import *
from scipy.stats import *
a=norm(loc=0,scale=1)
a_data = a.rvs(10)

problem = zeros(10)
print problem

h_x1_x2 = -sum(problem * log2(a_data))

print h_x1_x2
#NaN
----------------------------------------------------------

I need a way of handling NaNs for example R has the 'na.omit' option. Does anybody know if this exists?

Many thanks

Andy




 
Reply With Quote
 
 
 
 
Grant Edwards
Guest
Posts: n/a
 
      05-03-2006
On 2006-05-03, Andy McDonagh <(E-Mail Removed)> wrote:
> Dear python experts,
>
> I am new to python and this site, so I apologize if this is off topic (i.e. is it a SciPy question?). I will try to demonstrate my problem below:
> --------------------------------------------------------
> #!/usr/local/bin/python
>
> from scipy import *
> from scipy.stats import *
> a=norm(loc=0,scale=1)
> a_data = a.rvs(10)
>
> problem = zeros(10)
> print problem
>
> h_x1_x2 = -sum(problem * log2(a_data))
>
> print h_x1_x2
> #NaN
> ----------------------------------------------------------
>
> I need a way of handling NaNs


NaNs are handled.

Apparently they aren't handled the way you want them to be?

> for example R has the 'na.omit' option. Does anybody know if
> this exists?


It would help if you explain how you want NaNs handled, but I
don't recall that tehre are any "options" for handling NaNs
other than the default way in scipy.

--
Grant Edwards grante Yow! It's NO USE... I've
at gone to "CLUB MED"!!
visi.com
 
Reply With Quote
 
 
 
 
Ivan Vinogradov
Guest
Posts: n/a
 
      05-05-2006

> <snip>
> NaNs are handled.


Throwing an exception would be nice in regular Python (non-scipy).

This works to catch NaN on OSX and Linux:

# assuming x is a number
if x+1==x or x!=x:
#x is NaN

But is expensive as a precautionary measure.
Assert can be used for testing, if production code can be run with -0
or -OO.
 
Reply With Quote
 
Grant Edwards
Guest
Posts: n/a
 
      05-05-2006
On 2006-05-05, Ivan Vinogradov <(E-Mail Removed)> wrote:
>
>> <snip>
>> NaNs are handled.

>
> Throwing an exception would be nice in regular Python (non-scipy).


That would break most of my Python programs (at least most of
the ones in which I do floating point). My main problem with
NaNs (and Infs) is that there isn't a string represention that
is consistent across platforms.

> This works to catch NaN on OSX and Linux:
>
> # assuming x is a number
> if x+1==x or x!=x:
> #x is NaN
>
> But is expensive as a precautionary measure. Assert can be
> used for testing, if production code can be run with -0 or
> -OO.


There are those of us that need NaNs in production code, so it
would have to be something that could be configured. I find
that in my programs the places where I need to do something
"exceptional" with a NaN are very limited. The vast majority
of the time, I need them to propagate quietly.

--
Grant Edwards grante Yow! Thousands of days of
at civilians... have produced
visi.com a... feeling for the
aesthetic modules --
 
Reply With Quote
 
Ivan Vinogradov
Guest
Posts: n/a
 
      05-05-2006
> <snip>
> There are those of us that need NaNs in production code, so it
> would have to be something that could be configured. I find
> that in my programs the places where I need to do something
> "exceptional" with a NaN are very limited. The vast majority
> of the time, I need them to propagate quietly.


Our programming expectations may differ, but an option to catch NaNs as
an exception is a great idea.

Regards, Ivan.
 
Reply With Quote
 
Robert Kern
Guest
Posts: n/a
 
      05-05-2006
Ivan Vinogradov wrote:
>><snip>
>>There are those of us that need NaNs in production code, so it
>>would have to be something that could be configured. I find
>>that in my programs the places where I need to do something
>>"exceptional" with a NaN are very limited. The vast majority
>>of the time, I need them to propagate quietly.

>
> Our programming expectations may differ, but an option to catch NaNs as
> an exception is a great idea.


numpy lets the programmer control how NaNs are handled in numpy code. Producing
a NaN can be ignored, create a warning, raise an exception or call a function.
It's not well documented at the moment, but the functions are seterr(),
seterrcall(), seterrobj(), geterr(), geterrcall(), and geterrobj().

Pure Python has a similar, but somewhat less flexible method, on UNIX platforms.

http://docs.python.org/dev/lib/module-fpectl.html

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 
Reply With Quote
 
Grant Edwards
Guest
Posts: n/a
 
      05-05-2006
On 2006-05-05, Ivan Vinogradov <(E-Mail Removed)> wrote:
>> <snip>
>> There are those of us that need NaNs in production code, so it
>> would have to be something that could be configured. I find
>> that in my programs the places where I need to do something
>> "exceptional" with a NaN are very limited. The vast majority
>> of the time, I need them to propagate quietly.

>
> Our programming expectations may differ, but an option to
> catch NaNs as an exception is a great idea.


Unless it's done in hardware, it would be hopelessly slow.
There some platforms where it's possible for an application to
enable and handle FP interrupts (all of the exampls for that
seem to be in Fortran). I don't know if the common platforms
(IA32/MS-Windows, IA32/Linux) even have that ability.

--
Grant Edwards grante Yow! I'm into SOFTWARE!
at
visi.com
 
Reply With Quote
 
Grant Edwards
Guest
Posts: n/a
 
      05-05-2006
On 2006-05-05, Robert Kern <(E-Mail Removed)> wrote:

>> Our programming expectations may differ, but an option to catch NaNs as
>> an exception is a great idea.

>

[...]

> Pure Python has a similar, but somewhat less flexible method, on UNIX platforms.
>
> http://docs.python.org/dev/lib/module-fpectl.html


For which "Unix" platforms? It's not there under Linux:

Python 2.4.2 (#1, Feb 14 2006, 07:55:13)
[GCC 3.4.4 (Gentoo 3.4.4-r1, ssp-3.4.4-1.0, pie-8.7.] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import fpectl

Traceback (most recent call last):
File "<stdin>", line 1, in ?
ImportError: No module named fpectl
>>>


--
Grant Edwards grante Yow! Do I hear th'
at SPINNING of various
visi.com WHIRRING, ROUND, and WARM
WHIRLOMATICS?!
 
Reply With Quote
 
Robert Kern
Guest
Posts: n/a
 
      05-05-2006
Grant Edwards wrote:
> On 2006-05-05, Robert Kern <(E-Mail Removed)> wrote:


>>Pure Python has a similar, but somewhat less flexible method, on UNIX platforms.
>>
>> http://docs.python.org/dev/lib/module-fpectl.html

>
> For which "Unix" platforms? It's not there under Linux:
>
> Python 2.4.2 (#1, Feb 14 2006, 07:55:13)
> [GCC 3.4.4 (Gentoo 3.4.4-r1, ssp-3.4.4-1.0, pie-8.7.] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>
>>>>import fpectl

>
> Traceback (most recent call last):
> File "<stdin>", line 1, in ?
> ImportError: No module named fpectl


You might have to enable it during the Python build process.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 
Reply With Quote
 
Ivan Vinogradov
Guest
Posts: n/a
 
      05-05-2006

On 5-May-06, at 6:45 PM, Grant Edwards wrote:

> On 2006-05-05, Robert Kern <(E-Mail Removed)> wrote:
>
>>> Our programming expectations may differ, but an option to catch
>>> NaNs as
>>> an exception is a great idea.

>>

> [...]
>
>> Pure Python has a similar, but somewhat less flexible method, on
>> UNIX platforms.
>>
>> http://docs.python.org/dev/lib/module-fpectl.html

>
> For which "Unix" platforms? It's not there under Linux:

<snip>

It doesn't seem to be here under OSX either (universal Python install).

Since numpy seems to be working on a variety of platforms/hardware,
how hard would it be to extract this functionality from it to add to
Python proper?

Cheers, Ivan.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
converting int to float gives nan (int value is definitely non-nan) googlegroups11@mailinator.com C Programming 6 03-19-2009 03:59 PM
numpy: handling float('NaN') different in XP vs. Linux John [H2O] Python 6 06-16-2008 07:57 PM
Exception Handling/Divide By Zero/NaN Done on One Line John Kopanas Ruby 5 01-24-2007 10:34 PM
max(NaN,0) should be NaN norb1@yahoo.com C Programming 61 09-08-2006 05:41 PM
Point2D.Distance method returns NaN, how is this possible? DiscoStu Java 9 08-05-2003 04:42 AM



Advertisments