Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Socket exceptions aren't in the standard exception hierarchy

Reply
Thread Tools

Socket exceptions aren't in the standard exception hierarchy

 
 
John Nagle
Guest
Posts: n/a
 
      04-23-2007
Here are three network-related exceptions. These
were caught by "except" with no exception type, because
none of the more specific exceptions matched. This
is what a traceback produced:


1. File "D:\Python24\lib\socket.py", line 295, in read
data = self._sock.recv(recv_size)
timeout: timed out

2. File "D:\Python24\lib\socket.py", line 295, in read
data = self._sock.recv(recv_size)
error: (10054, 'Connection reset by peer')

3. File "D:\Python24\lib\socket.py", line 317, in readline
data = recv(1)
IOError: [Errno socket error] timed out

For 1 and 2, those are errors that aren't in the
exception hierarchy. Looking at the C code for "socketmodule.c",
it's clear that "socket.error" doesn't inherit from any standard
exception class. See, in "init_socket()":

socket_error = PyErr_NewException("socket.error", NULL, NULL);

That first NULL should be some parent exception, maybe "IOError".
As it is, "socket.error" is outside the standard exception hierarchy.
That's not too good.

Case #3, IOError, should have been caught by this:

except IOError, message: # I/O error

But it wasn't. The "IOError" fell through, was caught by the
next outer exception block, and was logged as a generic
error.

I can't find where in the Python socket module an "IOError"
could be raised. I would have expected "socket.timeout".

Anyway, I need to know the full set of exceptions that can
be raised by sockets. Thanks.

John Nagle
 
Reply With Quote
 
 
 
 
Dennis Lee Bieber
Guest
Posts: n/a
 
      04-23-2007
On Sun, 22 Apr 2007 23:20:25 -0700, John Nagle <(E-Mail Removed)>
declaimed the following in comp.lang.python:

>
> 2. File "D:\Python24\lib\socket.py", line 295, in read
> data = self._sock.recv(recv_size)
> error: (10054, 'Connection reset by peer')
>

That looks like M$ Windows version of UNIX/Linux error number 54
(pretty much all Windows socket errors are UNIX number+10000)

Errors coming from Windows may not be mapped to specific Python
exceptions, but rather to some general error conditions. {hypothesis} As
such, the Windows errors may not match what UNIX/Linux report.

--
Wulfraed Dennis Lee Bieber KD6MOG
http://www.velocityreviews.com/forums/(E-Mail Removed) (E-Mail Removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (E-Mail Removed))
HTTP://www.bestiaria.com/
 
Reply With Quote
 
 
 
 
John Nagle
Guest
Posts: n/a
 
      04-23-2007
Dennis Lee Bieber wrote:
> On Sun, 22 Apr 2007 23:20:25 -0700, John Nagle <(E-Mail Removed)>
> declaimed the following in comp.lang.python:
>
>
>>2. File "D:\Python24\lib\socket.py", line 295, in read
>>data = self._sock.recv(recv_size)
>>error: (10054, 'Connection reset by peer')
>>

>
> That looks like M$ Windows version of UNIX/Linux error number 54
> (pretty much all Windows socket errors are UNIX number+10000)
>
> Errors coming from Windows may not be mapped to specific Python
> exceptions, but rather to some general error conditions. {hypothesis} As
> such, the Windows errors may not match what UNIX/Linux report.


Actually, that's not what's happening. The socket module is
explicitly raising "socket.error" in C code. It's not an OSError or
a WindowsError, although the text makes it look like one.

The problem is not that socket errors aren't entirely portable.
It's that they're not even in the Python exception hierarchy.
See "http://docs.python.org/lib/module-exceptions.html".
They have their own hierarchy, which starts at "socket.error".
All built-in exceptions were supposed to be converted to the
standard exception hierarchy back before Python 2.0, but these
weren't.

Either they should be under IOError, or there should be
"NetworkError" under EnvironmentError, and they should be under
that. "NetworkError", alongside IOError in the hierarchy,
would be useful. All the things that go wrong in networking
belong under there. Socket-level errors, SSL/TLS-level errors,
and HTTP/FTP/etc. level errors all belong under NetworkError.

This has to be fixed before PEP 352, when the exception
hierarchy is enforced, or the socket exceptions won't even work
right.

John Nagle
 
Reply With Quote
 
Steve Holden
Guest
Posts: n/a
 
      04-23-2007
John Nagle wrote:
> Dennis Lee Bieber wrote:
>> On Sun, 22 Apr 2007 23:20:25 -0700, John Nagle <(E-Mail Removed)>
>> declaimed the following in comp.lang.python:
>>
>>
>>> 2. File "D:\Python24\lib\socket.py", line 295, in read
>>> data = self._sock.recv(recv_size)
>>> error: (10054, 'Connection reset by peer')
>>>

>> That looks like M$ Windows version of UNIX/Linux error number 54
>> (pretty much all Windows socket errors are UNIX number+10000)
>>
>> Errors coming from Windows may not be mapped to specific Python
>> exceptions, but rather to some general error conditions. {hypothesis} As
>> such, the Windows errors may not match what UNIX/Linux report.

>
> Actually, that's not what's happening. The socket module is
> explicitly raising "socket.error" in C code. It's not an OSError or
> a WindowsError, although the text makes it look like one.
>
> The problem is not that socket errors aren't entirely portable.
> It's that they're not even in the Python exception hierarchy.
> See "http://docs.python.org/lib/module-exceptions.html".
> They have their own hierarchy, which starts at "socket.error".
> All built-in exceptions were supposed to be converted to the
> standard exception hierarchy back before Python 2.0, but these
> weren't.
>
> Either they should be under IOError, or there should be
> "NetworkError" under EnvironmentError, and they should be under
> that. "NetworkError", alongside IOError in the hierarchy,
> would be useful. All the things that go wrong in networking
> belong under there. Socket-level errors, SSL/TLS-level errors,
> and HTTP/FTP/etc. level errors all belong under NetworkError.
>
> This has to be fixed before PEP 352, when the exception
> hierarchy is enforced, or the socket exceptions won't even work
> right.
>

John:

Where did you get this information? If true it would certainly need to
be logged as a bug, but under Windows on 2,4 I see

>>> issubclass(socket.gaierror, Exception)

True
>>>


and the same under Cygwin 2.5. I am presuming most other users will see
the same thing.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com

 
Reply With Quote
 
John Nagle
Guest
Posts: n/a
 
      04-23-2007
Steve Holden wrote:
> John Nagle wrote:
>
>> Dennis Lee Bieber wrote:
>>
>>> On Sun, 22 Apr 2007 23:20:25 -0700, John Nagle <(E-Mail Removed)>
>>> declaimed the following in comp.lang.python:
>>>
>>>
>>>> 2. File "D:\Python24\lib\socket.py", line 295, in read
>>>> data = self._sock.recv(recv_size)
>>>> error: (10054, 'Connection reset by peer')
>>>>
>>> That looks like M$ Windows version of UNIX/Linux error number 54
>>> (pretty much all Windows socket errors are UNIX number+10000)
>>>
>>> Errors coming from Windows may not be mapped to specific Python
>>> exceptions, but rather to some general error conditions. {hypothesis} As
>>> such, the Windows errors may not match what UNIX/Linux report.

>>
>>
>> Actually, that's not what's happening. The socket module is
>> explicitly raising "socket.error" in C code. It's not an OSError or
>> a WindowsError, although the text makes it look like one.
>>
>> The problem is not that socket errors aren't entirely portable.
>> It's that they're not even in the Python exception hierarchy.
>> See "http://docs.python.org/lib/module-exceptions.html".
>> They have their own hierarchy, which starts at "socket.error".
>> All built-in exceptions were supposed to be converted to the
>> standard exception hierarchy back before Python 2.0, but these
>> weren't.
>>
>> Either they should be under IOError, or there should be
>> "NetworkError" under EnvironmentError, and they should be under
>> that. "NetworkError", alongside IOError in the hierarchy,
>> would be useful. All the things that go wrong in networking
>> belong under there. Socket-level errors, SSL/TLS-level errors,
>> and HTTP/FTP/etc. level errors all belong under NetworkError.
>>
>> This has to be fixed before PEP 352, when the exception
>> hierarchy is enforced, or the socket exceptions won't even work
>> right.
>>

> John:
>
> Where did you get this information? If true it would certainly need to
> be logged as a bug, but under Windows on 2,4 I see
>
> >>> issubclass(socket.gaierror, Exception)

> True
> >>>

>
> and the same under Cygwin 2.5. I am presuming most other users will see
> the same thing.
>
> regards
> Steve


Ah. "socket.error" is a subclass of "Exception", but not
of "StandardError".

issubclass(socket.error,StandardError)

is False.


John Nagle
 
Reply With Quote
 
Daniel Nogradi
Guest
Posts: n/a
 
      04-23-2007
> >>>> 2. File "D:\Python24\lib\socket.py", line 295, in read
> >>>> data = self._sock.recv(recv_size)
> >>>> error: (10054, 'Connection reset by peer')
> >>>>
> >>> That looks like M$ Windows version of UNIX/Linux error number 54
> >>> (pretty much all Windows socket errors are UNIX number+10000)
> >>>
> >>> Errors coming from Windows may not be mapped to specific Python
> >>> exceptions, but rather to some general error conditions. {hypothesis} As
> >>> such, the Windows errors may not match what UNIX/Linux report.
> >>
> >>
> >> Actually, that's not what's happening. The socket module is
> >> explicitly raising "socket.error" in C code. It's not an OSError or
> >> a WindowsError, although the text makes it look like one.
> >>
> >> The problem is not that socket errors aren't entirely portable.
> >> It's that they're not even in the Python exception hierarchy.
> >> See "http://docs.python.org/lib/module-exceptions.html".
> >> They have their own hierarchy, which starts at "socket.error".
> >> All built-in exceptions were supposed to be converted to the
> >> standard exception hierarchy back before Python 2.0, but these
> >> weren't.
> >>
> >> Either they should be under IOError, or there should be
> >> "NetworkError" under EnvironmentError, and they should be under
> >> that. "NetworkError", alongside IOError in the hierarchy,
> >> would be useful. All the things that go wrong in networking
> >> belong under there. Socket-level errors, SSL/TLS-level errors,
> >> and HTTP/FTP/etc. level errors all belong under NetworkError.
> >>
> >> This has to be fixed before PEP 352, when the exception
> >> hierarchy is enforced, or the socket exceptions won't even work
> >> right.
> >>

> > John:
> >
> > Where did you get this information? If true it would certainly need to
> > be logged as a bug, but under Windows on 2,4 I see
> >
> > >>> issubclass(socket.gaierror, Exception)

> > True
> > >>>

> >
> > and the same under Cygwin 2.5. I am presuming most other users will see
> > the same thing.
> >
> > regards
> > Steve

>
> Ah. "socket.error" is a subclass of "Exception", but not
> of "StandardError".
>
> issubclass(socket.error,StandardError)
>
> is False.



On linux, python 2.5:

>>> import socket
>>> issubclass(socket.error,Exception)

True
 
Reply With Quote
 
Steve Holden
Guest
Posts: n/a
 
      04-23-2007
John Nagle wrote:
[socket.error bug report]
>>
>> Where did you get this information? If true it would certainly need to
>> be logged as a bug, but under Windows on 2,4 I see
>>
>> >>> issubclass(socket.gaierror, Exception)

>> True
>> >>>

>>
>> and the same under Cygwin 2.5. I am presuming most other users will see
>> the same thing.
>>
>> regards
>> Steve

>
> Ah. "socket.error" is a subclass of "Exception", but not
> of "StandardError".
>
> issubclass(socket.error,StandardError)
>
> is False.
>

Right, so this *is* a big, as long as Brett Cannon's upcoming (further)
reorganization of the standard exception hierarchy doesn't stamp on it.
It probably *was* overlooked in the reorganization of the hierarchy, and
this implies there may be other extensions that also make the same error.

It should be logged as a bug in the tracker - the fix is probably pretty
simple, but it'll need some consideration of the forward compatibility
considerations.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com

 
Reply With Quote
 
John Nagle
Guest
Posts: n/a
 
      04-24-2007
Steve Holden wrote:
> John Nagle wrote:
> [socket.error bug report]
>
>>>
>>> Where did you get this information? If true it would certainly need
>>> to be logged as a bug, but under Windows on 2,4 I see
>>>
>>> >>> issubclass(socket.gaierror, Exception)
>>> True
>>> >>>
>>>
>>> and the same under Cygwin 2.5. I am presuming most other users will
>>> see the same thing.
>>>
>>> regards
>>> Steve

>>
>>
>> Ah. "socket.error" is a subclass of "Exception", but not
>> of "StandardError".
>>
>> issubclass(socket.error,StandardError)
>>
>> is False.
>>

> Right, so this *is* a bug, as long as Brett Cannon's upcoming (further)
> reorganization of the standard exception hierarchy doesn't stamp on it.
> It probably *was* overlooked in the reorganization of the hierarchy, and
> this implies there may be other extensions that also make the same error.
>
> It should be logged as a bug in the tracker - the fix is probably pretty
> simple, but it'll need some consideration of the forward compatibility
> considerations.
>
> regards
> Steve


The bug tracker shows quite a number of problems related to
socket exceptions. Searching for "socket AND exception" returns
45 results. These seem to be the most relevant ones:

[ 805194 ] Inappropriate error received using socket timeout
[ 1019808 ] wrong socket error returned
[ 1571878 ] Improvements to socket module exceptions
[ 708927 ] socket timeouts produce wrong errors in win32

None of them quite cover this new issue, but it's clear
that the area needs some work. Should this new issue
be added as a new bug or as an added comment to one of the
above?

Just figuring out what exceptions can be raised from
the socket module is tough. I've seen exceptions
derived from "socket.error", exceptions from IOError,
and exceptions from the SSL layer, which patches the
sockets module when loaded. These are non-bug
exceptions; that is, the problem is out in the network,
external to the program. I'm still not sure I have all
the possibilities covered.

Retrying on unknown exceptions isn't the answer;
that leads to exception loops if there's a program bug.
The most important distinction with sockets is "external
network problem" vs. "local program program". I'd like
to see a "NetworkException" in the exception hierarchy,
with all the things that can go wrong due to conditions
external to the local machine under that exception.

I'd suggest the following:

1. Add "NetworkError" under "IOError" in
the exception hierarchy.

2. Put the existing "socket.error" under "NetworkError".
Since "socket.error" needs to be reparented anyway (it's currently
a direct descendant of "Exception") this provides a good place for it.

3. Find any places where the socket module can raise IOError
or OSError due to an external network condition, and make them raise
something under NetworkError instead. Code that catches IOError
will still work.

4. Put all errors in the various SSL modules (SSLError, etc.)
which can be raised due to external network conditions under "NetworkError".

5. Move "urllib2.URLError", which is currently under IOError, to be
under NetworkError.

6. Move the misc. errors from "urllib", like "ContentTooShortError",
which are currently under IOError, down a level under NetworkError.

Then, programs that catch NetworkError could be sure of catching
all network trouble conditions, but not local code bugs.

John Nagle
 
Reply With Quote
 
Steve Holden
Guest
Posts: n/a
 
      04-24-2007
John Nagle wrote:
> Steve Holden wrote:
>> John Nagle wrote:
>> [socket.error bug report]
>>
>>>> Where did you get this information? If true it would certainly need
>>>> to be logged as a bug, but under Windows on 2,4 I see
>>>>
>>>> >>> issubclass(socket.gaierror, Exception)
>>>> True
>>>> >>>
>>>>
>>>> and the same under Cygwin 2.5. I am presuming most other users will
>>>> see the same thing.
>>>>
>>>> regards
>>>> Steve
>>>
>>> Ah. "socket.error" is a subclass of "Exception", but not
>>> of "StandardError".
>>>
>>> issubclass(socket.error,StandardError)
>>>
>>> is False.
>>>

>> Right, so this *is* a bug, as long as Brett Cannon's upcoming (further)
>> reorganization of the standard exception hierarchy doesn't stamp on it.
>> It probably *was* overlooked in the reorganization of the hierarchy, and
>> this implies there may be other extensions that also make the same error.
>>
>> It should be logged as a bug in the tracker - the fix is probably pretty
>> simple, but it'll need some consideration of the forward compatibility
>> considerations.
>>
>> regards
>> Steve

>
> The bug tracker shows quite a number of problems related to
> socket exceptions. Searching for "socket AND exception" returns
> 45 results. These seem to be the most relevant ones:
>
> [ 805194 ] Inappropriate error received using socket timeout
> [ 1019808 ] wrong socket error returned
> [ 1571878 ] Improvements to socket module exceptions
> [ 708927 ] socket timeouts produce wrong errors in win32
>
> None of them quite cover this new issue, but it's clear
> that the area needs some work. Should this new issue
> be added as a new bug or as an added comment to one of the
> above?
>
> Just figuring out what exceptions can be raised from
> the socket module is tough. I've seen exceptions
> derived from "socket.error", exceptions from IOError,
> and exceptions from the SSL layer, which patches the
> sockets module when loaded. These are non-bug
> exceptions; that is, the problem is out in the network,
> external to the program. I'm still not sure I have all
> the possibilities covered.
>
> Retrying on unknown exceptions isn't the answer;
> that leads to exception loops if there's a program bug.
> The most important distinction with sockets is "external
> network problem" vs. "local program program". I'd like
> to see a "NetworkException" in the exception hierarchy,
> with all the things that can go wrong due to conditions
> external to the local machine under that exception.
>
> I'd suggest the following:
>
> 1. Add "NetworkError" under "IOError" in
> the exception hierarchy.
>
> 2. Put the existing "socket.error" under "NetworkError".
> Since "socket.error" needs to be reparented anyway (it's currently
> a direct descendant of "Exception") this provides a good place for it.
>
> 3. Find any places where the socket module can raise IOError
> or OSError due to an external network condition, and make them raise
> something under NetworkError instead. Code that catches IOError
> will still work.
>
> 4. Put all errors in the various SSL modules (SSLError, etc.)
> which can be raised due to external network conditions under "NetworkError".
>
> 5. Move "urllib2.URLError", which is currently under IOError, to be
> under NetworkError.
>
> 6. Move the misc. errors from "urllib", like "ContentTooShortError",
> which are currently under IOError, down a level under NetworkError.
>
> Then, programs that catch NetworkError could be sure of catching
> all network trouble conditions, but not local code bugs.
>
> John Nagle


All these notes should be included in the bug report, as I suspect the
module would benefit from additional clarity. I believe socket just grew
up as a clever way of making as much use of the platform socket library
as possible, and that the approach's deficiencies have become more
obvious as time has gone by and Python has matured.

The work you describe is not insignificant, so it should be fully
described if it's to stand any chance of being implemented. I *do* think
it would rationalize things considerably, but that's only me ...

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com

 
Reply With Quote
 
John Nagle
Guest
Posts: n/a
 
      04-24-2007
Steve Holden wrote:
> John Nagle wrote:
>
>> Steve Holden wrote:
>>
>>> John Nagle wrote:
>>> [socket.error bug report]


> All these notes should be included in the bug report, as I suspect the
> module would benefit from additional clarity.


Done. See

[ 1706815 ] socket.error exceptions not subclass of StandardError

Also see

[ 805194 ] Inappropriate error received using socket timeout
[ 1019808 ] wrong socket error returned
[ 1571878 ] Improvements to socket module exceptions
[ 708927 ] socket timeouts produce wrong errors in win32

for related but not identical problems in that area.

John Nagle
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: socket.unbind or socket.unlisten? - socket.error: (48, 'Addressalready in use') Laszlo Nagy Python 0 02-01-2009 07:37 AM
socket.unbind or socket.unlisten? - socket.error: (48, 'Addressalready in use') Laszlo Nagy Python 1 01-27-2009 05:05 PM
Re: socket.unbind or socket.unlisten? - socket.error: (48,'Address already in use') Jean-Paul Calderone Python 0 01-27-2009 01:41 PM
Best practise hierarchy for user-defined exceptions Slaunger Python 2 11-17-2008 01:23 PM
Custom exceptions -- inherit from exceptions.Exception? Paul Miller Python 3 11-12-2003 09:24 AM



Advertisments