Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: Any way to turn off exception handling? (debugging)

Reply
Thread Tools

Re: Any way to turn off exception handling? (debugging)

 
 
Paul Rubin
Guest
Posts: n/a
 
      02-11-2010
mk <(E-Mail Removed)> writes:
>> Um... run your code in a debugger.

>
> ..except the code in question is multithreaded and pdb is no good for
> that, and last time I checked, yappi was broken.


Try winpdb.org.
 
Reply With Quote
 
 
 
 
mk
Guest
Posts: n/a
 
      02-11-2010

> the uncommon, the exceptional, case. If one could somehow turn off
> exceptions, you can't even do a for loop: every for loop would become
> infinite-- exceptions are how Python signals the end of an iterator.
> Think about that, *every* for loop in Python suddenly breaks.


Ouch.

> Sure I can, but how do I get out of Python the info *what called
> particular class/instance*? That is, how do I know who's the caller?
>
>
> Um... run your code in a debugger.


...except the code in question is multithreaded and pdb is no good for
that, and last time I checked, yappi was broken.

>
> That's what debugger's are -for-.
>
> "pdb" is a simple one. Various IDE's have more advanced ones.


I tried using pdb on paramiko and my own threaded code, lost some hair
in the process, and gave up. It's good for tracing the main thread,
basically, not for threading.Threads.

> I think you're better served showing us what, "I'm getting an exception
> (on socket) handled..." actually means, what is actually happening with
> real code and output, and getting pointers on where to look, then trying
> to gather material to support an honestly impossible change


I'm using a threaded library paramiko (probably by now half of this
group is sick of hearing me saying this again). 2 of (rather broken)
hosts (Solaris, Linux) are really, really slow when responding over SSH,
like a minute for a key exchange. In paramiko, timeout doesn't work bc
the timeout specified there is timeout for a socket and not for SSH.

When calling transport.socket.close() via threading.Timer like this:

def printandclose(self):
print "\ntrying to close transport.sock"
self.trans.sock.close()
self.flag = True


self.flag = False
timer = threading.Timer(4, self.printandclose)
timer.start()
self.trans.start_client()
timer.cancel()

.... I'm getting this on the console:

ERROR (9, 'Bad file descriptor')

According to a guy who is very knowledgable on the library (thanks for
all the help, James) this is result of catching socket.error somewhere
in paramiko code, like "except socket.error".

I need to find that place. On top of lack of proper handling of
timeouts, this is important bc when calling paramiko SSHClient.connect()
(which in turn uses the same machinery as transport.start_client IIRC)
paramiko uses close to 100% of CPU when negotiating with such broken or
unresponsive hosts.

For my reasons, I need to get this fixed.

Regards,
mk


 
Reply With Quote
 
 
 
 
Peter Otten
Guest
Posts: n/a
 
      02-11-2010
mk wrote:

>
>> the uncommon, the exceptional, case. If one could somehow turn off
>> exceptions, you can't even do a for loop: every for loop would become
>> infinite-- exceptions are how Python signals the end of an iterator.
>> Think about that, *every* for loop in Python suddenly breaks.

>
> Ouch.
>
>> Sure I can, but how do I get out of Python the info *what called
>> particular class/instance*? That is, how do I know who's the caller?
>>
>>
>> Um... run your code in a debugger.

>
> ..except the code in question is multithreaded and pdb is no good for
> that, and last time I checked, yappi was broken.
>
>>
>> That's what debugger's are -for-.
>>
>> "pdb" is a simple one. Various IDE's have more advanced ones.

>
> I tried using pdb on paramiko and my own threaded code, lost some hair
> in the process, and gave up. It's good for tracing the main thread,
> basically, not for threading.Threads.
>
>> I think you're better served showing us what, "I'm getting an exception
>> (on socket) handled..." actually means, what is actually happening with
>> real code and output, and getting pointers on where to look, then trying
>> to gather material to support an honestly impossible change

>
> I'm using a threaded library paramiko (probably by now half of this
> group is sick of hearing me saying this again). 2 of (rather broken)
> hosts (Solaris, Linux) are really, really slow when responding over SSH,
> like a minute for a key exchange. In paramiko, timeout doesn't work bc
> the timeout specified there is timeout for a socket and not for SSH.
>
> When calling transport.socket.close() via threading.Timer like this:
>
> def printandclose(self):
> print "\ntrying to close transport.sock"
> self.trans.sock.close()
> self.flag = True
>
>
> self.flag = False
> timer = threading.Timer(4, self.printandclose)
> timer.start()
> self.trans.start_client()
> timer.cancel()
>
> ... I'm getting this on the console:
>
> ERROR (9, 'Bad file descriptor')
>
> According to a guy who is very knowledgable on the library (thanks for
> all the help, James) this is result of catching socket.error somewhere
> in paramiko code, like "except socket.error".
>
> I need to find that place. On top of lack of proper handling of
> timeouts, this is important bc when calling paramiko SSHClient.connect()
> (which in turn uses the same machinery as transport.start_client IIRC)
> paramiko uses close to 100% of CPU when negotiating with such broken or
> unresponsive hosts.
>
> For my reasons, I need to get this fixed.


You could try to shadow the exception class with None:

>>> ZeroDivisionError = None
>>> try:

.... 1/0
.... except ZeroDivisionError:
.... print "caught"
....
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ZeroDivisionError: integer division or modulo by zero

Applying that technique on a module containing

try:
...
except socket.error:
...

#untested
import socket

class SocketWrapper:
def __getattr__(self, name):
return getattr(socket, name)
error = None

import module_using_socket
module_using_socket.socket = SocketWrapper()

Peter
 
Reply With Quote
 
mk
Guest
Posts: n/a
 
      02-11-2010
Paul Rubin wrote:
> mk <(E-Mail Removed)> writes:
>>> Um... run your code in a debugger.

>> ..except the code in question is multithreaded and pdb is no good for
>> that, and last time I checked, yappi was broken.

>
> Try winpdb.org.


This is a treasure! In minutes I've had this attached to remote process
and debugging threads.

Muchos gracias Paul!

Regards,
mk


 
Reply With Quote
 
mk
Guest
Posts: n/a
 
      02-11-2010
Peter Otten wrote:

> try:
> ...
> except socket.error:
> ...
>
> #untested
> import socket
>
> class SocketWrapper:
> def __getattr__(self, name):
> return getattr(socket, name)
> error = None
>
> import module_using_socket
> module_using_socket.socket = SocketWrapper()


Very interesting solution. Thanks!

Regards,
mk




 
Reply With Quote
 
Vinay Sajip
Guest
Posts: n/a
 
      02-12-2010
On Feb 11, 7:12*pm, mk <(E-Mail Removed)> wrote:
> Peter Otten wrote:
> > try:
> > * *...
> > except socket.error:
> > * *...

>
> > #untested
> > import socket

>
> > class SocketWrapper:
> > * * def __getattr__(self, name):
> > * * * * return getattr(socket, name)
> > * * error = None

>
> > import module_using_socket
> > module_using_socket.socket = SocketWrapper()

>
> Very interesting solution. Thanks!
>
> Regards,
> mk


On Feb 11, 2:12 pm, mk <(E-Mail Removed)> wrote:
> Peter Otten wrote:
> > try:
> > ...
> > except socket.error:
> > ...

>
> > #untested
> > import socket

>
> > class SocketWrapper:
> > def __getattr__(self, name):
> > return getattr(socket, name)
> > error = None

>
> > import module_using_socket
> > module_using_socket.socket = SocketWrapper()

>
> Very interesting solution. Thanks!
>
> Regards,
> mk


You could refine Peter's suggestion further. For example, if you don't
want to change the flow of execution (as Peter's suggestion does) but
just log the occurrences, then you could try something like this:

# socktest.py

import socket

def test():
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
s.connect(('localhost', 9999))
except socket.error, e:
print '-- handler printing:', e

# sockwrap.py

import socket
import traceback
import logging

class SocketWrapper:
def __getattr__(self, name):
return getattr(socket, name)

@property
def error(self):
# In practice you could log to a specific logger
logging.exception('A socket error occurred')
return getattr(socket, 'error')

def wrap(mod):
mod.socket = SocketWrapper()

# Now try it out

>>> import socktest, sockwrap
>>> sockwrap.wrap(socktest)
>>> socktest.test()

ERROR:root:A socket error occurred
Traceback (most recent call last):
File "socktest.py", line 7, in test
s.connect(('localhost', 9999))
File "<string>", line 1, in connect
error: (10061, 'Connection refused')
-- handler printing: (10061, 'Connection refused')
>>>


Regards,

Vinay Sajip
 
Reply With Quote
 
Peter Otten
Guest
Posts: n/a
 
      02-12-2010
Duncan Booth wrote:

> Peter Otten <(E-Mail Removed)> wrote:
>
>> You could try to shadow the exception class with None:


> This works in Python 2.x but will break in Python 3. None is not a valid
> exception specification and Python 3 will check for that and complain.


> A better solution is to use an empty tuple as that is a valid exception
> specification so will work in both Python 2.x and 3.x:


Out of curiosity I tried a custom __subclasscheck__, but didn't get it to
work in 3.1:

$ cat instance_check.py
class ZeroDivisionError(BaseException):
class __metaclass__(type):
def __subclasscheck__(self, other):
print "performing subclass check --> %s" % catch
return catch

if __name__ == "__main__":
for catch in [True, False]:
print "catch = %s" % catch
try:
1/0
except ZeroDivisionError:
print "caught"

$ python2.6 instance_check.py
catch = True
performing subclass check --> True
caught
catch = False
performing subclass check --> False
Traceback (most recent call last):
File "instance_check.py", line 11, in <module>
1/0
ZeroDivisionError: integer division or modulo by zero
$

Peter
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Turn off exception Immortal Nephi C++ 2 07-30-2010 10:46 AM
Any way to turn off exception handling? (debugging) mk Python 0 02-11-2010 04:17 PM
Viewstate wont turn off even after setting it to off ! robert112 ASP .Net 1 04-26-2007 01:51 AM
Is there any way to turn off these STUPID!!!! event sounds in 1.0.4... peterf Firefox 12 07-08-2005 11:28 AM
Explorer page title, any way to turn off? Miles Computer Support 2 12-04-2003 07:41 PM



Advertisments