Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > handlers.SocketHandler and exceptions

Reply
Thread Tools

handlers.SocketHandler and exceptions

 
 
writeson
Guest
Posts: n/a
 
      01-16-2008
Hi all,

On our Linux systems at work I've written a Twisted logging server
that receives log messages from multiple servers/processes to post
them to a log file, essentially serializing all the process log
messages. This works well, that is until I tried this test code:

try:
t = 10 / 0
except Exception, e:
log.exception("divide by zero")

where log is the logger instance retreived from a call to getLogger().
The problem is the handlers.SocketHandler tries to cPickle.dump() the
log record, which in this case contains an exc_info tuple, the last
item of which is a Traceback object. The pickling fails with an
"unpickleable error" and that's that.

Does anyone have any ideas how to handle this situation? I'd hate to
have to give up using the log.exception(...) call as it's useful to
get strack trace information in the log file.

Thanks in advance,
Doug Farrell
 
Reply With Quote
 
 
 
 
Mark Tolonen
Guest
Posts: n/a
 
      01-17-2008

"writeson" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Hi all,
>
> On our Linux systems at work I've written a Twisted logging server
> that receives log messages from multiple servers/processes to post
> them to a log file, essentially serializing all the process log
> messages. This works well, that is until I tried this test code:
>
> try:
> t = 10 / 0
> except Exception, e:
> log.exception("divide by zero")
>
> where log is the logger instance retreived from a call to getLogger().
> The problem is the handlers.SocketHandler tries to cPickle.dump() the
> log record, which in this case contains an exc_info tuple, the last
> item of which is a Traceback object. The pickling fails with an
> "unpickleable error" and that's that.
>
> Does anyone have any ideas how to handle this situation? I'd hate to
> have to give up using the log.exception(...) call as it's useful to
> get strack trace information in the log file.
>
> Thanks in advance,
> Doug Farrell


Check out the traceback module. It can translate the traceback into a
variety of formats (such as a string) that can be pickled.

--Mark

 
Reply With Quote
 
 
 
 
writeson
Guest
Posts: n/a
 
      01-17-2008
Mark,

>
> Check out the traceback module. It can translate the traceback into a
> variety of formats (such as a string) that can be pickled.
>
> --Mark


Thanks for the reply. I was looking at the traceback module and
thinking along the same lines you are. The problem I'm having with
that is how to modify the behavior of the SocketHandler code so it
would call the traceback module functions. The point at which the
handlers.SocketHandler code fails is in the method makePickle(), and
I'm not sure how to overload/override that method. I tried creating my
own class:

class MySocketHandler(handlers.SocketHandler):
def makePickle(self, record):
# perform new code that transforms a Traceback object into a
string

but so far I haven't figured out how to get the logging module to use
my class. In my logging configuration file I tried something like
this:

[handler_local_server]
class=mydirectory.MySocketHandler
level=DEBUG
formatter=general
args=("localhost", handlers.DEFAULT_TCP_LOGGING_PORT + 1)

but I can't seem to get the logging module to include mydirectory in
its search path for modules.

So that's where I'm stuck now.

Again, thanks for your response,
Doug
 
Reply With Quote
 
Vinay Sajip
Guest
Posts: n/a
 
      01-17-2008
On Jan 17, 1:47 pm, writeson <(E-Mail Removed)> wrote:
> Mark,
>
>
>
> > Check out the traceback module. It can translate the traceback into a
> > variety of formats (such as a string) that can be pickled.

>
> > --Mark

>
> Thanks for the reply. I was looking at the traceback module and
> thinking along the same lines you are. The problem I'm having with
> that is how to modify the behavior of the SocketHandler code so it
> would call the traceback module functions. The point at which the
> handlers.SocketHandler code fails is in the method makePickle(), and
> I'm not sure how to overload/override that method. I tried creating my
> own class:
>
> class MySocketHandler(handlers.SocketHandler):
> def makePickle(self, record):
> # perform new code that transforms a Traceback object into a
> string
>
> but so far I haven't figured out how to get theloggingmodule to use
> my class. In myloggingconfiguration file I tried something like
> this:
>
> [handler_local_server]
> class=mydirectory.MySocketHandler
> level=DEBUG
> formatter=general
> args=("localhost", handlers.DEFAULT_TCP_LOGGING_PORT + 1)
>
> but I can't seem to get theloggingmodule to include mydirectory in
> its search path for modules.
>
> So that's where I'm stuck now.
>
> Again, thanks for your response,
> Doug


What version of Python are you running? Recent versions should have a
fix for this. Here's the makePickle method of SocketHandler:

def makePickle(self, record):
"""
Pickles the record in binary format with a length prefix, and
returns it ready for transmission across the socket.
"""
ei = record.exc_info
if ei:
dummy = self.format(record) # just to get traceback text
into record.exc_text
record.exc_info = None # to avoid Unpickleable error
s = cPickle.dumps(record.__dict__, 1)
if ei:
record.exc_info = ei # for next handler
slen = struct.pack(">L", len(s))
return slen + s

Notice the code to avoid the Unpickleable error.

Best regards,

Vinay Sajip
 
Reply With Quote
 
writeson
Guest
Posts: n/a
 
      01-17-2008
Vinay,

Thanks for your reply, very interesting. We're currently running
Python2.3 (though we are getting ready to move to Python2.5), so I'm
guessing the code you're showing comes from Python2.5? I'm wondering
if I can edit the handlers.py code in my Python2.3 installation, make
the changes you show above, and have things work? Any thoughts on
this?

Thanks for the help!!
Doug
 
Reply With Quote
 
Vinay Sajip
Guest
Posts: n/a
 
      01-17-2008
On Jan 17, 5:50 pm, writeson <(E-Mail Removed)> wrote:
> Vinay,
>
> Thanks for your reply, very interesting. We're currently running
> Python2.3 (though we are getting ready to move to Python2.5), so I'm
> guessing the code you're showing comes from Python2.5? I'm wondering
> if I can edit the handlers.py code in my Python2.3 installation, make
> the changes you show above, and have things work? Any thoughts on
> this?
>
> Thanks for the help!!
> Doug


It shouldn't be a problem if you edit your copy [make a backup first,
of course ] , but make sure you also edit __init__.py which has the
part that caches the exception text in record.exc_text. Just search
for exc_text in the logging code to find all the places to change.

Best regards,


Vinay
 
Reply With Quote
 
writeson
Guest
Posts: n/a
 
      01-17-2008
On Jan 17, 2:45 pm, Vinay Sajip <(E-Mail Removed)> wrote:

Vinay,

Again, thanks for your very timely help! I was just editing the
handlers.py code, and didn't really understand how that was going to
work, and of course it didn't. I was just about to write to you again,
and voila, you'd already responded with what I needed to know. I would
have been floundering around for quite awhile before I'd have found
(if ever) the change you mentioned to __init__.py. I made the changes
and my logging server is working as I expected! Exceptions are being
placed in the log file, complete with their tracebacks.

Again, thanks very much for your help, greatly appreciated!
Doug
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Exceptions - How do you make it work like built-in exceptions? Lie Python 3 01-14-2008 06:45 PM
Exceptions + Performance on path without exceptions gratch06@gmail.com C++ 3 04-16-2007 08:52 PM
Checked exceptions vs unchecked exceptions Ahmed Moustafa Java 5 07-14-2004 01:46 PM
ImageList Errors - Exceptions and...need HELP Herr Lucifer ASP .Net 0 06-21-2004 10:21 AM
Custom exceptions -- inherit from exceptions.Exception? Paul Miller Python 3 11-12-2003 09:24 AM



Advertisments