Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > log and figure out what bits are slow and optimize them.

Reply
Thread Tools

log and figure out what bits are slow and optimize them.

 
 
sajuptpm
Guest
Posts: n/a
 
      02-10-2012
Hi,

I want to log time taken to complete database requests inside a method/
function using decorator . is it possible ????
I think, i have to inject log code inside the method/fuctions or
modify it.
I wrote a decorator to log taken by a method/function to complete it
execution and its working well.

My requirement : log everything and figure out what bits are slow and
optimize them.

What are your suggestions ??
 
Reply With Quote
 
 
 
 
Arnaud Delobelle
Guest
Posts: n/a
 
      02-10-2012
On 10 February 2012 12:30, sajuptpm <(E-Mail Removed)> wrote:
> Hi,
>
> I want to log time taken to complete database requests inside a method/
> function using decorator . *is it possible ????
> I think, i have to inject log code inside the method/fuctions or
> modify it.
> I wrote a decorator to log taken by a method/function to complete it
> execution and its working well.
>
> My requirement : log everything and figure out what bits are slow and
> optimize them.
>
> What are your suggestions ??


Are you familiar with this?

http://docs.python.org/library/profile.html

--
Arnaud
 
Reply With Quote
 
 
 
 
sajuptpm
Guest
Posts: n/a
 
      02-10-2012
Hi,

Yes i saw profile module,
I think i have to do function call via

cProfile.run('foo()')

I know, we can debug this way.

But, i need a fixed logging system and want to use it in production.
I think, we can't permanently include profile's debugging code
in source code,
will cause any performance issue ??
 
Reply With Quote
 
Mark Lawrence
Guest
Posts: n/a
 
      02-10-2012
Please don't top post.

On 10/02/2012 12:59, Saju M wrote:
> Yes i saw profile module,
> I think, i have to do function call via
> cProfile.run('foo()')
> I know, we can debug this way.
> But, I need a fixed logging system and want to use it in production.
> I think, we can't permanently include profile's debugging code in
> source code,
> will cause any performance issue ??
>


How about http://docs.python.org/library/logging.html ?

--
Cheers.

Mark Lawrence.

 
Reply With Quote
 
Kev Dwyer
Guest
Posts: n/a
 
      02-10-2012
sajuptpm wrote:

> Hi,
>
> Yes i saw profile module,
> I think i have to do function call via
>
> cProfile.run('foo()')
>
> I know, we can debug this way.
>
> But, i need a fixed logging system and want to use it in production.
> I think, we can't permanently include profile's debugging code
> in source code,
> will cause any performance issue ??


*Any* instrumentation code is going to affect performance.

It's a trade-off that you need to analyse and manage in the context of your
application.

 
Reply With Quote
 
John Gordon
Guest
Posts: n/a
 
      02-10-2012
In <(E-Mail Removed)> Kev Dwyer <(E-Mail Removed)> writes:

> *Any* instrumentation code is going to affect performance.


Funny story about that...

I wanted to profile some code of mine, and a colleague recommended the
'hotshot' module.

It's pretty easy to use: there are functions to start profiling, stop
profiling and print results.

So I added the function calls and ran my code.... and it took a really
long time. I mean a REALLY long time. In fact I eventually had to kill
the process.

I briefly wondered if my coworker was playing a prank on me... then I
realized that I had neglected to call the function to stop profiling!

So when I went to print the results, it was still profiling... endlessly.

(Okay, maybe it wasn't that funny.)

--
John Gordon A is for Amy, who fell down the stairs
http://www.velocityreviews.com/forums/(E-Mail Removed) B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

 
Reply With Quote
 
sajuptpm
Guest
Posts: n/a
 
      02-11-2012

I decided to create a decorator like.

import cProfile
def debug_time(method):
def timed(*args, **kw):
prof = cProfile.Profile()
prof.enable(subcalls=False, builtins=False)
result = prof.runcall(method, *args, **kw)
#prof.print_stats()
msg = "\n\n\n\n##################################### ##"
msg += "\n\nURL : %s" %(tg.request.url)
msg += "\nMethod: %r" %(method.__name__)
print "--ddd--------", type(prof.getstats())
msg += "\n\nStatus : %s" %(prof.print_stats())
msg += "\n\n#######################################"
print msg
LOGGER.info(msg)
return result
return timed

Ref : http://stackoverflow.com/questions/5...ofiling-result


I want to log it in existing log file in my project,
so i tried prof.print_stats() and prof.getstats(). prof.getstats()
will need extra loop for fetch data.
prof.print_stats() will log library calls also.

Please suggest a better way to log profiler output.


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
windows xp64 bits and vista 64 bits beta2 Jordi Maycas Windows 64bit 3 06-15-2006 10:44 AM
optimize log parsing it_says_BALLS_on_your forehead Perl Misc 21 10-09-2005 10:25 PM
what about unsigned and signed 8 bits number, 16 bits, etc?? sarmin kho Python 2 06-15-2004 06:40 PM
Re: what about unsigned and signed 8 bits number, 16 bits, etc?? Miki Tebeka Python 1 06-14-2004 03:19 PM
8-Bits vs 12 or 16 bits/pixel; When does more than 8 bits count ? Al Dykes Digital Photography 3 12-29-2003 07:08 PM



Advertisments