Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   Exception raising, and performance implications. (http://www.velocityreviews.com/forums/t349838-exception-raising-and-performance-implications.html)

leo 10-03-2005 09:34 PM

Exception raising, and performance implications.
 
Hello all -

I was wondering about the performance implications of explicitly
raising exceptions to get information about the current frame.
Something like what the inspect module does, with:

---
def currentframe():
"""Return the frame object for the caller's stack frame."""
try:
raise 'catch me'
except:
return sys.exc_traceback.tb_frame.f_back
---

I come from a java background, where Exceptions are a big Avoid Me, but
are the performance implications the same in Python? We're expecting a
big load on our app (100,000 users/hour) , so we'd like to be as tuned
as possible.

Thanks,
leo


Paul Rubin 10-03-2005 09:53 PM

Re: Exception raising, and performance implications.
 
"leo" <leomendoza@gmail.com> writes:
> I come from a java background, where Exceptions are a big Avoid Me, but
> are the performance implications the same in Python?


Well, you could measure it experimentally pretty easily, but anyway,
Python exceptions are much less expensive than Java exceptions.

jepler@unpythonic.net 10-03-2005 09:54 PM

Re: Exception raising, and performance implications.
 
As for performance, you'll need to benchmark it.

However, I think the functionality you're asking for is available as
inspect.currentframe(), and if the implementation is in "C" it may have a tiny
performance advantage over the Python version.

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDQah+Jd01MZaTXX0RAv0KAJ4/H3m7sSaszecPpA4hN6DX+hkN9QCfYy9F
vc/97Vn8K+A30ZNAVKCi4R0=
=OyAA
-----END PGP SIGNATURE-----


Tony Nelson 10-03-2005 10:03 PM

Re: Exception raising, and performance implications.
 
In article <1128375280.034852.119910@o13g2000cwo.googlegroups .com>,
"leo" <leomendoza@gmail.com> wrote:

> Hello all -
>
> I was wondering about the performance implications of explicitly
> raising exceptions to get information about the current frame.
> Something like what the inspect module does, with:


Python uses exceptions internally, using StopIteration to terminate the
iterator in a for: loop.

> ---
> def currentframe():
> """Return the frame object for the caller's stack frame."""
> try:
> raise 'catch me'
> except:
> return sys.exc_traceback.tb_frame.f_back
> ---


This also does a traceback; you might want to measure the cost of that.

> I come from a java background, where Exceptions are a big Avoid Me, but
> are the performance implications the same in Python? We're expecting a
> big load on our app (100,000 users/hour) , so we'd like to be as tuned
> as possible.


Switching to Python, eh? Remember to measure, measure, measure!
__________________________________________________ ______________________
TonyN.:' *firstname*nlsnews@georgea*lastname*.com
' <http://www.georgeanelson.com/>

jepler@unpythonic.net 10-04-2005 12:06 AM

Re: Exception raising, and performance implications.
 
On Mon, Oct 03, 2005 at 02:34:40PM -0700, leo wrote:
> I come from a java background, where Exceptions are a big Avoid Me, but
> are the performance implications the same in Python? We're expecting a
> big load on our app (100,000 users/hour) , so we'd like to be as tuned
> as possible.


I don't know what you do for each user, but here's a data point: My 1.8GHz
Sempron (firmly in the "budget" category) uses a fairly unoptimized piece of
software called aether[1] to serve my blog and a few other things. It uses
Apache and good old fashioned cgi-bin one-process-per-request to serve up most
pages. It parses a substantial amount of Python code each time with execfile
(not using byte-compiled files). It still does this in 87ms on average (albeit
for a simple page), or about 41k requests per hour.

By simply buying a faster CPU, or by avoiding execfile, or by using a
higher-performance technology than CGI, or with judicious use of caching, I'm
sure I could reach 100,000 requests per hour, and by using several of the
techniques together I might be able to reach 400k requests per hour. Probably
it would be after these fairly obvious, high-level optimization ideas were
exhausted that I would look at the code at a microscopic level for optimization
ideas.

But because my website is stuck on the slow end of a DSL line, there's not much
point to any of this.

Jeff
[1] http://www.logarithmic.net/pfh/aether (not my site)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDQcd0Jd01MZaTXX0RAqMEAJ9UDEYsW+v9toSGbcepC1 oH/PEeNACdHgQk
tcUZnt/nBNFW+nNWhOKG29E=
=56lv
-----END PGP SIGNATURE-----


leo 10-04-2005 02:15 PM

Re: Exception raising, and performance implications.
 
> However, I think the functionality you're asking for is available as
> inspect.currentframe(), and if the implementation is in "C" it may have a tiny
> performance advantage over the Python version.


You're absolutely right, in fact the code snippet from my OP was taken
directly from inspect.currentframe. We're intending on using this in
production, and I'm trying to gauge what the implications may be.

> Python uses exceptions internally, using StopIteration to terminate the
> iterator in a for: loop.


Wow, I was unaware of this. If Python internally uses exceptions, maybe
they aren't as detrimental as I thought.

That said, I will be judiciously profiling my code and measuring as
much as possible, I just wanted to throw this question out to the NG in
case anyone had done this before (and so I can put off learning the
profiler a little bit longer :) )

Thanks all for the replies.


Steve Holden 10-04-2005 03:25 PM

Re: Exception raising, and performance implications.
 
leo wrote:
>>However, I think the functionality you're asking for is available as
>>inspect.currentframe(), and if the implementation is in "C" it may have a tiny
>>performance advantage over the Python version.

>
>
> You're absolutely right, in fact the code snippet from my OP was taken
> directly from inspect.currentframe. We're intending on using this in
> production, and I'm trying to gauge what the implications may be.
>
>
>>Python uses exceptions internally, using StopIteration to terminate the
>>iterator in a for: loop.

>
>
> Wow, I was unaware of this. If Python internally uses exceptions, maybe
> they aren't as detrimental as I thought.
>
> That said, I will be judiciously profiling my code and measuring as
> much as possible, I just wanted to throw this question out to the NG in
> case anyone had done this before (and so I can put off learning the
> profiler a little bit longer :) )
>
> Thanks all for the replies.
>

Do note, however, that detecting an exception inside the C framework of
the interpreter carries less overhead than detecting an exception in
Python itself.

That said, exceptions are probably rather more "lightweight" than you
might imagine, so benchmarking (the profiler may not be best - have you
come across "timeit.py"?) is the best way to go.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/


Phillip J. Eby 10-04-2005 03:48 PM

Re: Exception raising, and performance implications.
 
leo wrote:
>
> You're absolutely right, in fact the code snippet from my OP was taken
> directly from inspect.currentframe. We're intending on using this in
> production, and I'm trying to gauge what the implications may be.


Use sys._getframe() instead; it doesn't raise an exception.


> Wow, I was unaware of this. If Python internally uses exceptions, maybe
> they aren't as detrimental as I thought.


Exceptions raised from C code and caught by C code use considerably
less resources, so a "for" loop catching a StopIteration raised by a
built-in iterator will have better performance than Python code
catching an exception raised in Python code. So, it's not necessarily
the case that the "for" loop scenario matches your scenario(s). Always
measure.


Tom Anderson 10-04-2005 11:02 PM

Re: Exception raising, and performance implications.
 
On Mon, 3 Oct 2005, it was written:

> "leo" <leomendoza@gmail.com> writes:
>
>> I come from a java background, where Exceptions are a big Avoid Me, but
>> are the performance implications the same in Python?

>
> Well, you could measure it experimentally pretty easily, but anyway,
> Python exceptions are much less expensive than Java exceptions.


Really? How come? What is it that stops java using the same technique as
python? There's been quite a lot of work put into making java fast, so
it'd be interesting if we had something they didn't.

tom

--
What we learn about is not nature itself, but nature exposed to our methods of questioning. -- Werner Heisenberg


All times are GMT. The time now is 12:46 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.