Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   How to determine if printing is being a bottleneck in my code? (http://www.velocityreviews.com/forums/t955129-how-to-determine-if-printing-is-being-a-bottleneck-in-my-code.html)

 SherjilOzair 12-05-2012 02:25 AM

How to determine if printing is being a bottleneck in my code?

Hello list,

When it comes to printing things while some computation is being done, there are 2 extremes.

1. printing speed is slower than data-to-print generation speed. In this case, printing is a bottleneck. Examples: "for i in xrange(2**30): print i". Without the print, this code would be much faster.

2. data-to-print generation speed is slower than printing speed. So, this case, printing does now slow you down much. Example: for m in matrices: print m.inverse() # inverse is a time-taking function

These two cases are pretty easy. But, my question is, how to draw the line? How do I know that print is slowing me down, and I should probably remove some of them? Is there a scientific way to do this, rather than just intuition and guesswork?

I can clarify, if needed.
Thanks,
Sherjil Ozair

 Roy Smith 12-05-2012 02:36 AM

Re: How to determine if printing is being a bottleneck in my code?

SherjilOzair <sherjilozair@gmail.com> wrote:

> Hello list,
>
> When it comes to printing things while some computation is being done, there
> are 2 extremes.
>
> 1. printing speed is slower than data-to-print generation speed. In this
> case, printing is a bottleneck. Examples: "for i in xrange(2**30): print i".
> Without the print, this code would be much faster.
>
> 2. data-to-print generation speed is slower than printing speed. So, this
> case, printing does now slow you down much. Example: for m in matrices: print
> m.inverse() # inverse is a time-taking function
>
> These two cases are pretty easy. But, my question is, how to draw the line?
> How do I know that print is slowing me down, and I should probably remove
> some of them? Is there a scientific way to do this, rather than just
> intuition and guesswork?
>
> I can clarify, if needed.
> Thanks,
> Sherjil Ozair

The profiler (http://docs.python.org/2/library/profile.html) is your
friend.

 rusi 12-05-2012 02:49 AM

Re: How to determine if printing is being a bottleneck in my code?

On Dec 5, 7:36*am, Roy Smith <r...@panix.com> wrote:
>
>
>
>
>
>
>
>
>
> *SherjilOzair <sherjiloz...@gmail.com> wrote:
> > Hello list,

>
> > When it comes to printing things while some computation is being done, there
> > are 2 extremes.

>
> > 1. printing speed is slower than data-to-print generation speed. In this
> > case, printing is a bottleneck. Examples: "for i in xrange(2**30): print i".
> > Without the print, this code would be much faster.

>
> > 2. data-to-print generation speed is slower than printing speed. So, this
> > case, printing does now slow you down much. Example: for m in matrices:print
> > m.inverse() # inverse is a time-taking function

>
> > These two cases are pretty easy. But, my question is, how to draw the line?
> > How do I know that print is slowing me down, and I should probably remove
> > some of them? Is there a scientific way to do this, rather than just
> > intuition and guesswork?

>
> > I can clarify, if needed.
> > Thanks,
> > Sherjil Ozair

>
> The profiler (http://docs.python.org/2/library/profile.html) is your
> friend.

One added caveat: When 'printing' is the bottleneck it can be
- in the actual disk/console/network I/O
- in the str/repr builtin/customized that precedes it

[Or both of course]

So after you identify printing as the bottleneck, it may be worthwhile
to print to a dummy device.
On Unix one usually uses /dev/null for this but whether that will help
clarify or muddy the picture I am not sure (Whats the overhead to
writing to null?)
A more internal-to-python method may be to replace the print with a
str/repr assigned to say a global variable

 All times are GMT. The time now is 03:43 AM.