Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > python memory analysis

Reply
Thread Tools

python memory analysis

 
 
Alexander Hoffmann
Guest
Posts: n/a
 
      09-06-2004
Hello,

I wrote a deamon like program in python and after a while I realized that it
consumes more and more memory over time. So my question is: How to debug
memory leaks in python ?
Of course I found the built-in profiler but it only helps to make
performance analysis. Is there a way (or a tools) to view all the instances
that are kept im memory by the interpreter ?


Thanks a lot,
Alex
 
Reply With Quote
 
 
 
 
Alex Martelli
Guest
Posts: n/a
 
      09-06-2004
Alexander Hoffmann <(E-Mail Removed)> wrote:

> Hello,
>
> I wrote a deamon like program in python and after a while I realized that it
> consumes more and more memory over time. So my question is: How to debug
> memory leaks in python ?
> Of course I found the built-in profiler but it only helps to make
> performance analysis. Is there a way (or a tools) to view all the instances
> that are kept im memory by the interpreter ?


It's actually quite a problem, sigh. gc.get_objects() gives you a list
of all objects tracked by the garbage collector (pretty close to, "all
the objects" period) but that's just the beginning, now you have to find
out what each object IS, why is it around (functions get_referents and
get_referrers can help), and hardest of all how much memory that
particular object IS actually consuming.

How to estimate the latter is a headache. E.g. consider a list of
integers. A list with a 1,000,000 items which are all worth 1438397
presumably has a single copy of that particular int shared by all slots
(but that's not certain... you'd need to check for that) so you're
paying for a million slots plus one int (and there is no clear doc as to
how much that means in bytes on a given platform, btw). If the items
are all different then you're paying for a million slots plus a million
ints (unless some of those ints are actually also used and still needed
elsewhere).

Still I really wish I had a call gc.memory_howmuch(objs) returning a
tuple of two ints X <= Y, telling me that the memory taken up by all the
objs together, the memory I could get back for other purposes if all
references to all of the objs went away, is somewhere between X and Y
bytes -- i.e. if all of the objs went away I'd save at least X bytes but
surely no more than Y. How small one could make Y-X, how tight the
bounds could be, may be harder to gauge, but it may depend on how much
one wants to invest in digging deep down...


Alex
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Evaluating static analysis and Dynamic analysis tools for C/C++ ssubbarayan C Programming 5 11-03-2009 12:50 AM
? ? crash dump analysis / memory check ? ? User Name Computer Support 6 01-11-2008 09:12 PM
Memory Profiler for Heap Analysis Moritz Wissenbach Java 14 06-20-2007 10:37 AM
Re: python memory analysis Alexander Hoffmann Python 3 09-06-2004 06:04 PM
Understanding memory leaks in Java & start using HAT for further analysis qazmlp Java 3 01-07-2004 09:30 PM



Advertisments