Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > memory measurements

Reply
Thread Tools

memory measurements

 
 
Eric Mahurin
Guest
Posts: n/a
 
      09-23-2005
In ruby, is there a portable way to measure memory usage? Any
of these would be useful:

- peak
- average
- current
- any of the above for just the memory that GC maintains

It would be nice if Benchmark did this if there was a portable
way. Many times there is a tradeoff between runtime and memory
usage and it would be good to see this easily.



=09
__________________________________=20
Yahoo! Mail - PC Magazine Editors' Choice 2005=20
http://mail.yahoo.com


 
Reply With Quote
 
 
 
 
Stephen Kellett
Guest
Posts: n/a
 
      09-23-2005
In message <(E-Mail Removed) o.com>,
Eric Mahurin <(E-Mail Removed)> writes
>In ruby, is there a portable way to measure memory usage? Any
>of these would be useful:


Not portable, but if you are using Windows you may want to take a look
at Ruby Memory Validator. No website description at the present time but
you can apply for the beta at http://www.softwareverify.com

Stephen
--
Stephen Kellett
Object Media Limited http://www.objmedia.demon.co.uk/software.html
Computer Consultancy, Software Development
Windows C++, Java, Assembler, Performance Analysis, Troubleshooting
 
Reply With Quote
 
 
 
 
Eric Mahurin
Guest
Posts: n/a
 
      09-23-2005
I'm a linux guy. Here's what I do to get the memory of the
current ruby process:

IO.readlines("/proc/#{Process.pid}/status").grep(/VmSize/).display

Of course this is non-portable too.

I'd imagine to get a non-portable solution, you'd need
something in GC/ObjectSpace to figure out how much its objects
are taking up.

--- Stephen Kellett <(E-Mail Removed)> wrote:

> In message
> <(E-Mail Removed) o.com>,=20
> Eric Mahurin <(E-Mail Removed)> writes
> >In ruby, is there a portable way to measure memory usage?=20

> Any
> >of these would be useful:

>=20
> Not portable, but if you are using Windows you may want to
> take a look=20
> at Ruby Memory Validator. No website description at the
> present time but=20
> you can apply for the beta at http://www.softwareverify.com
>=20
> Stephen
> --=20
> Stephen Kellett
> Object Media Limited =20
> http://www.objmedia.demon.co.uk/software.html
> Computer Consultancy, Software Development
> Windows C++, Java, Assembler, Performance Analysis,
> Troubleshooting
>=20
>=20



__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around=20
http://mail.yahoo.com=20


 
Reply With Quote
 
Stephen Kellett
Guest
Posts: n/a
 
      09-23-2005
In message <(E-Mail Removed) o.com>,
Eric Mahurin <(E-Mail Removed)> writes
>I'd imagine to get a non-portable solution, you'd need
>something in GC/ObjectSpace to figure out how much its objects
>are taking up.


That is what Ruby Memory Validator does. Plus provide lots of view and
metrics to examine the reference graphs and so on.

Stephen
--
Stephen Kellett
Object Media Limited http://www.objmedia.demon.co.uk/software.html
Computer Consultancy, Software Development
Windows C++, Java, Assembler, Performance Analysis, Troubleshooting
 
Reply With Quote
 
Eric Mahurin
Guest
Posts: n/a
 
      09-23-2005
--- Stephen Kellett <(E-Mail Removed)> wrote:

> In message
> <(E-Mail Removed) o.com>,=20
> Eric Mahurin <(E-Mail Removed)> writes
> >I'd imagine to get a non-portable solution, you'd need
> >something in GC/ObjectSpace to figure out how much its

> objects
> >are taking up.

>=20
> That is what Ruby Memory Validator does. Plus provide lots of
> view and=20
> metrics to examine the reference graphs and so on.


I don't care about graphs and such. Just a method (or several)
in GC/ObjectSpace to get some memory info. If the above did
what I'm thinking about (walk through each
object/stack/symbol-table/etc and add up the memory used), it
would be portable, but you said it is windows only. Also, I
couldn't find any download link.

What would be ideal would be if the C call get_rusage worked on
linux (it returns 0 for various memory fields) and the
equivalent was available on various platforms. Of course you
could also make a method that did whatever was appropriate for
each platform too.




=09
__________________________________=20
Yahoo! Mail - PC Magazine Editors' Choice 2005=20
http://mail.yahoo.com


 
Reply With Quote
 
Chris McGrath
Guest
Posts: n/a
 
      09-23-2005
On 9/23/05, Eric Mahurin <(E-Mail Removed)> wrote:
> --- Stephen Kellett <(E-Mail Removed)> wrote:

...snipped...
> I don't care about graphs and such. Just a method (or several)
> in GC/ObjectSpace to get some memory info. If the above did
> what I'm thinking about (walk through each
> object/stack/symbol-table/etc and add up the memory used), it
> would be portable, but you said it is windows only. Also, I
> couldn't find any download link.


You might find http://rubyforge.org/snippet/detail....snippet&id=3D=
70
does some of what you want.

Chris


 
Reply With Quote
 
Stephen Kellett
Guest
Posts: n/a
 
      09-23-2005
In message <(E-Mail Removed) o.com>,
Eric Mahurin <(E-Mail Removed)> writes
>I don't care about graphs and such. Just a method (or several)
>in GC/ObjectSpace to get some memory info. If the above did
>what I'm thinking about (walk through each
>object/stack/symbol-table/etc and add up the memory used), it
>would be portable,


If you examine the source for Ruby you'll find that the above is
non-trivial and is implemented in C, in the form of the garbage
collector. I have offered to write a C API to collect such stats (and
will provide a simple Ruby API to provide other data to Ruby in the form
of a snapshot) but have not yet had the time to do it. Is this portable?
Yes if you regard compiling C on each platform to get the code working.

Someone else (sorry can't remember who) did write a Ruby snippet to walk
the object space. The main problem with this approach and any approach
that tries to trace ruby memory usage using say a set_trace_func() type
approach is the act of sending data to the ruby callback or walking the
object space in ruby allocates more objects, thus distorting the
picture, sometimes quite badly.

>but you said it is windows only.


It is written in C++ and assembly. It is impossible to do low level
hooking of the type required to write this software in anything other
than similar languages. You could not write these software tools in
Ruby, Python, Java, Lua, etc.

>Also, I
>couldn't find any download link.


I think I covered that when I wrote:

>No website description at the present time but you can apply for the

beta >at

Stephen
--
Stephen Kellett
Object Media Limited http://www.objmedia.demon.co.uk/software.html
Computer Consultancy, Software Development
Windows C++, Java, Assembler, Performance Analysis, Troubleshooting
 
Reply With Quote
 
Eric Mahurin
Guest
Posts: n/a
 
      09-23-2005


--- Chris McGrath <(E-Mail Removed)> wrote:

> On 9/23/05, Eric Mahurin <(E-Mail Removed)> wrote:
> > --- Stephen Kellett <(E-Mail Removed)> wrote:

> ...snipped...
> > I don't care about graphs and such. Just a method (or

> several)
> > in GC/ObjectSpace to get some memory info. If the above

> did
> > what I'm thinking about (walk through each
> > object/stack/symbol-table/etc and add up the memory used),

> it
> > would be portable, but you said it is windows only. Also,

> I
> > couldn't find any download link.

>=20
> You might find
> http://rubyforge.org/snippet/detail....nippet&id=3D70
> does some of what you want.


It looks to gauge memory usage by counting objects. Far from
accurate. And then it uses the same non-portable (linux only)
method I've been using to get the real answer:
/proc/<pid>/status.




=09
=09
__________________________________________________ ____=20
Yahoo! for Good=20
Donate to the Hurricane Katrina relief effort.=20
http://store.yahoo.com/redcross-donate3/=20



 
Reply With Quote
 
Scott Ellsworth
Guest
Posts: n/a
 
      09-23-2005
In article <dNwpj3Pr+DNDFws$@objmedia.demon.co.uk>,
Stephen Kellett <(E-Mail Removed)> wrote:

> In message <(E-Mail Removed) o.com>,
> Eric Mahurin <(E-Mail Removed)> writes


> >but you said it is windows only.

>
> It is written in C++ and assembly. It is impossible to do low level
> hooking of the type required to write this software in anything other
> than similar languages. You could not write these software tools in
> Ruby, Python, Java, Lua, etc.


None of the following is in any way a slight on what you have done - it
seems to be the only way _to_ do it with the current state of Ruby.

Java has a GC system that can track objects, return sizes, and all sorts
of groovy stuff, and communicate it over a socket to an external
profiler or debugger. Ruby, as best as I can read it, does not have
such hooks built in, so you have to grope the native memory model.

You can write such software tools in Java for Java. You will not be able
to write them in Ruby for Ruby, until such hooks are added into the Ruby
runtime.

I, for one, would like to see such hooks added, as I believe a lack of
memory profiling tools (built in at the lowest levels) is one of the
bigger things in the way of Ruby scaling. If a webapp is going to run
for months at a time, it really helps to be able to introspect long
lived and large objects. (Some of my Java webapps have over a year of
uptime, possible because we did that kind of memory leak testing. I
would like to be able to do the same with Ruby on Rails.)

Scott

--
Scott Ellsworth
http://www.velocityreviews.com/forums/(E-Mail Removed)
Java and database consulting for the life sciences
 
Reply With Quote
 
Stephen Kellett
Guest
Posts: n/a
 
      09-23-2005
In message <(E-Mail Removed)>, Scott
Ellsworth <(E-Mail Removed)> writes
>You can write such software tools in Java for Java.


Yes I appreciate that. I knew it when I wrote it. I was wondering who
would spot the language specific mistake and how quickly. Bonus points
for scoring on the first day! However for our tools it is nonsensical to
rearchitect and reimplement a fully working Coverage/Profiler/Flow
Tracer etc from C++ to Java just to support Java when we can do it
better and faster (both in implementation time and execution time) from
C++ in the first place (also, I'd much rather use JVMPI, JVMDI, JVMTI
than any of that over the wire Java stuff). I've used Java on and off
since 1996 and I just don't like it. It feels far too clumsy and lacks
expression, although interfaces are useful. C++ and assembly I am happy
with. Ruby I have a good feeling about although the syntax errors are
obscure and often in the "wrong" place if you know what I mean. Main
gripe about Ruby is that I don't have enough of a reason to use it for
real work (it isn't suited for writing high performance software tools)
as opposed to me experimenting with it - so I probably don't think quite
in the Ruby way like those of you that get to use Ruby more than I. I
digress.

>I, for one, would like to see such hooks added, as I believe a lack of
>memory profiling tools (built in at the lowest levels) is one of the
>bigger things in the way of Ruby scaling.


Well, I'll get around to the low level C side of things for the memory
API in the near future. I know what to write and how to write it. It
just a case of getting the time and having the energy to do it. It
should be easy for someone to wrap a Ruby layer for sockets around what
I write so that you can have the interface you describe.

Stephen
--
Stephen Kellett
Object Media Limited http://www.objmedia.demon.co.uk/software.html
Computer Consultancy, Software Development
Windows C++, Java, Assembler, Performance Analysis, Troubleshooting
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
medians for degree measurements Steve Howell Python 15 01-25-2010 06:36 PM
Sensor resolution, any sites with actual measurements? Rich Digital Photography 21 03-08-2006 07:39 AM
Re: interpreting the fractional portion of time.clock() vs time.time()measurements Peter Hansen Python 0 02-22-2006 12:03 AM
Tkinter measurements John Velman Python 3 09-28-2004 04:29 PM
Epson printers and lightfastness measurements Mike Digital Photography 13 12-22-2003 07:03 AM



Advertisments