Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Memoize and GC

Reply
Thread Tools

Memoize and GC

 
 
Intransition
Guest
Posts: n/a
 
      04-13-2010
I have a quick question. I just re-read:

http://www.engineyard.com/blog/2010/...on-and-id2ref/

I am wondering about garbage collection. Am I right to think that
using

def foo
@foo ||= something
end

will mean that @foo be GC'd when the object is GC'd, but if I do

FOO = []

def foo
FOO[object_id] ||= something
end

Then the memorized entry will just hang around forever?

If so, then how do we do it without polluting the instance var space
(as in example 2), but still get the garbage collection (as in example
1)?

Thanks.

 
Reply With Quote
 
 
 
 
Robert Klemme
Guest
Posts: n/a
 
      04-13-2010
On 04/13/2010 06:26 PM, Intransition wrote:
> I have a quick question. I just re-read:
>
> http://www.engineyard.com/blog/2010/...on-and-id2ref/
>
> I am wondering about garbage collection. Am I right to think that
> using
>
> def foo
> @foo ||= something
> end
>
> will mean that @foo be GC'd when the object is GC'd, but if I do
>
> FOO = []
>
> def foo
> FOO[object_id] ||= something
> end
>
> Then the memorized entry will just hang around forever?
>
> If so, then how do we do it without polluting the instance var space
> (as in example 2), but still get the garbage collection (as in example
> 1)?


I would live with polluting the instance var space. Btw, I usually do
not use memoize but rather an explicit Hash with default_proc. That way
I have control over where I place the Hash and hence how long it lives.

Of course you can hack something together using #object_id and an
external Hash but then you also need to create a finalizer and soon
things get overly complicated. If you have issues with polluting the
instance variable namespace you can a) put all memoize containers into a
single instance variable (typically a Hash) and / or b) generate
instance variable names which are likely to be conflict free.

Kind regards

robert

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/
 
Reply With Quote
 
 
 
 
Jesús Gabriel y Galán
Guest
Posts: n/a
 
      04-13-2010
On Tue, Apr 13, 2010 at 6:26 PM, Intransition <(E-Mail Removed)> wrote:
> I have a quick question. I just re-read:
>
> =A0http://www.engineyard.com/blog/2010/memoization-and-id2ref/
>
> I am wondering about garbage collection. Am I right to think that
> using
>
> =A0def foo
> =A0 =A0@foo ||=3D something
> =A0end
>
> will mean that @foo be GC'd when the object is GC'd, but if I do
>
> =A0FOO =3D []
>
> =A0def foo
> =A0 =A0FOO[object_id] ||=3D something
> =A0end
>
> Then the memorized entry will just hang around forever?
>
> If so, then how do we do it without polluting the instance var space
> (as in example 2), but still get the garbage collection (as in example
> 1)?
>
> Thanks.
>
>


One way could be using a closure that redefines the method:

irb(main):019:0> class A
irb(main):020:1> def a
irb(main):021:2> puts "expensive calculation of a..."
irb(main):022:2> a =3D "complex value"
irb(main):023:2> self.class.send(:define_method, :a) do
irb(main):024:3* a
irb(main):025:3> end
irb(main):026:2> a
irb(main):027:2> end
irb(main):028:1> end
=3D> nil
irb(main):029:0> obj =3D A.new
=3D> #<A:0xb7c9ef20>
irb(main):030:0> obj.a
expensive calculation of a...
=3D> "complex value"
irb(main):031:0> obj.a
=3D> "complex value"

Jesus.

 
Reply With Quote
 
Caleb Clausen
Guest
Posts: n/a
 
      04-13-2010
On 4/13/10, Intransition <(E-Mail Removed)> wrote:
> will mean that @foo be GC'd when the object is GC'd, but if I do
>
> FOO = []
>
> def foo
> FOO[object_id] ||= something
> end


I have to assume that you meant FOO to be a hash... tho an array will
work here... sorta.

But rather than either an array or hash, what if you could write:

FOO=Cache.new

That is, FOO would behave mostly like a hash, but unused entries would
age out after a while. Now if only someone would invent an appropriate
Cache class.

 
Reply With Quote
 
Intransition
Guest
Posts: n/a
 
      04-13-2010


On Apr 13, 1:28=A0pm, Caleb Clausen <(E-Mail Removed)> wrote:
> On 4/13/10, Intransition <(E-Mail Removed)> wrote:
>
> > will mean that @foo be GC'd when the object is GC'd, but if I do

>
> > =A0 FOO =3D []

>
> > =A0 def foo
> > =A0 =A0 FOO[object_id] ||=3D something
> > =A0 end

>
> I have to assume that you meant FOO to be a hash... tho an array will
> work here... sorta.
>
> But rather than either an array or hash, what if you could write:
>
> =A0 FOO=3DCache.new
>
> That is, FOO would behave mostly like a hash, but unused entries would
> age out after a while. Now if only someone would invent an appropriate
> Cache class.


Ah, very good thought! An LRUCache would do pretty well here.

Well then, that leads to me to another project. My implementation of
LRUCache (lrucache gem) kind of sucks in that it only handles max
size, so I am looking for a new one to replace it. Any
recommendations?


 
Reply With Quote
 
Robert Klemme
Guest
Posts: n/a
 
      04-14-2010
2010/4/13 Intransition <(E-Mail Removed)>:
>
>
> On Apr 13, 1:28=A0pm, Caleb Clausen <(E-Mail Removed)> wrote:
>> On 4/13/10, Intransition <(E-Mail Removed)> wrote:
>>
>> > will mean that @foo be GC'd when the object is GC'd, but if I do

>>
>> > =A0 FOO =3D []

>>
>> > =A0 def foo
>> > =A0 =A0 FOO[object_id] ||=3D something
>> > =A0 end

>>
>> I have to assume that you meant FOO to be a hash... tho an array will
>> work here... sorta.
>>
>> But rather than either an array or hash, what if you could write:
>>
>> =A0 FOO=3DCache.new
>>
>> That is, FOO would behave mostly like a hash, but unused entries would
>> age out after a while. Now if only someone would invent an appropriate
>> Cache class.

>
> Ah, very good thought! An LRUCache would do pretty well here.
>
> Well then, that leads to me to another project. My implementation of
> LRUCache (lrucache gem) kind of sucks in that it only handles max
> size, so I am looking for a new one to replace it. Any
> recommendations?


What else do you want to handle? My implementation [1] does also only
consider size but I have also seen implementations that consider
insertion or access time. Personally I dislike those solutions
because they might throw away stuff too early and make the
implementation more complex. The main point (resource control) is
already achieved by limiting the size of a LRU map.

Kind regards

robert

http://github.com/rklemme/muppet-lab...lib/lruhash.rb


--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

 
Reply With Quote
 
Robert Dober
Guest
Posts: n/a
 
      04-14-2010
2010/4/13 Jes=FAs Gabriel y Gal=E1n <(E-Mail Removed)>:
> On Tue, Apr 13, 2010 at 6:26 PM, Intransition <(E-Mail Removed)> wrote=

:
>> I have a quick question. I just re-read:
>>
>> =A0http://www.engineyard.com/blog/2010/memoization-and-id2ref/
>>
>> I am wondering about garbage collection. Am I right to think that
>> using
>>
>> =A0def foo
>> =A0 =A0@foo ||=3D something
>> =A0end
>>
>> will mean that @foo be GC'd when the object is GC'd, but if I do
>>
>> =A0FOO =3D []
>>
>> =A0def foo
>> =A0 =A0FOO[object_id] ||=3D something
>> =A0end
>>
>> Then the memorized entry will just hang around forever?
>>
>> If so, then how do we do it without polluting the instance var space
>> (as in example 2), but still get the garbage collection (as in example
>> 1)?
>>
>> Thanks.
>>
>>

>
> One way could be using a closure that redefines the method:
>
> irb(main):019:0> class A
> irb(main):020:1> def a
> irb(main):021:2> puts "expensive calculation of a..."
> irb(main):022:2> a =3D "complex value"
> irb(main):023:2> self.class.send(:define_method, :a) do

I like the closure idea, but I guess the memozing got lost a little bit .
thus I would write
def a
_a =3D nil
self.class.module_eval do
define_method :a do _a ||=3D "expensive calculation" end
define_method :_invalidate_a do _a =3D nil end
end
a
end
Cheers
R.

--=20
Learning without thought is labor lost; thought without learning is perilou=
s.=94
--- Confucius

 
Reply With Quote
 
Intransition
Guest
Posts: n/a
 
      04-14-2010

On Apr 14, 7:02=A0am, Robert Klemme <(E-Mail Removed)> wrote:

> What else do you want to handle? =A0My implementation [1] does also only
> consider size but I have also seen implementations that consider
> insertion or access time. =A0Personally I dislike those solutions
> because they might throw away stuff too early and make the
> implementation more complex. =A0The main point (resource control) is
> already achieved by limiting the size of a LRU map.


Fair enough, I just thought it would be nice to have the *option* of a
time-based cache.

 
Reply With Quote
 
Robert Klemme
Guest
Posts: n/a
 
      04-15-2010
2010/4/14 Intransition <(E-Mail Removed)>:
>
> On Apr 14, 7:02=A0am, Robert Klemme <(E-Mail Removed)> wrote:
>
>> What else do you want to handle? =A0My implementation [1] does also only
>> consider size but I have also seen implementations that consider
>> insertion or access time. =A0Personally I dislike those solutions
>> because they might throw away stuff too early and make the
>> implementation more complex. =A0The main point (resource control) is
>> already achieved by limiting the size of a LRU map.

>
> Fair enough, I just thought it would be nice to have the *option* of a
> time-based cache.


Definitively! But this would be a different beast for me. One might
be able to implement this by using a LRUHash or by inheriting from it.
But I would not combine that in a single class. Another option could
be to have a modular approach where the container gets a
DeletionPolicy that is responsible for deciding when or what to
delete. The interface could look like

interface DeletionPolicy
def insert(key,val)
def remove(key, val)
def access(key, val)
def next_delete # yields key
end

Just a quick hack though withou

Kind regards

robert

--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

 
Reply With Quote
 
Intransition
Guest
Posts: n/a
 
      04-15-2010


On Apr 15, 3:16=A0am, Robert Klemme <(E-Mail Removed)> wrote:
> 2010/4/14 Intransition <(E-Mail Removed)>:
>
>
>
> > On Apr 14, 7:02=A0am, Robert Klemme <(E-Mail Removed)> wrote:

>
> >> What else do you want to handle? =A0My implementation [1] does also on=

ly
> >> consider size but I have also seen implementations that consider
> >> insertion or access time. =A0Personally I dislike those solutions
> >> because they might throw away stuff too early and make the
> >> implementation more complex. =A0The main point (resource control) is
> >> already achieved by limiting the size of a LRU map.

>
> > Fair enough, I just thought it would be nice to have the *option* of a
> > time-based cache.

>
> Definitively! =A0But this would be a different beast for me. =A0One might
> be able to implement this by using a LRUHash or by inheriting from it.
> =A0But I would not combine that in a single class. =A0Another option coul=

d
> be to have a modular approach where the container gets a
> DeletionPolicy that is responsible for deciding when or what to
> delete. =A0The interface could look like
>
> interface DeletionPolicy
> =A0 def insert(key,val)
> =A0 def remove(key, val)
> =A0 def access(key, val)
> =A0 def next_delete # yields key
> end
>
> Just a quick hack though withou


I kind of like that idea.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
memoize and yaml Brian Buckley Ruby 10 11-09-2005 05:35 PM
problem using both Memoize::Expire and DB_File samwyse Perl Misc 2 07-21-2005 03:11 AM
RE: How to memoize functions? sismex01@hebmex.com Python 2 06-27-2003 05:25 PM
How to memoize functions? Chris Reedy Python 3 06-27-2003 02:18 PM
RE: How to memoize functions? sismex01@hebmex.com Python 0 06-26-2003 08:45 PM



Advertisments