Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > Using SoftReferences for caching

Reply
Thread Tools

Using SoftReferences for caching

 
 
Jesper Nordenberg
Guest
Posts: n/a
 
      02-12-2004
I'm writing an application where I have a memory cache of objects
stored on disk. There are no limitations on the object sizes, one
object can be as large as the heap. A long value is used to identify
an object, so there can be a huge amount of objects.

I want to use a fixed amount of the heap or all the available heap for
the cache. I can't cache a fixed number of objects since I don't know
how how much memory an object occupies, and thus the program can run
out of memory. So, using SoftReferences seems like a good solution.
One problem is that the HashMap used in the cache can become very big
and needs to be cleared of SoftReferences that has been nullified.

If someone have some implementation or design suggestion that would be
great. Maybe there is some open source project that implement a
similar cache?

/Jesper Nordenberg
 
Reply With Quote
 
 
 
 
Conor White
Guest
Posts: n/a
 
      02-12-2004
You need to associate each SoftReference with a ReferenceQueue object.
When a SoftReference is Garbage collected it is placed in the
ReferenceQueue. This gives you a chance to do a cleanup before the
object is finally lost.


http://www.velocityreviews.com/forums/(E-Mail Removed) (Jesper Nordenberg) wrote in message news:<(E-Mail Removed) om>...
> I'm writing an application where I have a memory cache of objects
> stored on disk. There are no limitations on the object sizes, one
> object can be as large as the heap. A long value is used to identify
> an object, so there can be a huge amount of objects.
>
> I want to use a fixed amount of the heap or all the available heap for
> the cache. I can't cache a fixed number of objects since I don't know
> how how much memory an object occupies, and thus the program can run
> out of memory. So, using SoftReferences seems like a good solution.
> One problem is that the HashMap used in the cache can become very big
> and needs to be cleared of SoftReferences that has been nullified.
>
> If someone have some implementation or design suggestion that would be
> great. Maybe there is some open source project that implement a
> similar cache?
>
> /Jesper Nordenberg

 
Reply With Quote
 
 
 
 
Carl Howells
Guest
Posts: n/a
 
      02-12-2004
Conor White wrote:
> You need to associate each SoftReference with a ReferenceQueue object.
> When a SoftReference is Garbage collected it is placed in the
> ReferenceQueue. This gives you a chance to do a cleanup before the
> object is finally lost.
>


Be careful which object you're talking about... When a SoftReference is
placed in a ReferenceQueue, the object the SoftReference referred to has
already been collected. There is no way to get the object back at that
point. All you can get out of the ReferenceQueue is the SoftReference.

This does still solve the OP's problem... But be careful how you phrase
things.
 
Reply With Quote
 
Jesper Nordenberg
Guest
Posts: n/a
 
      02-13-2004
(E-Mail Removed) (Conor White) wrote in message news:<(E-Mail Removed). com>...
> You need to associate each SoftReference with a ReferenceQueue object.
> When a SoftReference is Garbage collected it is placed in the
> ReferenceQueue. This gives you a chance to do a cleanup before the
> object is finally lost.


I tried this. Two problems I encountered:

- You need another (reverse) HashMap to look up the entry of the
SoftReference that was placed in the ReferenceQueue.

- When testing with multiple threads I sometimes ran out of memory. It
seemed there was some nullified SoftReferences in the map that haven't
been placed in the ReferenceQueue. Maybe it was an error in my
program.

My current solution is to regurarly traverse the map and remove
nullified SoftReferences.

/Jesper Nordenberg
 
Reply With Quote
 
Adam Jenkins
Guest
Posts: n/a
 
      02-13-2004
Jesper Nordenberg wrote:
> (E-Mail Removed) (Conor White) wrote in message news:<(E-Mail Removed). com>...
>
>>You need to associate each SoftReference with a ReferenceQueue object.
>>When a SoftReference is Garbage collected it is placed in the
>>ReferenceQueue. This gives you a chance to do a cleanup before the
>>object is finally lost.

>
>
> I tried this. Two problems I encountered:
>
> - You need another (reverse) HashMap to look up the entry of the
> SoftReference that was placed in the ReferenceQueue.


Typically you would just use your own subclass of SoftReference, with
some extra fields containing whatever information you need to clean up
after it. No need for a reverse map.

Adam

 
Reply With Quote
 
Conor White
Guest
Posts: n/a
 
      02-15-2004
Look at the code for the java.util.WeakHashMap class. This does what
you want except it uses weak references instead of soft references.

Adam Jenkins <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>...
> Jesper Nordenberg wrote:
> > (E-Mail Removed) (Conor White) wrote in message news:<(E-Mail Removed). com>...
> >
> >>You need to associate each SoftReference with a ReferenceQueue object.
> >>When a SoftReference is Garbage collected it is placed in the
> >>ReferenceQueue. This gives you a chance to do a cleanup before the
> >>object is finally lost.

> >
> >
> > I tried this. Two problems I encountered:
> >
> > - You need another (reverse) HashMap to look up the entry of the
> > SoftReference that was placed in the ReferenceQueue.

>
> Typically you would just use your own subclass of SoftReference, with
> some extra fields containing whatever information you need to clean up
> after it. No need for a reverse map.
>
> Adam

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Disable page caching without disabling caching of jpegs andstylesheets etc JimLad ASP .Net 3 01-21-2010 10:13 AM
Re: Controlling SoftReferences Kevin McMurtrie Java 3 07-07-2008 07:49 PM
Memory leak and SoftReferences Joe Java 11 04-02-2006 09:35 PM
Fragment Caching inside page caching? Troy Simpson ASP .Net 0 01-19-2004 11:57 AM
trouble with caching or caching the trouble Hypo ASP .Net 6 08-01-2003 07:11 AM



Advertisments