Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > How to persist and query a hash lookup in memory with Perl ?

Reply
Thread Tools

How to persist and query a hash lookup in memory with Perl ?

 
 
Jack
Guest
Posts: n/a
 
      11-29-2007
Hi I have a web application that needs to on an adhoc basis query a
hash lookup which should be persisted in memory - I have seen this
module "Persistent::Memory - A Persistent Class implemented using
Memory (RAM)" but I wanted to see if there was a much simpler approach
to taking a hash map and making it 'callable' while persisted in
memory using standard features not requiring a special Perl module
install..

Any thoughts would be appreciated.
Thanks,
Jack

 
Reply With Quote
 
 
 
 
Gunnar Hjalmarsson
Guest
Posts: n/a
 
      11-29-2007
Jack wrote:
> Hi I have a web application that needs to on an adhoc basis query a
> hash lookup which should be persisted in memory - I have seen this
> module "Persistent::Memory - A Persistent Class implemented using
> Memory (RAM)" but I wanted to see if there was a much simpler approach
> to taking a hash map and making it 'callable' while persisted in
> memory using standard features not requiring a special Perl module
> install..
>
> Any thoughts would be appreciated.


Why not a tied hash? See e.g. "perldoc DB_File".

--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl
 
Reply With Quote
 
 
 
 
jackposemsky@yahoo.com
Guest
Posts: n/a
 
      11-29-2007
On Nov 28, 6:37 pm, Gunnar Hjalmarsson <(E-Mail Removed)> wrote:
> Jack wrote:
> > Hi I have a web application that needs to on an adhoc basis query a
> > hash lookup which should be persisted in memory - I have seen this
> > module "Persistent::Memory - A Persistent Class implemented using
> > Memory (RAM)" but I wanted to see if there was a much simpler approach
> > to taking a hash map and making it 'callable' while persisted in
> > memory using standard features not requiring a special Perl module
> > install..

>
> > Any thoughts would be appreciated.

>
> Why not a tied hash? See e.g. "perldoc DB_File".
>
> --
> Gunnar Hjalmarsson
> Email:http://www.gunnar.cc/cgi-bin/contact.pl


Hi and thank you however that looks too complex - just trying to put a
hash in memory, why is that so hard to do ? Is there a simpler way ?
 
Reply With Quote
 
A. Sinan Unur
Guest
Posts: n/a
 
      11-29-2007
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote in
news:(E-Mail Removed):

> On Nov 28, 6:37 pm, Gunnar Hjalmarsson <(E-Mail Removed)> wrote:
>> Jack wrote:
>> > Hi I have a web application that needs to on an adhoc basis query a
>> > hash lookup which should be persisted in memory - I have seen this
>> > module "Persistent::Memory - A Persistent Class implemented using
>> > Memory (RAM)" but I wanted to see if there was a much simpler
>> > approach to taking a hash map and making it 'callable' while
>> > persisted in memory using standard features not requiring a special
>> > Perl module install..

>>
>> > Any thoughts would be appreciated.

>>
>> Why not a tied hash? See e.g. "perldoc DB_File".
>>
>> --
>> Gunnar Hjalmarsson
>> Email:http://www.gunnar.cc/cgi-bin/contact.pl


Don't quote sigs.

> Hi and thank you however that looks too complex - just trying to put a
> hash in memory, why is that so hard to do ? Is there a simpler way ?


It looks like programming might be too hard for you.

Sinan

--
A. Sinan Unur <(E-Mail Removed)>
(remove .invalid and reverse each component for email address)
clpmisc guidelines: <URL:http://www.augustmail.com/~tadmc/clpmisc.shtml>

 
Reply With Quote
 
Ted Zlatanov
Guest
Posts: n/a
 
      11-29-2007
On Wed, 28 Nov 2007 18:22:34 -0800 (PST) Jack <(E-Mail Removed)> wrote:

J> Hi I have a web application that needs to on an adhoc basis query a
J> hash lookup which should be persisted in memory - I have seen this
J> module "Persistent::Memory - A Persistent Class implemented using
J> Memory (RAM)" but I wanted to see if there was a much simpler approach
J> to taking a hash map and making it 'callable' while persisted in
J> memory using standard features not requiring a special Perl module
J> install..

Are you talking about sharing the data between several instances of your
web application, so whichever one handles a request can read or update
data?

Use a database. There are cases where this is not the right approach,
but it doesn't sound like it in yours.

Ted
 
Reply With Quote
 
xhoster@gmail.com
Guest
Posts: n/a
 
      11-29-2007
(E-Mail Removed) wrote:
> On Nov 28, 6:37 pm, Gunnar Hjalmarsson <(E-Mail Removed)> wrote:
> > Jack wrote:
> > > Hi I have a web application that needs to on an adhoc basis query a
> > > hash lookup which should be persisted in memory - I have seen this
> > > module "Persistent::Memory - A Persistent Class implemented using
> > > Memory (RAM)" but I wanted to see if there was a much simpler
> > > approach to taking a hash map and making it 'callable' while
> > > persisted in memory using standard features not requiring a special
> > > Perl module install..

> >
> > > Any thoughts would be appreciated.

> >
> > Why not a tied hash? See e.g. "perldoc DB_File".

>
> Hi and thank you however that looks too complex - just trying to put a
> hash in memory, why is that so hard to do ? Is there a simpler way ?


If you aren't willing to install modules and you aren't willing to use the
ones that are (often) standard, then I guess the next option would be to
hire someone who is willing to do one of those things.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.
 
Reply With Quote
 
Jack
Guest
Posts: n/a
 
      11-29-2007
On Nov 29, 9:56 am, Jim Gibson <(E-Mail Removed)> wrote:
> In article
> <(E-Mail Removed)>,
>
>
>
>
>
> <(E-Mail Removed)> wrote:
> > On Nov 28, 6:37 pm, Gunnar Hjalmarsson <(E-Mail Removed)> wrote:
> > > Jack wrote:
> > > > Hi I have a web application that needs to on an adhoc basis query a
> > > > hash lookup which should be persisted in memory - I have seen this
> > > > module "Persistent::Memory - A Persistent Class implemented using
> > > > Memory (RAM)" but I wanted to see if there was a much simpler approach
> > > > to taking a hash map and making it 'callable' while persisted in
> > > > memory using standard features not requiring a special Perl module
> > > > install..

>
> > > > Any thoughts would be appreciated.

>
> > > Why not a tied hash? See e.g. "perldoc DB_File".

>
> > > --
> > > Gunnar Hjalmarsson
> > > Email:http://www.gunnar.cc/cgi-bin/contact.pl

>
> > Hi and thank you however that looks too complex - just trying to put a
> > hash in memory, why is that so hard to do ? Is there a simpler way ?

>
> Your question is confusing. All Perl variables, including hashes, are
> in memory. The trick to achieving "persistence" between execution
> instances of your web application is to have the data on a permanent
> data store, such as a disk file. Memory variables disappear once your
> Perl process terminates. Using a tied hash, per Gunnar's suggestion, is
> one way to accomplish this. There are others, such as using the
> Storable module to serialize your hash so you can write it to memory.
>
> I have not use Persistent::Memory, but you might want to check out the
> Persistent::File module instead. Both of those modules seem to be part
> of the Persistent framework of Perl modules.
>
> See
> <http://search.cpan.org/~dwinters/Per.../Persistent.po
> d>
>
> --
> Jim Gibson
>
> Posted Via Usenet.com Premium Usenet Newsgroup Services
> ----------------------------------------------------------
> ** SPEED ** RETENTION ** COMPLETION ** ANONYMITY **
> ----------------------------------------------------------
> http://www.usenet.com- Hide quoted text -
>
> - Show quoted text -


Thanks I am willing to install Perl Modules, really wondering if there
is a native ability to do it out of the box in Perl. I got persistent
memory to work but its not clear how to "pin" a hash lookup into
memory since .pl files and the persistent memory code example itself
are only as good as the life of the execution of the script !! Still
not clear how to hold a has in Memory - even if you were to create a
windows service out of it with activestate PDK, not clear how to write
the script so the script doesnt stop execution but sits and waits for
a socket connection or something.. Do I just have script 1 generate
a hash with "sleep forever" and then script 2 can call that hash ??
I dont want to use another database for this. So, not using another
DB, how does one pin and persist and then query a hash in memory using
perl or a perl module.

Thanks for the help all !
Jack
 
Reply With Quote
 
Ben Morrow
Guest
Posts: n/a
 
      11-29-2007

Quoth Jack <(E-Mail Removed)>:
<snip: some sort of persistant shared memory>
>
> Thanks I am willing to install Perl Modules, really wondering if there
> is a native ability to do it out of the box in Perl.


Well, on systems that support it you can use SysV shared memory, for
which Perl natively has the raw shm* calls. This doesn't work on
Windows, though.

You could try IPC::Shareable for Unix and Win32::MMF::Shareable (which
has the same interface) for Win32, which provide an easier interface to
shared memory.

Another alternative would be mmap, using the Sys::MMap module. mmap is
substantially better behaved than SysV shm, and portable to Windows.

> I got persistent memory to work but its not clear how to "pin" a hash
> lookup into memory since .pl files and the persistent memory code
> example itself are only as good as the life of the execution of the
> script !!


In what sense then is this memory 'persistent'? I think you are rather
confused about what is and isn't possible under modern operating
systems: unless you take steps to prevent it, all memory allocated by a
process is released when that process exits. This is a feature .

Ben

 
Reply With Quote
 
J. Gleixner
Guest
Posts: n/a
 
      11-29-2007
Jack wrote:
[...]
> So, not using another DB


If you're already using a Database then just store the data there, why
would you use another Database to store it?

>how does one pin and persist and then query a hash in memory using
> perl or a perl module.


Whatever 'pin' a hash means.

Why do you want to do this? Maybe that's the real problem. You
think that's what you want to do, but it's really not needed.

You could have a daemon running, that reads/creates the hash and
then it would have the data in memory. You could then then connect
to it via a socket, but why the heck do you want to do all of
that? That's really not Perl 101 and I'm pretty sure you'd
end up spending a lot of time understanding it and trying to get it
working, however it'll take 20-seconds to write and will be very
fast, using a tied hash.

There are a lot of examples online and you can read the documentation
that comes with perl.

perldoc DB_File

If you want to go the whole daemon and IPC/socket route:

perldoc perlipc
 
Reply With Quote
 
Jack
Guest
Posts: n/a
 
      11-30-2007
On Nov 29, 3:35 pm, Ben Morrow <(E-Mail Removed)> wrote:
> Quoth Jack <(E-Mail Removed)>:
> <snip: some sort of persistant shared memory>
>
>
>
> > Thanks I am willing to install Perl Modules, really wondering if there
> > is a native ability to do it out of the box in Perl.

>
> Well, on systems that support it you can use SysV shared memory, for
> which Perl natively has the raw shm* calls. This doesn't work on
> Windows, though.
>
> You could try IPC::Shareable for Unix and Win32::MMF::Shareable (which
> has the same interface) for Win32, which provide an easier interface to
> shared memory.
>
> Another alternative would be mmap, using the Sys::MMap module. mmap is
> substantially better behaved than SysV shm, and portable to Windows.
>
> > I got persistent memory to work but its not clear how to "pin" a hash
> > lookup into memory since .pl files and the persistent memory code
> > example itself are only as good as the life of the execution of the
> > script !!

>
> In what sense then is this memory 'persistent'? I think you are rather
> confused about what is and isn't possible under modern operating
> systems: unless you take steps to prevent it, all memory allocated by a
> process is released when that process exits. This is a feature .
>
> Ben


ok.. lets talk with code - I got this to work with DB_File.. my
fundamental question is, once you create the tied hash like the below,
and the program is done executing, how do I access the hash and query
it from another Perl execution program ?? I tried using the API
interface afterwards as described here http://perldoc.perl.org/DB_File.html
, see the error at bottom:
### RUN PROGRAM 1 #####
use warnings ;
use strict ;
use DB_File ;
our (%h, $k, $v) ; unlink "fruit" ;
tie %h, "DB_File", "fruit", O_RDWR|O_CREAT, 0666, $DB_HASH
or die "Cannot open file 'fruit': $!\n"; # Add a few key/
value pairs to the file
$h{"apple"} = "red" ;
$h{"orange"} = "orange" ;
$h{"banana"} = "yellow" ;
$h{"tomato"} = "red" ; # Check for existence of a key
print "Banana Exists\n\n" if $h{"banana"} ; # Delete a key/
value pair.
delete $h{"apple"} ; # print the contents of the file
while (($k, $v) = each %h)
{ print "$k -> $v\n" }
untie %h ;

#####START SEPARATE PROGRAM 2 ###
use warnings ;
use strict ;
use DB_File ;
$db = tie %h, "DB_File", "testfile1" ;
foreach (keys %h)
{ print "$_\n" }
### PROGRAM 2 ERRORS:
Global symbol "$db" requires explicit package name at
test_dbfile1_query.pl .
Global symbol "%h" requires explicit package name at
test_dbfile1_query.pl .
Global symbol "%h" requires explicit package name at
test_dbfile1_query.pl 5.
Execution of test_dbfile1_query.pl aborted due to compilation errors.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: How include a large array? Edward A. Falk C Programming 1 04-04-2013 08:07 PM
hash of hash of hash of hash in c++ rp C++ 1 11-10-2011 04:45 PM
Hash#select returns an array but Hash#reject returns a hash... Srijayanth Sridhar Ruby 19 07-02-2008 12:49 PM
Perl hash and rehash seeds; deterministic hash ordering ozgune@gmail.com Perl Misc 4 01-22-2007 07:58 PM
Hash of hash in perl Shashank Khanvilkar Perl Misc 3 11-18-2004 08:44 PM



Advertisments