Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Perl Misc (http://www.velocityreviews.com/forums/f67-perl-misc.html)
-   -   Using MLDBM on objects? (http://www.velocityreviews.com/forums/t893113-using-mldbm-on-objects.html)

hachuah@gmail.com 06-30-2005 11:08 PM

Using MLDBM on objects?
 
Hi,

I'm trying to use MLDBM on an object I've written in Perl, and cannot
get it to work right. Here's my problem:

I'm trying to parse a file, and build a data structure in memory that
will enable me to easily make transformations, and then write the file
back out. I'm using OOP to do this. My object consists of:

in Bigobject.pm
$self->{hash1} = \%hash1 (multidimensional hash);
where $hash1{$a}{$b} = \$a_dag_node
$self->{hash2} = \%hash2 (multidimensional hash);
where $hash2{$c}{$d} = \$another_dag_node
$self->{rootnode} - Points to tree of DAG_Nodes. (from DAG_Node.pm)

in main.pl
$bo = Bigobject->new(); $bo->parse("$file");

Now, the problem is that I'm running out-of-memory while building this
object; therefore, I'm trying to use MLDBM to put all or part of it on
disk. I've tried the following, and it doesn't work (memory usage is
still huge):

-----1----------------------
in main.pl
use MLDBM; # this gets SDBM and Data::Dumper
use Fcntl; # to get 'em constants

$dbm = tie %bo, 'MLDBM', 'testmldbm', O_CREAT|O_RDWR, 0640 or die $!;
$bo{1} = Bigobject->new(); $bo{1}->parse("$file");

-----2----------------------
in Bigobject.pm
sub parse {
<snip>
use MLDBM; # this gets SDBM and Data::Dumper
use Fcntl; # to get 'em constants
$dbm = tie %hash1, 'MLDBM', 'testmldbm', O_CREAT|O_RDWR, 0640 or die
$!;
$self->{hash1} = \%hash1;
<...>
$hash1{$a}{$b} = $pointer_to_dag_node

Anyone have any ideas?

thanks,
hachuah


Joe Y Perro 07-01-2005 07:30 PM

Re: Using MLDBM on objects?
 

>I'm trying to use MLDBM on an object I've written in Perl, and cannot
>get it to work right. Here's my problem:
>

I ran into this problem also. Here are two ways to solve it:

1. Use a different module .... look at storable:
http://search.cpan.org/~ams/Storable-2.15/Storable.pm
I think this will do exactly what you want. It does for me -- by
coincidence my application also uses Tree::DAG_Node... with storable I
build complex hash structures and write them to disk using the modules
store method, and later read them via its retrieve methods.

2. MLDBM only handles simple one level hashes, i.e. $hash{key} = $x
..... it can't handle structures like the one you write:

$hash1{$a}{$b} = $pointer_to_dag_node

As a workaround I overcame this problem by building a temporary hash
and then assigning it to the one you tie with MLDBM. Its been a few
years since I did this so memory is fading, but here is the basic
idea:

my (%H, $hash_file, %tmp);
tie %H, "MLDBM", $hash_file, O_CREAT|O_RDWR, 0777 || die "couldn't
tie hash: $!";
#
$tmp{key1}{key2} = value1;
$temp{kley1}{key3} = value3;
#
$H{key1} = $tmp{key1};


Joe



Anno Siegel 07-01-2005 09:43 PM

Re: Using MLDBM on objects?
 
Joe Y Perro <joe_y_perroREMOVETHIS@yahooREMOVETHISALSOcom> wrote in comp.lang.perl.misc:
>
> >I'm trying to use MLDBM on an object I've written in Perl, and cannot
> >get it to work right. Here's my problem:


[...]

> 2. MLDBM only handles simple one level hashes, i.e. $hash{key} = $x
> .... it can't handle structures like the one you write:
>
> $hash1{$a}{$b} = $pointer_to_dag_node


That isn't true. It is the express purpose of MLDBM (Multi Level Database
mumble) to support nested structures.

> As a workaround I overcame this problem by building a temporary hash
> and then assigning it to the one you tie with MLDBM. Its been a few
> years since I did this so memory is fading, but here is the basic
> idea:
>
> my (%H, $hash_file, %tmp);
> tie %H, "MLDBM", $hash_file, O_CREAT|O_RDWR, 0777 || die "couldn't
> tie hash: $!";
> #
> $tmp{key1}{key2} = value1;
> $temp{kley1}{key3} = value3;
> #
> $H{key1} = $tmp{key1};


MLDBM appears to have an issue with autovivification, but that's a bug,
not a deliberate property. You can also work around the bug without
an auxiliary variable:

$H{ key1} = { key2 => { key3 => 'value3'}};

Anno


All times are GMT. The time now is 03:57 AM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.