Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > Handling a 450,000x450,000 array with Perl

Reply
Thread Tools

Handling a 450,000x450,000 array with Perl

 
 
francescomoi@usa.com
Guest
Posts: n/a
 
      10-30-2006
Hi.

I'd like to create a program to handle data (natural numbers)
within a 450,000 x 450,000 array. I'm considering Perl, but
I don't know is it's powerful enough to manage it.

Thank you very much.

 
Reply With Quote
 
 
 
 
Ingo Menger
Guest
Posts: n/a
 
      10-30-2006

http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:
> Hi.
>
> I'd like to create a program to handle data (natural numbers)
> within a 450,000 x 450,000 array. I'm considering Perl, but
> I don't know is it's powerful enough to manage it.


Perl is, but you'll find few computers that could store 202 billion
"natural numbers".
Remember, a PC has a RAM of around 2 billion bytes, give or take a few.
You could hold a maximum of 16 billion natural numbers as long as they
are 0 or 1. But, 16 billion is still not 202 billion.

In any case, this will be a system where native integers have 64 bits.

 
Reply With Quote
 
 
 
 
francescomoi@usa.com
Guest
Posts: n/a
 
      10-30-2006
Hi.

Thank you very much for your quick answers. I try to handle data from a
450x450 meter surface.

I've got 100,000 elements distributed on this surface, and each
square millimeter can hold one or more elements. So the array to create
is a sparse one: most of square millimeters holds zero elements, and
the rest, 20 or less.

I'm trying to find the square meter with more elements inside.

Thank you very much.

On Oct 30, 3:24 pm, "Ingo Menger" <(E-Mail Removed)> wrote:
> (E-Mail Removed) wrote:
> > Hi.

>
> > I'd like to create a program to handle data (natural numbers)
> > within a 450,000 x 450,000 array. I'm considering Perl, but
> > I don't know is it's powerful enough to manage it.Perl is, but you'll find few computers that could store 202 billion

> "natural numbers".
> Remember, a PC has a RAM of around 2 billion bytes, give or take a few.
> You could hold a maximum of 16 billion natural numbers as long as they
> are 0 or 1. But, 16 billion is still not 202 billion.
>
> In any case, this will be a system where native integers have 64 bits.


 
Reply With Quote
 
Ted Zlatanov
Guest
Posts: n/a
 
      10-30-2006
On 30 Oct 2006, bugbear@trim_papermule.co.uk_trim wrote:

(E-Mail Removed) wrote:
>> Thank you very much for your quick answers. I try to handle data from a
>> 450x450 meter surface.
>>
>> I've got 100,000 elements distributed on this surface, and each
>> square millimeter can hold one or more elements. So the array to create
>> is a sparse one: most of square millimeters holds zero elements, and
>> the rest, 20 or less.
>>
>> I'm trying to find the square meter with more elements inside.

>
> Ah. OK.
>
> Your question should really be "what data structure should
> I use to..."
>
> It's not really a perl question at all, it's
> really an algorithm and data structure question.


Well no, some languages have sparse arrays and some don't. So this is
definitely a language-specific question, the data structure is pretty
obvious.

For Perl 5, you want either a relevant CPAN module or just use hashes
that combine x and y uniquely. See
http://search.cpan.org/search?query=sparse&mode=all for some ideas.

Ted
 
Reply With Quote
 
xhoster@gmail.com
Guest
Posts: n/a
 
      10-30-2006
(E-Mail Removed) wrote:
> Hi.
>
> Thank you very much for your quick answers. I try to handle data from a
> 450x450 meter surface.
>
> I've got 100,000 elements distributed on this surface, and each
> square millimeter can hold one or more elements. So the array to create
> is a sparse one: most of square millimeters holds zero elements, and
> the rest, 20 or less.
>
> I'm trying to find the square meter with more elements inside.


So then, I assume the square meters' must be rotationally aligned parallel
to the axes of the millimeter scale. But can one square meter "slide" by
a millimeter in order to make another square meter, or does it have to
slide by a full meter? I.e. can the potential square meters overlap each
other?

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
 
Reply With Quote
 
anno4000@radom.zrz.tu-berlin.de
Guest
Posts: n/a
 
      10-30-2006
bugbear <bugbear@trim_papermule.co.uk_trim> wrote in comp.lang.perl.misc:
> (E-Mail Removed) wrote:
> > Hi.
> >
> > Thank you very much for your quick answers. I try to handle data from a
> > 450x450 meter surface.
> >
> > I've got 100,000 elements distributed on this surface, and each
> > square millimeter can hold one or more elements. So the array to create
> > is a sparse one: most of square millimeters holds zero elements, and
> > the rest, 20 or less.
> >
> > I'm trying to find the square meter with more elements inside.


"The square meter" as in, divide the surface into meter-squares like
a checkerboard and find the one with the most elements? Or as in,
find the one-meter square that holds the most elements? The first
ones must start on integer meter-coordinates, the other can start
anywhere. The second search space is much larger.

> Ah. OK.
>
> Your question should really be "what data structure should
> I use to..."
>
> It's not really a perl question at all, it's
> really an algorithm and data structure question.


There is, however, a likely Perl answer: Use a hash. It is often
the most straight-forward way to represent a sparse anything.

Here one could use blank-separated integer coordinates (in mm) for
keys and the (nonzero) number of elements for values. Values of
zero are *not* entered. Some care may be needed not to autovivify
these non-entries.

Anno
 
Reply With Quote
 
Peter Scott
Guest
Posts: n/a
 
      10-31-2006
On Mon, 30 Oct 2006 05:10:07 -0800, francescomoi wrote:
> I'd like to create a program to handle data (natural numbers)
> within a 450,000 x 450,000 array. I'm considering Perl, but
> I don't know is it's powerful enough to manage it.


Any time you want to work with a matrix of lots of numbers in Perl,
consider http://pdl.perl.org/.

--
Peter Scott
http://www.perlmedic.com/
http://www.perldebugged.com/

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
signal handling and (structured) exception handling Peter C++ 34 10-17-2009 10:03 AM
python list handling and Lisp list handling Mark Tarver Python 22 04-26-2009 09:36 PM
Is faster handling hexadecimal values than handling chars? IƱaki Baz Castillo Ruby 1 04-15-2008 09:04 AM
array handling in perl Folkert van Heusden Perl 0 02-13-2007 02:32 PM
Perl Help - Windows Perl script accessing a Unix perl Script dpackwood Perl 3 09-30-2003 02:56 AM



Advertisments