Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > size limits of arrays/vectors ?

Reply
Thread Tools

size limits of arrays/vectors ?

 
 
William Zumwalt
Guest
Posts: n/a
 
      04-03-2004

Hey all,

I'm writing a science application that was previously written in Fortan
and it has arrays that literally had millions of elements. Fortunately
the elements were integers. I'm moving it to Java because of the GUI and
portability is a must. Can Java handle Vectors (or at least primitive
arrays) of this size? The machine certainly has the resources.
 
Reply With Quote
 
 
 
 
Lee Weiner
Guest
Posts: n/a
 
      04-03-2004
In article <(E-Mail Removed)>, William
Zumwalt <(E-Mail Removed)> wrote:
>I'm writing a science application that was previously written in Fortan
>and it has arrays that literally had millions of elements. Fortunately
>the elements were integers. I'm moving it to Java because of the GUI and
>portability is a must. Can Java handle Vectors (or at least primitive
>arrays) of this size? The machine certainly has the resources.


It took me about 90 seconds to test this out, probably less time than it took
you to post the question.

public static void main ( String[] args )
{
Vector vec = new Vector();
for( int i = 0; i < 2500000; i++ )
vec.add( new Integer( i ) );
}

This one worked. When I changed the number to 3,000,000, I got an OutOfMemory
Exception.

Lee Weiner
lee AT leeweiner DOT org
 
Reply With Quote
 
 
 
 
Roedy Green
Guest
Posts: n/a
 
      04-03-2004
On Sat, 03 Apr 2004 01:13:17 GMT, William Zumwalt
<(E-Mail Removed)> wrote or quoted :

>I'm writing a science application that was previously written in Fortan
>and it has arrays that literally had millions of elements. Fortunately
>the elements were integers. I'm moving it to Java because of the GUI and
>portability is a must. Can Java handle Vectors (or at least primitive
>arrays) of this size? The machine certainly has the resources.


Java ints are 32 bits so arrays and Vectors can hold +2,147,483,647
elements [2^31-1] aka Integer.MAX_VALUE.

You eventually bang into the addressing limit of your hardware.

Sun is offering 64 bit workstations at a discount to help you over
that hump.

--
Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
 
Reply With Quote
 
Roedy Green
Guest
Posts: n/a
 
      04-03-2004
On Sat, 03 Apr 2004 02:48:26 GMT, http://www.velocityreviews.com/forums/(E-Mail Removed) (Lee Weiner) wrote or
quoted :

>When I changed the number to 3,000,000, I got an OutOfMemory
>Exception.


which could be extended with a java.exe command line option. See
http://mindprod.com/jgloss/javaexe.html

--
Canadian Mind Products, Roedy Green.
Coaching, problem solving, economical contract programming.
See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
 
Reply With Quote
 
Chris Uppal
Guest
Posts: n/a
 
      04-03-2004
Lee Weiner wrote:

> Vector vec = new Vector();
> for( int i = 0; i < 2500000; i++ )
> vec.add( new Integer( i ) );


Argh!!! Don't even *consider* using Vectors of Integers to hold huge datasets.

Consider the memory reqs (assuming the typical 8-byte overhead for an object
header):

For an array of 2500000 ints:
2,500,000 * 4
= 10 MBytes (plus a little bit for the array header).

For a Vector (or any other collection) of 2500000 Integers:
2,500,000 * (4 + for the Integer objects.
2,500,000 * 4 for the internal array of refs to the Integers
= 50 MBytes (plus a little bit for the Vector itself).

And the difference will be even bigger on a 64-bit JVM.

That's not to mention the massive extra load that 2.5 million extra object will
put on the garbage collector.

Nor does it mention the horrible way that 2.5 million pointers will largely
destroy any cache coherence in memory accesses.

(Of course, the same observations go for Vectors of Floats, Doubles, etc).

Lee, William, I'm sorry if I'm saying things that are already obvious to you.
But I've seen several recent posts by people pulling exactly this kind of trick
(and then wondering why they had problems!), so I don't think the point can be
made strongly enough.

(And it also suggests that there's something wrong with the way that we are
teaching programming, but that's another story).

-- chris


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
xd card size limits for Fujifilm S5100 Salmonella Digital Photography 0 11-21-2004 05:24 AM
Size and concurrency limits on Jet 4.0 dB engine? Joe Van Meer ASP .Net 2 05-05-2004 09:18 PM
Re: IBM JVM for Linux limits array size to 256Mbytes Roedy Green Java 0 12-09-2003 12:59 AM
Re: J2ME jar size limits Darryl L. Pierce Java 0 08-05-2003 01:25 PM
Re: J2ME jar size limits Andreas Rueckert Java 2 08-04-2003 08:44 PM



Advertisments