Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > java.lang.OutOfMemoryError: Java heap space

Reply
Thread Tools

java.lang.OutOfMemoryError: Java heap space

 
 
Goofball
Guest
Posts: n/a
 
      10-29-2006
Does anyone have any ideas on why that happened:

[java] Exception in thread "main" java.lang.OutOfMemoryError: Java heap
space

Explaining the situation. I have such code:

for (int i=0; i<10000-1; i++) {
for (int j=i+1; j<10000; j++) {
compareElements(arr[i], arr[j]);
}
}

Element of the array is a simple class that has two fields (one is
string and another is BitVector) and getters and setters for those
fields. Function compareElements compares the BitVectors of two
elements.

Now the problem: when I run the code, program runs only till then 1334
element in the first cycle. Then I receive the above exception.

Does anyone know, what can that be? Thanks for any help.

 
Reply With Quote
 
 
 
 
Brandon McCombs
Guest
Posts: n/a
 
      10-29-2006
Goofball wrote:
> Does anyone have any ideas on why that happened:
>
> [java] Exception in thread "main" java.lang.OutOfMemoryError: Java heap
> space
>
> Explaining the situation. I have such code:
>
> for (int i=0; i<10000-1; i++) {
> for (int j=i+1; j<10000; j++) {
> compareElements(arr[i], arr[j]);
> }
> }
>
> Element of the array is a simple class that has two fields (one is
> string and another is BitVector) and getters and setters for those
> fields. Function compareElements compares the BitVectors of two
> elements.
>
> Now the problem: when I run the code, program runs only till then 1334
> element in the first cycle. Then I receive the above exception.
>
> Does anyone know, what can that be? Thanks for any help.
>


Yeah, your JVM is allocated a certain amount of memory from the OS and
you are creating so many objects at a certain size that your JVM is
running out of memory, hence the OutOfMemoryError exception. You can
tune the size of the memory for the JVM by passing in various parameters
to the java command. I believe they look similar to -X256m or something
like that. You could also think about how your programming is allocating
the data and possibly restructure your algorithm or, if possibly, when
you are done working with the data then get rid of it so the GC can
reclaim the memory.
 
Reply With Quote
 
 
 
 
Daniel Pitts
Guest
Posts: n/a
 
      10-30-2006

Goofball wrote:
> Does anyone have any ideas on why that happened:
>
> [java] Exception in thread "main" java.lang.OutOfMemoryError: Java heap
> space
>
> Explaining the situation. I have such code:
>
> for (int i=0; i<10000-1; i++) {
> for (int j=i+1; j<10000; j++) {
> compareElements(arr[i], arr[j]);
> }
> }
>
> Element of the array is a simple class that has two fields (one is
> string and another is BitVector) and getters and setters for those
> fields. Function compareElements compares the BitVectors of two
> elements.
>
> Now the problem: when I run the code, program runs only till then 1334
> element in the first cycle. Then I receive the above exception.
>
> Does anyone know, what can that be? Thanks for any help.


If you post an SSCCE (http://www.physci.org/codes/sscce/), we would be
better able to help you.

 
Reply With Quote
 
Goofball
Guest
Posts: n/a
 
      10-30-2006
I was experimenting with gc() in different ways. I was callling it
after processing 1000 elements. However, that didn't help and the
problem remained the same: after processing 1334 (or 1335) elements I
received the OutOfMemory error.

Brandon McCombs wrote:
> Goofball wrote:
> > Does anyone have any ideas on why that happened:
> >
> > [java] Exception in thread "main" java.lang.OutOfMemoryError: Java heap
> > space
> >
> > Explaining the situation. I have such code:
> >
> > for (int i=0; i<10000-1; i++) {
> > for (int j=i+1; j<10000; j++) {
> > compareElements(arr[i], arr[j]);
> > }
> > }
> >
> > Element of the array is a simple class that has two fields (one is
> > string and another is BitVector) and getters and setters for those
> > fields. Function compareElements compares the BitVectors of two
> > elements.
> >
> > Now the problem: when I run the code, program runs only till then 1334
> > element in the first cycle. Then I receive the above exception.
> >
> > Does anyone know, what can that be? Thanks for any help.
> >

>
> Yeah, your JVM is allocated a certain amount of memory from the OS and
> you are creating so many objects at a certain size that your JVM is
> running out of memory, hence the OutOfMemoryError exception. You can
> tune the size of the memory for the JVM by passing in various parameters
> to the java command. I believe they look similar to -X256m or something
> like that. You could also think about how your programming is allocating
> the data and possibly restructure your algorithm or, if possibly, when
> you are done working with the data then get rid of it so the GC can
> reclaim the memory.


 
Reply With Quote
 
Goofball
Guest
Posts: n/a
 
      10-30-2006
Don't know, how much is that an SSCCE, but I am posting you my example:

import java.util.ArrayList;
import java.util.BitSet;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class Example {

private class ArrElem {

private String id;
private BitSet fp;

public BitSet getFp() { return fp; }
public void setFp(BitSet fp) { this.fp = fp; }
public String getId() { return id; }
public void setId(String id) { this.id = id;}
}

private List<ArrElem> arr = new ArrayList<ArrElem>();

private Map<ArrElem, Collection<ArrElem>> rez = new HashMap<ArrElem,
Collection<ArrElem>>();

public void processArr() {
//fillArr(10000);
for (int i = 0; i < 10000-1; i++) {
for (int j = i+1; j < 10000; j++) {
compareElements(arr.get(i), arr.get(j));
}
}
}

private void compareElements(ArrElem elem1, ArrElem elem2) {
int bitM = elem1.getFp().cardinality();
int bitN = elem2.getFp().cardinality();

BitSet commonBits = elem1.getFp();
commonBits.and(elem2.getFp());
int bitCommon = commonBits.cardinality();

double index = bitCommon/(bitM+bitN+bitCommon);
if (index < 0. {
if (! rez.containsKey(elem1))
rez.put(elem1, new ArrayList<ArrElem>());
rez.get(elem1).add(elem2);
}
}
}


Daniel Pitts wrote:
> Goofball wrote:
> > Does anyone have any ideas on why that happened:
> >
> > [java] Exception in thread "main" java.lang.OutOfMemoryError: Java heap
> > space
> >
> > Explaining the situation. I have such code:
> >
> > for (int i=0; i<10000-1; i++) {
> > for (int j=i+1; j<10000; j++) {
> > compareElements(arr[i], arr[j]);
> > }
> > }
> >
> > Element of the array is a simple class that has two fields (one is
> > string and another is BitVector) and getters and setters for those
> > fields. Function compareElements compares the BitVectors of two
> > elements.
> >
> > Now the problem: when I run the code, program runs only till then 1334
> > element in the first cycle. Then I receive the above exception.
> >
> > Does anyone know, what can that be? Thanks for any help.

>
> If you post an SSCCE (http://www.physci.org/codes/sscce/), we would be
> better able to help you.


 
Reply With Quote
 
Ingo Menger
Guest
Posts: n/a
 
      10-30-2006

Goofball wrote:

> private void compareElements(ArrElem elem1, ArrElem elem2) {
> int bitM = elem1.getFp().cardinality();
> int bitN = elem2.getFp().cardinality();
>
> BitSet commonBits = elem1.getFp();
> commonBits.and(elem2.getFp());
> int bitCommon = commonBits.cardinality();
>
> double index = bitCommon/(bitM+bitN+bitCommon);
> if (index < 0. {
> if (! rez.containsKey(elem1))
> rez.put(elem1, new ArrayList<ArrElem>());
> rez.get(elem1).add(elem2);
> }
> }


There you have it.
The index will be 0.0 unless bitM and bitN are both 0. So you end up
creating at least a new ArrayList element in each cycle.

 
Reply With Quote
 
Red Orchid
Guest
Posts: n/a
 
      10-30-2006
"Goofball" <(E-Mail Removed)> wrote or quoted in
Message-ID: <(E-Mail Removed) .com>:

> ---------- snip -------------
>
> private List<ArrElem> arr = new ArrayList<ArrElem>();
>
> private Map<ArrElem, Collection<ArrElem>> rez = new HashMap<ArrElem,
> Collection<ArrElem>>();
>
> public void processArr() {
> //fillArr(10000);
> for (int i = 0; i < 10000-1; i++) {
> for (int j = i+1; j < 10000; j++) {
> compareElements(arr.get(i), arr.get(j));
> }
> }
> }
>



It is mere guesswork.


The above code calls 'compareElements' (10000 * 10000) times.
Namely, 100 M times.





> private void compareElements(ArrElem elem1, ArrElem elem2) {
> int bitM = elem1.getFp().cardinality();
> int bitN = elem2.getFp().cardinality();
>
> BitSet commonBits = elem1.getFp();
> commonBits.and(elem2.getFp());
> int bitCommon = commonBits.cardinality();
>
> double index = bitCommon/(bitM+bitN+bitCommon);
>




Min of 'index' is 0.
Max of 'index' is smaller than 1.



>
> if (index < 0. {
>



If 'cardinality()' is random,
the probability of calling the bellow code will be about 8/10.

That is, 80 M times (10000 * 10000 * 0.




> if (! rez.containsKey(elem1))
> rez.put(elem1, new ArrayList<ArrElem>());
> rez.get(elem1).add(elem2);
> }
> }
> }
>
>



The above code adds an element to 'rez' whenever she is executed.
A total of 80 M elements.

320 MBytes memory (80M * 4) will be required at least.







 
Reply With Quote
 
Ingo Menger
Guest
Posts: n/a
 
      10-30-2006

Red Orchid wrote:

> > double index = bitCommon/(bitM+bitN+bitCommon);
> >

> Min of 'index' is 0.
> Max of 'index' is smaller than 1.


Yes, it's 0!
Proof:
case 1) bitM and bitM are 0. Then so will bitComnmon. This will result
in zero divide. (BTW, this is a programming error)
case 2) bitM or bitM are > 0. Then the integer division will result in
0.

Thus, index will never be anything else than 0.0. And the programm will
crash occasionally.

> > if (index < 0. {


Thus, this is always true.

> >

>
>
> If 'cardinality()' is random,
> the probability of calling the bellow code will be about 8/10.


1, to be exact.


> The above code adds an element to 'rez' whenever she is executed.
> A total of 80 M elements.


It adds an element everytime. A total of 100M elements.

 
Reply With Quote
 
Dijon Yu
Guest
Posts: n/a
 
      11-01-2006

Goofball wrote:
> I was experimenting with gc() in different ways. I was callling it
> after processing 1000 elements. However, that didn't help and the
> problem remained the same: after processing 1334 (or 1335) elements I
> received the OutOfMemory error.


I have meet the problem too.
Don't depend on gc(), whatever 。
If you want resolve the problem, you can set the JVM.

You can use java -Xmx256m to avoid the problem.
But I think maybe you can refactor and redesign the algorithm.

 
Reply With Quote
 
Goofball
Guest
Posts: n/a
 
      11-08-2006

Thanks a lot. That helped. I redesigned the alsorithm. Now it works
fine. But when I test it on about 1 million of elements - it crashes.
Starting JVM with more memory helps, but that sometimes cannot be done
(because of the requirements). So, I decided to write my own
implementations of List and Map interfaces, that will flush the results
on disk. My requirements allow that.
Thanks again.

Ingo Menger wrote:
> Red Orchid wrote:
>
> > > double index = bitCommon/(bitM+bitN+bitCommon);
> > >

> > Min of 'index' is 0.
> > Max of 'index' is smaller than 1.

>
> Yes, it's 0!
> Proof:
> case 1) bitM and bitM are 0. Then so will bitComnmon. This will result
> in zero divide. (BTW, this is a programming error)
> case 2) bitM or bitM are > 0. Then the integer division will result in
> 0.
>
> Thus, index will never be anything else than 0.0. And the programm will
> crash occasionally.
>
> > > if (index < 0. {

>
> Thus, this is always true.
>
> > >

> >
> >
> > If 'cardinality()' is random,
> > the probability of calling the bellow code will be about 8/10.

>
> 1, to be exact.
>
>
> > The above code adds an element to 'rez' whenever she is executed.
> > A total of 80 M elements.

>
> It adds an element everytime. A total of 100M elements.


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
How to increase stack space/heap space Ajay C Programming 9 05-14-2006 12:50 AM
java.lang.OutOfMemoryError: Java heap space timasmith@hotmail.com Java 2 05-08-2006 03:35 AM
Why Python style guide (PEP-8) says 4 space indents instead of 8 space??? 8 space indents ever ok?? Christian Seberino Python 21 10-27-2003 04:20 PM
Re: Why Python style guide (PEP-8) says 4 space indents instead of8 space??? 8 space indents ever ok?? Ian Bicking Python 2 10-23-2003 07:07 AM
Stack space, global space, heap space Shuo Xiang C Programming 10 07-11-2003 07:30 PM



Advertisments