«

Hi All,

I've got one idea I'm currently trying to get this code to better use
its int arrays, but as written below the system runs out of memory
after about 6 hours of crunching. The code below is in a loop called
hundreds of thousands of times, and I'm 97% sure the imgdata int array
is not getting properly cleaned up. While I'm waiting to see if my
idea fixed the issue I thought I'd ask if anyone here has a good idea.

This code splits up a large image into smaller images and saves each
subimage. The problem code is designed to inspect every 3rd pixel of
an image and if they are all the same, we assume the image is just a
solid color so instead of saving the entire image, we check a cache of
1x1 images for a 1x1 image of the matching color or else we scale it
to 1x1, put it into the cache and in either case and we set it up to
save the 1x1 image further below. When I commented out the code that
states "//begin area of code that is the issue" the system did not run
out of memory. Anyway, an alternate question is: Is there a better way
to detect if the image is just a solid color?

Thanks in advance for your help!

BufferedImage subImage;
ByteArrayOutputStream baos;
int[] imgdata; //variable that is the issue
for(int i = 0; i < w.tcnt; i++) {
for (int j = 0; j < w.tcnt; j++) {
//begin area of code that is the issue
subImage = w.img.getSubimage(i * tilesize, j * tilesize, tilesize,
tilesize);
imgdata =
((DataBufferInt)subImage.getRaster().getDataBuffer ()).getData();
if(imgdata.length>0) {
int firstpixel = imgdata[0];
int length = imgdata.length;
while((length=length-3)>0) {
if(firstpixel!=imgdata[length]) {
break;
}
}
if(length<4) {
if(images1x1.containsKey(firstpixel)) {
subImage = images1x1.get(firstpixel);
}
else {
subImage = subImage.getSubimage(0,0,1,1);
images1x1.put(firstpixel, subImage);
}
}
}
//end code that is the issue
baos = new ByteArrayOutputStream();
try {
ImageIO.write(subImage, "PNG", baos);
//internal call to save the image
} catch (IOException e) {
System.err.println("Error converting tile!");
}
}
}
subImage = null;
baos = null;
imgdata = null;
w.img = null;

jw wrote:
> Hi All,
>
> I've got one idea I'm currently trying to get this code to better use
> its int arrays, but as written below the system runs out of memory
> after about 6 hours of crunching. The code below is in a loop called
> hundreds of thousands of times, and I'm 97% sure the imgdata int array
> is not getting properly cleaned up. While I'm waiting to see if my
> idea fixed the issue I thought I'd ask if anyone here has a good idea.
>
> This code splits up a large image into smaller images and saves each
> subimage. The problem code is designed to inspect every 3rd pixel of
> an image and if they are all the same, we assume the image is just a
> solid color so instead of saving the entire image, we check a cache of
> 1x1 images for a 1x1 image of the matching color or else we scale it
> to 1x1, put it into the cache and in either case and we set it up to
> save the 1x1 image further below. When I commented out the code that
> states "//begin area of code that is the issue" the system did not run
> out of memory. Anyway, an alternate question is: Is there a better way
> to detect if the image is just a solid color?
>
> Thanks in advance for your help!
>
> BufferedImage subImage;
> ByteArrayOutputStream baos;
> int[] imgdata; //variable that is the issue
> for(int i = 0; i < w.tcnt; i++) {
> for (int j = 0; j < w.tcnt; j++) {
> //begin area of code that is the issue
> subImage = w.img.getSubimage(i * tilesize, j * tilesize, tilesize,
> tilesize);
> imgdata =
> ((DataBufferInt)subImage.getRaster().getDataBuffer ()).getData();
> if(imgdata.length>0) {
> int firstpixel = imgdata[0];
> int length = imgdata.length;
> while((length=length-3)>0) {
> if(firstpixel!=imgdata[length]) {
> break;
> }
> }
> if(length<4) {
> if(images1x1.containsKey(firstpixel)) {
> subImage = images1x1.get(firstpixel);
> }
> else {
> subImage = subImage.getSubimage(0,0,1,1);
> images1x1.put(firstpixel, subImage);

I guess images1x1 is a HashMap (defined with a suitably large initial
size). As firstpixel is an int and int has around 4 billion possible
values this object could conceivably grow quite large.

Maybe you could check (assert?)
- that the value of firstpixel is within the range you expect.
- that images1x1.size() is less than some reasonable limit.


> }
> }
> }
> //end code that is the issue
> baos = new ByteArrayOutputStream();
> try {
> ImageIO.write(subImage, "PNG", baos);
> //internal call to save the image
> } catch (IOException e) {
> System.err.println("Error converting tile!");
> }
> }
> }
> subImage = null;
> baos = null;
> imgdata = null;
> w.img = null;


--
RGB

Thanks for the suggestion RGB, but I know because of the images I'm
working with there are only about 7-8 possible solid color images.

Anyway, I first tried moved the declaration of the int array up,
hoping that would force the system to reuse teh same memory area each
time through the loop, but that didn't make a difference.

Then I made two other changes, and I'm not sure which was the actual
fix. I explicitly called System.gc() every 100th time through the
loop. Secondly I made a more fundamental change to the algorithm. I
switched from getting the BufferedImage's int array to repeatedly
using the BufferedImage's getRGB(x,y) method to get a single pixel of
info. When I wrote this before, I don't know if that method couldn't
be used for some reason (different image type for example) or if I
thought accessing the int array would be faster, or I just didn't know
about the method. Anyway, I didn't test these separately, so I don't
know which one made a difference, but I did want to post a follow up
for anyone interested or for future reference.

Forgot to mention I did echo out the size of the Hashmap as suggested
and it never grew larger than 8 objects, just to be sure.

bugbear wrote:
> jw wrote:
>>

>> thought accessing the int array would be faster, or I just didn't know
>> about the method. Anyway, I didn't test these separately, so I don't
>> know which one made a difference.
>
> assuming it's not too hard, it would be helpful if you did,
> to people searching the USENET archive for similar
> issues.

Yes, and post an SSCCE of both. This is supposed to be a community
helping each other, not a tech support line. If you don't post up your
solutions, how the can the rest of us benefit from your discoveries? It
puts you in the position of only asking for help and never giving any.

I'd like to see a SSCCE of both myself, so I can understand better what
the original problem was....