What's the best/fastest way to access image data

I have an application that performs some specialized image-processing
which is simple, but not supported by the JAI or other Java API. I
pull in data from an existing image, process it, and store it back in
a new image.

For my application, I am using the Java BufferedImage class for images
that are about 2 megapixels in size. The processing is requiring about
250 milliseconds, which isn't bad, though I my application will be
processing a LOT of images and is user-interactive, so I'd like to
trim that if I can.

Anyway, I added some more instrumentation to my time measurements and
realized that of that 250 milliseconds, 200 or so was happening in the
BufferedImage.getRGB() method that I was using to extract the raw data
from the image:

long time0 = System.currentTimeMillis();
int w = image.getWidth();
int h = image.getHeight();
int n = w * h;
int[] rgb = image.getRGB(0, 0, w, h, new int[n], 0, w);
long time1 = System.currentTimeMillis();
long accessTime = time1-time0;

I found the fact that the access took so much longer than my own
processing kind of a surprise... Ordinarily, when I see an unexpected
result like this, it's usually a indication that I'm doing the wrong
thing or using the wrong tool.

So, I was wondering if I might be using the wrong approach in pulling
out the raw data. Perhaps BufferedImage isn't even the right class to
use? I've read the API document on the many Java image classes and
find the nuances of "which one to use when" to be rather non-obvious.

Could anyone point me in the direction of the best way to do this? Is
there a web page that provides information about the design of the
image classes that might clarify this issue.

Thanks for your help.

Gary

G.W. Lucas

G.W. Lucas wrote:

> Anyway, I added some more instrumentation to my time measurements and
> realized that of that 250 milliseconds, 200 or so was happening in the
> BufferedImage.getRGB() method that I was using to extract the raw data
> int[] rgb = image.getRGB(0, 0, w, h, new int[n], 0, w);


I'm not sure, but several of the Java image APIs are asynchronous. They
return immediately and load in the background. You're getting the whole
image here so it's possible that read() is waiting on IO to complete, I
suppose.

Can you produce a complete example? I realize that you can't really
send an image, but if you could at least post the source that reproduces
this we could take a look at it.

markspace

On Nov 2, 12:37*pm, markspace <nos...@nowhere.com> wrote:

> [snip] Can you produce a complete example?


Sure. The code follows. The actual application is generating content
based on data analysis performed in background threads (behind the
scenes) and storing it in BufferedImage objects. There are a number of
different kinds of analysis routines running (some of them quite
complicated). At run time, several images may get overlaid on top of
each other to make make one composite. This, the images are declared
TYPE_INT_ARGB when the application creates them. So far, I have been
favorably impressed by Java's performance.

Anyway, you can see where I don't think that asynchronous behavior is
the issue, though I could certainly be wrong.

This snippet of code simulates the conditions under which I am
observing the performance issues. Running this on my computer, I
observed 200 millisecond access time. I set the command-line memory
option to -Xmx512m to ensure plenty of memory for the test.

Hope this helps.

Gary



import java.awt.image.BufferedImage;

/**
*
*/
public class TimeTest {

public static void main(String args[]) {
TimeTest test = new TimeTest();
for(int i=0; i<5; i++)
test.run();
}

public void run() {
BufferedImage image = new BufferedImage(
2000,
1500,
BufferedImage.TYPE_INT_ARGB);

// Diagnostic graphics operations to exercise the
// idea that some work had been done before we tried
// extracting the contents of the image. In testing,
// doesn't seem to make much difference.
//Graphics2D g2d = image.createGraphics();
//g2d.setColor(Color.ORANGE);
//g2d.drawRect(0, 0, 500, 600);
//g2d.dispose();
//try {
// Thread.sleep(1000);
//} catch (InterruptedException ex) {
//}

long time0 = System.currentTimeMillis();
int w = image.getWidth();
int h = image.getHeight();
int n = w * h;
int[] rgb = image.getRGB(0, 0, w, h, new int[n], 0, w);
long time1 = System.currentTimeMillis();
long accessTime = time1 - time0;
System.out.println("Access time " + accessTime);
System.out.flush();
}
}






> G.W. Lucas wrote:
> > Anyway, I added some more instrumentation to my time measurements and
> > realized that of that 250 milliseconds, 200 or so was happening in the
> > BufferedImage.getRGB() method that I was using to extract the raw data
> > * * * * int[] rgb = image.getRGB(0, 0, w, h, new int[n], 0, w);
>
> I'm not sure, but several of the Java image APIs are asynchronous. *They
> return immediately and load in the background. *You're getting the whole
> image here so it's possible that read() is waiting on IO to complete, I
> suppose.
>
> Can you produce a complete example? *I realize that you can't really
> send an image, but if you could at least post the source that reproduces
> this we could take a look at it.

G.W. Lucas

G.W. Lucas wrote:
> This snippet of code simulates the conditions under which I am
> observing the performance issues. Running this on my computer, I
> observed 200 millisecond access time. I set the command-line memory
> option to -Xmx512m to ensure plenty of memory for the test.

> int[] rgb = image.getRGB(0, 0, w, h, new int[n], 0, w);


This line of code calls getRBG(), which basically does this (cut and
paste from the source code itself):

for (int y = startY; y < startY+h; y++, yoff+=scansize) {
off = yoff;
for (int x = startX; x < startX+w; x++) {
rgbArray[off++] = colorModel.getRGB(
raster.getDataElements(x,y,data));
}
}

Copying data like this is never going to be fast. You want the internal
buffer itself, probably, not a copy, so your operations on the data will
be fast.

So I think the trick is to use a different constructor, so that you
already have access to the internal argb array. There's no way to just
get a pointer to it in the BufferedImage API, that I can see.

Now: are you loading data from disk? Or are you creating the data
wholesale inside your program, as you example seems to imply? The
answer is different depending on what you are doing.

markspace

In article <hcnd3b$fpk$>,
markspace <> wrote:

> G.W. Lucas wrote:
> > This snippet of code simulates the conditions under which I am
> > observing the performance issues. Running this on my computer, I
> > observed 200 millisecond access time. I set the command-line memory
> > option to -Xmx512m to ensure plenty of memory for the test.
>
> > int[] rgb = image.getRGB(0, 0, w, h, new int[n], 0, w);
>
>
> This line of code calls getRBG(), which basically does this (cut and
> paste from the source code itself):
>
> for (int y = startY; y < startY+h; y++, yoff+=scansize) {
> off = yoff;
> for (int x = startX; x < startX+w; x++) {
> rgbArray[off++] = colorModel.getRGB(
> raster.getDataElements(x,y,data));
> }
> }
>
> Copying data like this is never going to be fast. You want the
> internal buffer itself, probably, not a copy, so your operations on
> the data will be fast.
>
> So I think the trick is to use a different constructor, so that you
> already have access to the internal argb array. There's no way to
> just get a pointer to it in the BufferedImage API, that I can see.
>
> Now: are you loading data from disk? Or are you creating the data
> wholesale inside your program, as you example seems to imply? The
> answer is different depending on what you are doing.

It should be possible to operate on the Raster directly:

<http://sites.google.com/site/drjohnbmatthews/raster>

Of course, that still leaves leaves 2000 x 1500 pixels work on.

--
John B. Matthews
trashgod at gmail dot com
<http://sites.google.com/site/drjohnbmatthews>

John B. Matthews

John B. Matthews wrote:

>
> It should be possible to operate on the Raster directly:
>
> <http://sites.google.com/site/drjohnbmatthews/raster>
>
> Of course, that still leaves leaves 2000 x 1500 pixels work on.


I came up with the code below as a direct replacement for the OP's
example. However, I'm not really sure if this is what he wants or not.
It does run 10x faster than his example.


public class Main
{
public static void main( String[] args )
{
for( int i = 0; i < 5; i++ ) {
long startTime = System.nanoTime();

SinglePixelPackedSampleModel neoSPPSM =
new SinglePixelPackedSampleModel(
DataBuffer.TYPE_INT, 2000,
1500,
new int[]{0xFF0000, 0xFF00, 0xFF, 0xFF000000} );
int[] rawARGB = new int[2000 * 1500];
DataBufferInt neoDBI = new DataBufferInt( rawARGB, 2000 *
1500 );

WritableRaster neoWR =
Raster.createWritableRaster( neoSPPSM, neoDBI,
null );
DirectColorModel dcm = new DirectColorModel( 32, 0xFF0000,
0xFF00,
0xFF, 0xFF000000 );
BufferedImage image2 = new BufferedImage( dcm, neoWR,
false, null );
long endTime = System.nanoTime();
System.out.println( "Loop time (" + image2.hashCode() +
"): " +
(endTime - startTime) / 1000000 );
}
}

markspace

Thank you. That's truly impressive. Of course, now I have about six
new things to learn

Looking at your earlier post, I can see where invoking getRGB 3
million times might result in sub-optimal performance. I must say that
I'm a little surprised to see something like that in a core API. But
it explains a lot.

I modified my test program based on your example, drawing a few
primitives to the BufferedImage after it was created and inspecting
the contents of the rawARGB array to see if it changed. It worked
like a charm. And, as you say, the speed improvement is easily a
factor of 10.

I do have a question. From your code, it looks like the trick is to
supply the BufferedImage constructor with the memory that you want it
to write to so that you don't have to ask for it later on. That makes
sense. The thing I was wondering about is if the resulting
BufferedImage will have the same performance as the ones which I am
currently creating with the less-advanced constructor. The real core
function of my application is in the rendering of graphics (using
Graphics2D) to produce the raw images. The rendering involves a lot of
graphics primitives, so performance is critical. The image processing
that I am doing is really just an extra.

Naturally, I'm going to do my homework (read up on the API elements
you used and also do a lot of testing) before I commit to an approach,
but I was wondering whether I should be alert to any special tricks or
techniques as I do so.

Thanks again for your well-informed and insightful suggestion!

Gary

On Nov 2, 5:30*pm, markspace <nos...@nowhere.com> wrote:
> John B. Matthews wrote:
>
> > It should be possible to operate on the Raster directly:
>
> > <http://sites.google.com/site/drjohnbmatthews/raster>
>
> > Of course, that still leaves leaves 2000 x 1500 pixels work on.
>
> I came up with the code below as a direct replacement for the OP's
> example. *However, I'm not really sure if this is what he wants or not.
> * It does run 10x faster than his example.
>
> public class Main
> {
> * * *public static void main( String[] args )
> * * *{
> * * * * *for( int i = 0; i < 5; i++ ) {
> * * * * * * *long startTime = System.nanoTime();
>
> * * * * * * *SinglePixelPackedSampleModel neoSPPSM =
> * * * * * * * * * * *new SinglePixelPackedSampleModel(
> * * * * * * * * * * *DataBuffer.TYPE_INT, 2000,
> * * * * * * * * * * *1500,
> * * * * * * * * * * *new int[]{0xFF0000, 0xFF00, 0xFF, 0xFF000000} );
> * * * * * * *int[] rawARGB = new int[2000 * 1500];
> * * * * * * *DataBufferInt neoDBI = new DataBufferInt( rawARGB, 2000 *
> * * * * * * * * * * *1500 );
>
> * * * * * * *WritableRaster neoWR =
> * * * * * * * * * * *Raster.createWritableRaster( neoSPPSM, neoDBI,
> * * * * * * * * * * *null );
> * * * * * * *DirectColorModel dcm = new DirectColorModel( 32, 0xFF0000,
> * * * * * * * * * * *0xFF00,
> * * * * * * * * * * *0xFF, 0xFF000000 );
> * * * * * * *BufferedImage image2 = new BufferedImage( dcm, neoWR,
> * * * * * * * * * * *false, null );
> * * * * * * *long endTime = System.nanoTime();
> * * * * * * *System.out.println( "Loop time (" + image2.hashCode() +
> * * * * * * * * * * *"): *" +
> * * * * * * * * * * *(endTime - startTime) / 1000000 );
> * * * * *}
>
> }
>
>

G.W. Lucas

G.W. Lucas wrote:
> The thing I was wondering about is if the resulting
> BufferedImage will have the same performance as the ones which I am
> currently creating with the less-advanced constructor.


I honestly don't know. I believe it will have the same performance,
because I looked at the constructors for BufferedImage and related
classes, and basically just did the same thing as they do.

However, I haven't tested this yet, so you're going to be the first. If
you don't see any speed drop right away, then I'm going to guess there
there won't be any, because as I said I'm just doing the same thing that
Sun's API does.

Please do report back on your findings if you can. I'm interested if
this technique is general and will work for other folks.

markspace

On Nov 2, 8:02*pm, markspace <nos...@nowhere.com> wrote:
> G.W. Lucas wrote:
> > The thing I was wondering about is if the resulting
> > BufferedImage will have the same performance as the ones which I am
> > currently creating with the less-advanced constructor. [snip]
>
> Please do report back on your findings if you can. *I'm interested if
> this technique is general and will work for other folks.

Fortunately, I've always been interested in performance considerations
(who isn't?), so I've got plenty of timing instrumentation
already in place for my application. I was
able to run the program a dozen or so times alternating between
the different constructors. Although the sample set was pretty small,
I'm pretty sure there are no statistically significant difference
in the time required to build images (for a while there, I actually
thought the WritableRaster constructor might have an edge, but
that was just a result of a noisy test environment).

Thanks again for all your help.

g.

G.W. Lucas

In article
<8a970851-2353-4106-af7b->,
"G.W. Lucas" <> wrote:

> On Nov 2, 8:02Â*pm, markspace <nos...@nowhere.com> wrote:
> > G.W. Lucas wrote:
> > > The thing I was wondering about is if the resulting BufferedImage
> > > will have the same performance as the ones which I am currently
> > > creating with the less-advanced constructor. [snip]
> >
> > Please do report back on your findings if you can. Â*I'm interested
> > if this technique is general and will work for other folks.
>
> Fortunately, I've always been interested in performance
> considerations (who isn't?), so I've got plenty of timing
> instrumentation already in place for my application. I was able to
> run the program a dozen or so times alternating between the different
> constructors. Although the sample set was pretty small, I'm pretty
> sure there are no statistically significant difference in the time
> required to build images (for a while there, I actually thought the
> WritableRaster constructor might have an edge, but that was just a
> result of a noisy test environment).
>
>Thanks again for all your help.

Interesting; thank you for reporting your results. I missed the import
of markspace's suggestion: it was predicated on the reasonable
assumption that array access would be faster than method invocation. My
experience with mixing WritableRaster and Graphics2D operations is that
the latter tend to dominate and the former are fast enough. Still, 'it's
interesting to see how to construct a BufferedImage with one's own data
buffer.

--
John B. Matthews
trashgod at gmail dot com
<http://sites.google.com/site/drjohnbmatthews>

John B. Matthews