How to handle huge data/images in RAM in Java?

Question

Summary

1. I am reading a large binary file which contains image data.
2. Cumulative Count Cut analysis is performed on data [It requires another array with same size as the image].
3. The data is stretched between 0 to 255 stored in BufferedImage pixel by pixel, to draw the image on JPanel.
4. On this image, zooming is performed using AffineTransform.

Problems

1. Small Image(<.5GB)

   1.1 When I am increasing the scale factor for performing zooming, after a
   point exception is thrown:-

java.lang.OutOfMemoryError: Java heap space.

Below is the code used for zooming-

    scaled = new BufferedImage(width, height, BufferedImage.TYPE_BYTE_GRAY);
    Graphics2D g2d = (Graphics2D)scaled.createGraphics();
    AffineTransform transformer = new AffineTransform();
    transformer.scale(scaleFactor, scaleFactor); 
    g2d.setTransform(transformer);

2. Large Image(>1.5GB)
   • While loading a huge image(>1.5GB), same exception occurs as appeared in 1.1, even is the image is small enough to be loaded, sometimes, I get the same error.

Solutions Tried

1. I tried using BigBufferedImage in place of BufferedImage to store the stretched data. BigBufferedImage image = BigBufferedImage.create(newCol,newRow, BufferedImage.TYPE_INT_ARGB);

2. But it couldn't be passed to g2d.drawImage(image, 0, 0, this); because the repaint method of JPanel just stops for some reason.

3. I tried loading image in low resolution where pixel is read and few columns and rows are jumped/skipped. But the problem is how to decide what number of pixels to skip as image size varies therefore I am unable to decide how to decide the jump parameter.

    MappedByteBuffer buffer = inChannel.map(FileChannel.MapMode.READ_ONLY,0, inChannel.size());
    buffer.order(ByteOrder.LITTLE_ENDIAN);
    FloatBuffer floatBuffer = buffer.asFloatBuffer();
    for(int i=0,k=0;i<nrow;i=i+jump)  /*jump is the value to be skipped, nrow is height of image*/
    {
        for(int j=0,l=0;j<ncol1;j=j+jump)   //ncol is width of image
        {
                index=(i*ncol)+j;
                oneDimArray[(k*ncolLessRes)+l] = floatBuffer.get(index);//oneDimArray is initialised to size of Low Resolution image.
                l++;
        }
        k++;
    }

The problem is to decide how many column and row to skip i.e what value of jump should be set.

4. I tried setting Xmx but image size varies and we cannot dynamically set the Xmx values. Here are some values -

table, th, td {
  border: 1px solid black;
}

<table style="width:100%">
  <tr>
    <th>Image Size</th>
    <th>Xmx</th>
    <th>Xms</th>
    <th>Problem</th>
  </tr>
  <tr>
    <td>83Mb</td>
    <td>512m</td>
    <td>256m</td>
    <td>working</td>
  </tr>
  <tr>
    <td>83Mb</td>
    <td>3096m</td>
    <td>2048m</td>
    <td>System hanged</td>
  </tr>
   <tr>
    <td>3.84Gb</td>
    <td>512m</td>
    <td>256m</td>
    <td>java.lang.OutOfMemoryError: Java heap space
  </tr>
  <tr>
    <td>3.84Gb</td>
    <td>3096m</td>
    <td>512m</td>
    <td>java.lang.OutOfMemoryError: Java heap space
  </tr>
</table>

5. For this I tried finding memory allocated to program:-

 try(BufferedWriter bw= new BufferedWriter(new FileWriter(dtaFile,true))){
    Runtime runtime=Runtime.getRuntime();
    runtime.gc();
    double oneMB=Math.pow(2,20);
    long[] arr= Instream.range(0,(int)(10.432*long.BYTES*Math.pow(2,20))).asLongStream().toArray();
    runtime.gc();
    long freeMemory= runtime.freeMemory();
    long totalMemory= runtime.totalMemory();
    long usedMemory= totalMemory-freeMemory;
    long maxMemory= runtime.maxMemory();
    String fileLine= String.format(" %9.3f  %9.3f   %9.3f " , usedMemory/oneMb, freeMemory/oneMB, totalMemory/oneMb, maxMemory/oneMB);
    bw.write();
}

Following results were obtained
Memory Allocation
This approach failed because the available memory increases as per usage of my code. As a result it will not be useful for me to make a decision for jump.

Result Expected

A way to access the amount of available memory before the loading of the image so that I could use it to make decision on value of the jump. Is there any other alternative to decide jump value (i.e., how much I can lower the resolution?).

Exterior

Please add to your question what values of Xmx you tried. – Oyer

4. How about setting Xmx to arbitralily large number (100 GB) In such case JVM will simply die when system can't provide. With oracle JVM method to obtain system memory is here #5512878. However if you don't need entire image in memory (it's raw I guess?) why not simply read it line by line (or pixel by pixel). CCC does not need entire image in memory IIRC – Sagesagebrush

What if you need the entire image in the memory. To paint it using Graphics on a Panel, you need the entire Image, right? – Edana

@Ellen Spertus Using manually set Xmx values means, you can't have arbitrarily large images as mentioned by above. Also, can we set the Xmx values after starting the JVM? What if there is a scenario where we can't restart the JVM? – Edana

@EllenSpertus I have added the values in the quesion. – Exterior

Answer 1

You can read the specific portion of an image, then scale it with reduced resolution for display purpose.

So in your case you can read the image in chunk (read image portions just like we read the data from db row by row)

For example:

// Define the portion / row size 50px or 100px
int rowHeight = 50;
int rowsToScan = imageHeight / rowHeight;
if(imageHeight % rowHeight > 0) rowsToScan++;

int x = 0;
int y = 0;
int w = imageWidth;
int h = rowHeight;

ArrayList<BufferedImage> scaledImagePortions = new ArrayList<>();

for(int i = 1; i <= rowsToScan; i++) {
    // Read the portion of an image scale it
    // and push the scaled version in lets say array
    BufferedImage scalledPortionOfImage = this.getScaledPortionOfImage(img, x, y, w, h);
    scaledImagePortions.add(scalledPortionOfImage);

    y = (rowHeight * i);
}

// Create single image out of scaled images portions

Thread which can help you to get portion of an image Read region from very large image file in Java

Thread which can help you to scale the image (my quick search result :) ) how to resize Image in java?

Thread which can help you in merging the buffered images: Merging two images

You can always tweak the snippets :)

Dextran

Answer 2

1. OutOfMemoryError that is self explainatory - you are out of memory. That is beeing said not physical RAM you have on your machine, but rather JVM hits upper memory allocation limit set by -xmx setting
2. Your xmx setting testing makes little sense as you try to put 3,8GB size of an image into 512MB memory block. It cannot work - you cannot put 10 liters of water in 5 liters bottle. For memory usage you need at least the size of image x3 as you are storing every pixel separately and that contains of 3 bytes (RGB). And that is just for pure image data. What is left is whole app and data object structure overhead + additional space required for computation and probably plenty more that I didn't mention and I am not even aware of.
3. You don't want to "dynamicly set" -xmx. Set it to maximum possible value in your system (trial and error). JVM will not take that much of memory unless it will need it. By additional -X settings you can tell JVM to free up unused memory so you don't have to worry about unused memory beeing "freezed" by JVM.
4. I never worked on image processing applications. Is Photoshop or Gimp is capable of opening and doing something usefull with such big images? Maybe you should looks for clues about processing that much of data there (if it is working)
5. If point above is just a naive as you need this for scientific purposes (and that is not what Photoshop or Gimp are made for unless you are flatearther :) ), you will need scientific grade hardware.
6. One thing that comes into my mind, is not to read image into memory at all but process it on the fly. This could reduce memory consumption to order of megabytes.

Take a closer look into ImageReader API as it suggest (readTile method) it might be possible to read only area of image (eg for zooming in)

Consuelaconsuelo