Extracting a region of interest from an image file without reading the entire image

Question

I am searching for a library (in any language) that is capable of reading a region of an image file (any format) without having to initially read that entire image file.

I have come across a few options such as vips, which does indeed not keep the entire image in memory, but still seems to need to read it entirely to begin with.

I realize this may not be available for compressed formats such as jpegs, but in theory it sounds like bmps or tiffs should allow for this type of reading.

Rayraya

You'll have to be more specific. Most image formats are compressed. Otherwise the images are huge and storing them is a waste of space. You could always write custom file mapping reads into specific locations, though this may not be faster or less memory consuming than reading most of the image at least once. – Morley

The images are stored on a NAS in a different location, so space will not be an issue. The problem is that out of >100 megapixels an image can have, only a subsection of that (for which we already know the coords) will be used, and it's the extra bandwidth needed to read the unused portion of the file that becomes the performance bottleneck. AFAIK bmp is uncompressed and tiff CAN be uncompressed as well, this is why i'm curious if there are any existing implementations of this reading method. – Rayraya

Does a NAS allow random reads of files ? – Divisible

Accessing a sub-region in raw arrays is pretty trivial. start reading at the beginning of each ROI row. You will have to balance this with the reduced bandwidth of compressed images. – Morley

The NAS will be NFS-based so we're expecting it, in theory, to allow random access. – Rayraya

@Adi: It does sound trivial, i know, what's why i'd like to know if there's already an implemented library that accomplishes this before i start looking into implementing my own. – Rayraya

No offence to dogs, but NFS is generally dog-slow. Are you using a standard server running NFS or some sort of high-speed appliance? Can you ssh into the NFS server, for example, and use vips to do the extraction there? – Shashaban

If you already know the coordinates, you must presumably also know the filenames you are going to be needing, so maybe you could cache them to your local machine ahead of time so that bandwidth is not an issue. – Shashaban

Have you abandoned this question? You don't appear to be commenting on anyone's answers, or thanking them or clarifying much. – Shashaban

this question isn't abandoned :) i'm currently investigating a few solutions, i will definitely update this with my findings – Rayraya

libvips does what you need, I think. I've added an answer with some timings. – Cassondra

You can't (in practice) read a random part of a regular TIFF image, even though they are organised internally as a set of strips. The two bigs problems are that 1) strips can be any size up to (and larger than) the image itself, so caching is very hard, and 2) random access via strips will have catastrophic performance for operations like 90 degree rotate -- to write one strip of output, you'd need to read every strip of input! To rotate a whole file, you'd have to read it many, many times. – Cassondra

Answer 1

libvips will read just the part you need, when it can. For example, if you crop 100x100 pixels from the top-left of a large PNG, it's fast:

$ time vips crop wtc.png x.jpg 0 0 100 100
real    0m0.063s
user    0m0.041s
sys 0m0.023s

(the four numbers are left, top, width, height of the area to be cropped from wtc.png and written to x.jpg)

But a 100x100 pixel region from near the bottom is rather slow, since it has to read and decompress the pixels before the pixels you want to get to the right point in the file:

$ time vips crop wtc.png x.jpg 0 9000 100 100
real    0m3.063s
user    0m2.884s
sys 0m0.181s

JPG and strip TIFF work in the same way, though it's less obvious since they are much faster formats.

Some formats support true random-access read. For example, tiled TIFF is fast everywhere, since libvips can use libtiff to read only the tiles it needs:

$ vips copy wtc.png wtc.tif[tile]
$ time vips crop wtc.tif x.jpg 0 0 100 100
real    0m0.033s
user    0m0.013s
sys 0m0.021s
$ time vips crop wtc.tif x.jpg 0 9000 100 100
real    0m0.037s
user    0m0.021s
sys 0m0.017s

OpenSlide, vips, tiled OpenEXR, FITS, binary PPM/PGM/PBM, HDR, RAW, Analyze, Matlab and probably some others all support true random access like this.

If you're interested in more detail, there's a chapter in the API docs describing how libvips opens a file:

http://libvips.github.io/libvips/API/current/How-it-opens-files.md.html

Here's crop plus save in Python using pyvips:

import pyvips

image = pyvips.Image.new_from_file(input_filename, access='sequential')
tile = image.crop(left, top, width, height)
tile.write_to_file(output_filename)

The access= is a flag that hints to libvips that it's OK to stream this image, in case the underlying file format does not support random access. You don't need this for formats that do support random access, like tiled TIFF.

You don't need to write to a file. For example, this will make a buffer object containing the file encoded as a JPG:

buffer = tile.write_to_buffer('.jpg', Q=85)

Or this will write directly to stdout:

target = pyvips.Target.new_from_descriptor(0)
tile.write_to_target('.jpg', Q=85)

The Q=85 is an optional argument to set the JPG Q factor. You can set any of the file save options.

Cassondra

This seems to be the best option yet. The answer from Cris Luengo with extracting specific tiles from a tiff without reading the whole file was useful, however this addresses extracting a truly custom ROI while adding minimal coding overhead to the problem! – Rayraya

I'd use the python interface pypi.python.org/pypi/pyvips you should get good performance and it makes coding very simple too. Ask on the pyvips issue tracker if you have any questions github.com/jcupitt/pyvips/issues – Cassondra

Is there a way to make vips write on stdout rather than to a disk-based file such as x.jpg above? – Shashaban

Yes, just use the format extension, eg. vips crop x.png .jpg 0 0 10 10 | cat > x.jpg will crop and write to stdout as a JPG. – Cassondra

Answer 2

ITK can do it with some formats. There is a method CanStreamRead which returns true for formats which support streaming, such as MetaImageIO. An example can be found here. You can ask more detailed questions on ITK's forum.

Electrophone

Answer 3

If have control over the file format, I would suggest you use tiled TIFF files. These are typically used in digital pathology whole slide images, with average sizes of 100kx30k pixels or so.

LibTiff makes it easy to read the tiles corresponding to a selected ROI. Tiles can be compressed without making it less efficient to read a small region (no need to decode whole scan lines).

Yeorgi

Using libtiff to extract tiles has been tested and seems to work as intended (ie good performance extracting a tiles from a large tiff over a low bandwidth connection). I'm still looking a bit at the other solutions as well, but this definitely solves the issue :) – Rayraya

@andrei, indeed, this is exactly what the tiled TIFF format was designed for! – Yeorgi

Answer 4

The BMP format (uncompressed) is simple enough that you can write the function yourself.

TIFF is a little less easy, as there are so many subformats. But the TIFF library (TIFFlib) supports a "tile-oriented" I/O mode. http://www.libtiff.org/libtiff.html#Tiles

Divisible

That's an outdated site for LibTiff. Use simplesystems.org/libtiff instead. – Yeorgi

Answer 5

I don't know of such a library solution.
Low level, file-read access is format specific and in particular, file mapping is OS specific.

If you have access to the raw bytes then assuming you know the width, height, depth and number of channels etc. then calculating file offsets is trivial so just roll your own.

If you're transferring the extracted data over a network you might consider compressing the extracted ROI in-memory if it's relatively big before sending it over the network.

Morley