What is wrong with FileInputStream.read(byte[])?

Question

In response to my answer to a file-reading question, a commenter stated that FileInputStream.read(byte[]) is "not guaranteed to fill the buffer."

File file = /* ... */  
long len = file.length();
byte[] buffer = new byte[(int)len];
FileInputStream in = new FileInputStream(file);
in.read(buffer);

(The code assumes that the file length does not exceed 2GB)

Apart from an IOException, what could cause the read method to not retrieve the entire file contents?

EDIT:

The idea of the code (and the goal of the OP of the question I answered) is to read the entire file into a chunk of memory in one swoop, that's why buffer_size = file_size.

Slapjack

if the goal is to read a whole file in one call I would suggest reusing or creating a library that does this for you, which potentially uses multiple read calls behind the scenes. – Roderic

Answer 1

Apart from an IOException, what could cause the read method to not retrieve the entire file contents?

In my own API implementation, and on my home rolled file-system I simply choose to fill half the buffer...... just kidding.

My point is that even if I wasn't kidding, technically speaking it wouldn't be a bug. It is a matter of method contract. This is the contract (documentation) in this case is:

Reads up to b.length bytes of data from this input stream into an array of bytes.

i.e., it gives no guarantees for filling the buffer.

Depending on the API implementation, and perhaps on the file-system the read method may choose not to fill the buffer. It's basically a question of what the contract of the method says.

---

Bottom line: It probably works, but is not guaranteed to work.

Blinders

+1 - it's the contract, not a particular implementation. [p] Even if FileInputStream provided a guarantee, it can be useful to work as much as possible with the InputStream abstraction, which appropriately also lacks the guarantee. – Rudd

Answer 2

what could cause the read method to not retrieve the entire file contents?

If, for example, the file is fragmented on the filesystem and the low-level implementation knows that it will have to wait for the HD to seek to the next fragment (which is something that takes a LOT of time relative to CPU operations), it would make sense for the read() call to return with part of the buffer unfilled to give the application the chance to already do something with the data it has recieved.

Now I don't know whether any implementation actually works like that, but the point is that you must not rely on the buffer being filled, because it's not guaranteed by the API contract.

Lethargy

Answer 3

Well, first off you've made yourself a false dichotomy. One perfectly normal circumstance is that the buffer won't be filled because there aren't that many bytes left in the file. That is not an IOException, but it doesn't mean the whole file's contents have not been read.

The spec says the method will either return -1 indicating end-of-stream or will block until at least one byte is read. Implementers of InputStream can optimize as they see fit (e.g. a TCP stream might return data as soon as the packet comes in regardless of the caller's choice of buffer size). A FileInputStream might fill the buffer with one block's worth of data. As the caller, you have no idea except that until the method returns -1, you need to keep on reading.

Edit

In practice, with your example, the only circumstance I would see where the buffer wouldn't be filled (with a standard implementation) is if the file changed size after you allocated the buffer but before you started reading it. Since you haven't locked the file down this is a possibility.

Roderic

But... I specifically allocated a buffer that is exactly the length of the file. If the file is 85000 bytes long, if I allocate an 85000-byte array, wouldn't the read method fill it completely? (This is for a raw binary file) – Slapjack

@Jen see my edits and sorry for not following how you allocated the buffer. It's really up to the implementation, though I would guess that good implementations would fill the whole buffer. You should never ignore the return value of read. – Roderic

Answer 4

People have talked about read on a FileInputStream as hypothetically not filling the buffer. In fact it is a reality in some circumstances:

• If you open a FileInputStream on a "/dev/tty" or a named pipe, then a read will only return you the data that is currently available. Other device files may behave the same way. (These files will probably return 0L as the file size though.)

• A FUSE file system can be implemented to not completely fill the read buffer if the file system has been mounted with the direct_io option, or a file is opened with the corresponding flag.

The above apply to Linux, but there could well be similar cases for other operating systems and/or Java implementations. The bottom line is that the javadocs allow this behavior and you can get into trouble if your application assumes that it won't occur.

There are 3rd party libraries that implement "read fully" behavior; e.g. Apache commons provides FileUtils.readFileToByteArray or IOUtils.toByteArray and similar methods. If you want / need that behavior you should use one of those libraries, or implement it yourself.

Carilla

Answer 5

It's not guaranteed to Fill the buffer.

The file size may be smaller than the buffer, or the remainder of the file may be smaller than the buffer.

Trothplight

But... I specifically allocated a buffer that is exactly the length of the file. If the file is 85000 bytes long, if I allocate an 85000-byte array, wouldn't the read method fill it completely? (This is for a raw binary file) – Slapjack

Answer 6

Your question is self-contradictory. There is no guarantee that it will read the whole buffer, even if there are no imaginable circumstances in which it won't. There is no guarantee so you can't assume it.

Farseeing