DataInputStream.read returning less than len

Question

I am using DataInputStream to read some bytes from a socket. I have a expected number of bytes to read from the stream (after decoding a header, I know how many bytes are in the message) It works 99% of the time but occasionally I will have the number of bytes read be less than len.

int numRead = dis.read(buffer, 0, len);

What could cause numRead to be less than len? It's not -1. I would expect the behavior of read to block until the stream is closed or EOF is reached, but if it's a socket underlying the streams this shouldn't happen unless the socket closes, right?

Is there a way of reading bytes from a socket that will always ensure that you read len bytes?

Thanks

Mcpherson

Answer 1

EDIT: For a general stream, you just keep reading until you've read everything you want to, basically. For an implementation of DataInput (such as DataInputStream) you should use readFully, as suggested by Peter Lawrey. Consider the rest of this answer to be relevant for the general case where you just have an InputStream.

It's entirely reasonable for an InputStream of any type to give you less data than you asked for, even if more is on its way. You should always code for this possibility - with the possible exception of ByteArrayInputStream. (That can still return less data than was requested, of course, if there's less data left than you asked for.)

Here's the sort of loop I'm talking about:

byte[] data = new byte[messageSize];
int totalRead = 0;

while (totalRead < messageSize) {
    int bytesRead = stream.read(data, totalRead, messageSize - totalRead);
    if (bytesRead < 0) {
        // Change behaviour if this isn't an error condition
        throw new IOException("Data stream ended prematurely");
    }
    totalRead += bytesRead;
}

Coonhound

ByteArrayInputStream will still read less than requested if you reach its end. – Spratt

@Stephen: That's true, but if you absolutely know that there's more data there, then I'd be very surprised to see it return less data. I'll edit to clarify though. – Coonhound

Nice explanation of what was happening. However, I think Peter Lawrey's suggestion to use readFully is cleaner than using a while loop to read the full amount of data. – Elizabetelizabeth

@Dunes: In the case of DataInputStream, you're absolutely right :) I'd missed that. Useful to have the loop version for non-DataInput. Will edit. – Coonhound

Answer 2

You can use DataInputStream this way.

byte[] bytes = new byte[len];
dis.readFully(bytes);

This will either return with all the data read or throw an IOException.

Miyasawa

Answer 3

read returns each time with the bits that were available at that time and -1 when done, you are typically supposed to do

while (true) {
  int numRead = dis.read(buffer, 0, len);
  if (numRead == -1) break;
  total.append(buffer, numRead);
}

Bentlee

yeah, len can be constant here e.g. 1024 – Bentlee

Answer 4

I would expect the behavior of read to block until the stream is closed or EOF is reached.

Then you need to check the Javadocs. The contract of read() is that it will read at least one byte, blocking if necessary until it has done so, or until EOS or an exception occurs. There is nothing in the specification that says it will read the entire length you requested. That's why it returns a length.

Nurmi

Answer 5

Is there a way of reading bytes from a socket that will always ensure that you read len bytes?

You can use Apache Commons IOUtils, they have a method that does exactly what you need:

    byte[] buf = new byte[BUFFER_SIZE];
    int length;

    do {
        length = IOUtils.read(inputStream, buf);
        if (length > 0) {
            //do something with buf
        }
    } while (length == BUFFER_SIZE);

Seise