java socket outputstream and broken pipe

Question

I have to send a dynamic buffer size to the socket stream. It works correctly, but when I try to send multiple buffers with a size bigger than int my_buffer_size =18 * 1024; (this is an indicative value)

I get the error (for some write):

Java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)

My code is very simple: For example If I want to send a big file I read a file stream with

byte[] bs = new byte[my_buffer_size];
while (... ){ 
fileInputStream.read(bs);
byte[] myBufferToSend = new byte[sizeBuffer];
DataOutputStream out = new DataOutputStream(cclient.getoutputStream());
out.writeInt(myBufferToSend.length);
out.write(myBufferToSend);
out.flush();
}

(The file is just a test the buffer size can be variable)

the SendBufferSize is 146988.

Is there a way to fix the broken pipe error? I read around but actually I didn’t solve the problem.

Thank you any help is appreciated

I use the classic ServerSocket serverSocket; and Socket cclient

Ablative

Would you please describe, how can you solve the problem? – Farris

Answer 1

'Broken pipe' means that you've written data to a connection that has already been closed by the other end.

Ergo the problem lies at the other end, not in this code. Possibly the other end doesn't really understand your length-word protocol for example, or doesn't implement it correctly.

If it's anything like this code it won't, because you're ignoring the result returned by read() and assuming that it fills the buffer. It isn't specified to do that, only to transfer at least one byte.

Caduceus

thank you! I understand that I was finding the problem to the wrong place. – Ablative

Answer 2

In common, receiving huge blocks is not supported by DataInputStream, because the readmethod just delegates to the underlying socket input stream and that socket input stream does not complain about not having read all. E.g. in Oracle Java 8 you get some 2^16 bytes and the rest is ignored. So when you close the socket after DataInputStream.read has returned, the sender node observes a "pipe broken" while still trying the send the rest of the huge block. Solution is a windowed read. Below is a DataInputStream-subclass, which does precisely this.

import java.io.DataInputStream;
import java.io.IOException;
import java.io.InputStream;

public class HugeDataInputStream extends DataInputStream
{
    int maxBlockLength;

    public HugeDataInputStream(InputStream in)
    {
        this(in, 0x8000);
    }

    public HugeDataInputStream(InputStream in, int maxBlockLength)
    {
        super(in);
        this.maxBlockLength = maxBlockLength;
    }

    public int readHuge(byte[] block) throws IOException
    {
        int n = block.length;
        if (n > maxBlockLength)
        {
            int cr = 0;
            while (cr < n)
            {
                cr += super.read(block, cr, Math.min(n - cr, maxBlockLength));
            }
            return cr;
        }
        else
        {
            return super.read(block);
        }
    }
}

Dumpcart