How to identify end of InputStream in java

Question

I am trying to read bytes from server using Socket program, ie I am using InputStream to read the bytes. If I pass the length size I am able to read the bytes, but I am not sure what may be the length. So I am not able initialize the byte array.

Also I tried while (in.read() != -1), I observered it loop works fine when the data is sent, but the next line after the loop is not executable, I feel its still looking for the data in the stream but there is no data. If I close the Server connection, then my client will execute the next line followed to the loop.

I am not sure where I am going wrong?

this.in = socket.getInputStream();

int dataInt = this.in.read();

while(dataInt != -1){
    System.out.print(","+i+"--"+dataInt);
    i++;
    dataInt = this.in.read();
}

System.out.print("End Of loop");

I get the output as:-

,1--0,2--62,3--96,4--131,5--142,6--1,7--133,8--2,9--16,10--48,11--56,12--1,13--0,14--14,15--128,16--0,17--0,18--0,19--48,20--0,21--0,22--0,23--0,24--0,25--1,26--0,27--0,28--38,29--114,30--23,31--20,32--70,33--3,34--20,35--1,36--133,37--48,38--51,39--49,40--52,41--49,42--55,43--49,44--52,45--52,46--54,47--55,48--50,49--51,50--52,51--48,52--53,53--56,54--51,55--48,56--48,57--57,58--57,59--57,60--57,61--57,62--57,63--57,64--56

But no output for :- End Of loop

Please guide how shall I close the loop?

Looking forward for you response. Thanking you all in advance.

Roarke

Answer 1

It's looking for more data because nothing's told it that there won't be more data. The other end of the network could send more data at any time.

It's not clear whether you're designing the client/server protocol or just trying to implement it, but typically there are three common ways of detecting the end of a message:

• Closing the connection at the end of the message
• Putting the length of the message before the data itself
• Using a separator; some value which will never occur in the normal data (or would always be escaped somehow)

Personally I favour length-prefixing when possible; it makes the reading code significantly simpler, but still allows multiple messages on the same connection.

(Additionally, I agree with Daniel that you should be using the overload of read which reads a whole buffer at a time, instead of a single byte. This will be much more efficient - but doesn't fundamentally change the nature of your current issue.)

Officer

I am the client trying to pass some data to server. The server will respond in the bytes, I am trying to read this bytes. The server is already created & maintaned by some one else. Here my doubt would be when the server has responsded it has to close If I am not wrong. Also How would I close the connection (Client) as I am still not able to identify that I have read the complete response – Roarke

@Vardhaman: It entirely depends on the protocol. I wasn't suggesting that the reading side should close the connection - the writing side (the server in this case) should close its connection, at which point you'll see the end of the data. But that's only if the protocol is designed that way... what protocol is it using? – Officer

Its TCP I guess :) as told to me they gave the port & Ip address & told me to connect through Socket – Roarke

@Vardhaman: It's not just TCP though. There's a higher level protocol involved, which describes the format of the messages, the control flow etc. You should ask "them" what the expected behaviour is. – Officer

From a view of a general solution, is there any way to identify end of InputStream, rather than relying on "Putting the length of the message before the data itself“ or "Using a separator"? In iOS, this man makes it: github.com/robbiehanson/CocoaAsyncSocket. – Frenzied

@ikzjfr0: If it's the genuine end of a stream which has been closed, you just read from it and get -1 bytes. But if nothing closes the stream, there's no way of knowing whether any more data may come. – Officer

@JonSkeet, but CocoaAsyncSocket indeed has made it – Frenzied

@ikzjfr0: Well I'm not going to pore over the code to understand how. TCP/IP is a stream-based protocol; anything over that has to take account of it. Now a library on top of it can potentially do the message-length-prefixing for you, but that doesn't remove the need for it... it just does it for you. Those are very different. – Officer

Answer 2

I think you've actually answered your own question.

The reason you are not exiting the loop is that the end of input stream only happens on the client end after the server end closes its socket. (Or more precisely, after it closes its socket output stream ... or the equivalent ... and the close event has propagated to the client end.)

Until that event happens, it is possible that the server could decide to write more data to the socket. So the client-side read blocks ... until it either gets more data or it sees the protocol event that says that the server end has closed.

(Actually, other events can unblock the client read, but they will all result in an IOException of some kind, and probably a stream that you can't read any more data from.)

---

Now, if you don't want to close the server end now because you want to send more stuff on the socket later on, you are going to have to change your application protocol to use some kind of framing mechanism. For instance, the server might send a frame (or record, or whatever) consisting of byte count followed by the given number of bytes. Alternatvely, it could use a distinguished byte value or byte sequence to mark the end of a frame.

Elephantine

Thank you , for the response.please correct me if I am wrong. I need to know the byte length that I am going to receive from the Socket server. So that I can come out of the loop based on the loop. – Roarke

@Roarke - if you want to do it that way, then the server has to sent the byte length to the client first. – Elephantine

Thank you guys actually the 1st two byte give me the length of the message format, So based on the 1st two bytes I will identfiy the byte array, I need to handle the error – Roarke

Answer 3

You can run this example.

If you are going to wait for the end of the stream you have to close it on the sending side for and EOF (-1) to be received.

For sending multiple binary messages I prefer to send the length before the message as it allow the code to read large blocks at once (i.e. because it knows how much it is going to get)

ServerSocket ss = new ServerSocket(0);
Socket s = new Socket("localhost", ss.getLocalPort());
Socket s2 = ss.accept();

final OutputStream out = s.getOutputStream();
out.write("Hello World!".getBytes());
out.close();

final InputStream in = s2.getInputStream();
for (int b = 0; ((b = in.read()) >= 0);) {
    System.out.println(b + " " + (char) b);
}
System.out.println("End of stream.");
s.close();
s2.close();
ss.close();

prints

72 H
101 e
108 l
108 l
111 o
32  
87 W
111 o
114 r
108 l
100 d
33 !
End of stream.

Disoblige

Sorry, the -- is a really bad choice of seperator ;) – Disoblige

Answer 4

For a one time TCP connection, this works for me:

ByteArrayOutputStream buffer = new ByteArrayOutputStream();
InputStream input = socket.getInputStream();
final byteSize = 1024;
byte[] data = new byte[byteSize];
int nRead;
while ((nRead = input.read(data, 0, data.length)) != -1) {
    buffer.write(data, 0, nRead);
    if (nRead < byteSize) {
        break;
    }
}
buffer.flush();
byte[] byteArray = buffer.toByteArray();

For reference, this is adapted from Java InputStream to Byte Array and ByteBuffer.

Succinate

Answer 5

I've had a similar problem where in.read() just hangs at the end of the data rather than returning a -1.

If you have no control over the server code, is there a reliable marker you can detect in the returned data (e.g., </html>) and use that to end the reading loop?

Failing that, consider using socket.setSoTimeout(reasonableValue) so at least your app won't be kept hanging forever...

Voorhis

No you haven't. You've had a problem confusing end of message with end of stream. – Poacher

How else can I detect end of stream, so I don't end up with in.read() hanging my application? – Voorhis

@Voorhis - See my answer, or Jon Skeet's, or Peter Lawrey's. – Elephantine

Answer 6

In case -1 it does not end the loop. Put 65535 in the condition and i am 99% sure it will stop the loop.

while(dataInt != 65535){
    System.out.print(","+i+"--"+dataInt);
    i++;
    dataInt = this.in.read();
}

Her

Answer 7

I had also the problem that I did not come out of the loop. My solution looked similar to this:

while (in.ready()){
    System.out.print(","+i+"--"+dataInt);
    i++;
    dataInt = this.in.read();
}

Viguerie

That only returns whatever is currently in the socket receive buffer. There is no guarantee that that is a complete message. – Poacher

Answer 8

You can safely close the stream after the while loop. dataInt being -1 means that there is nothing more to read from it.

http://download.oracle.com/javase/1.4.2/docs/api/java/io/InputStream.html#read()

If the while loop is not exiting it means that there is still data being written at the other end of the stream. Close the stream at the other end. If you can post code where you write data to stream that will be helpful.

Personalty

I think you've misunderstood the question... dataInt is never -1. – Officer

He is reading from a Socket - semantics are different than for FileIO – Emersen

Thank you but I mean to say this condition is not getting called – Roarke

Hi Nishan, Please find my code, I am the client there is already a socket server. Which I dont have control on as it is been control by Others , They have given the IP & port number – Roarke

public void sendMesageTOServerSocket(String messageBody){ byte []arraybyte = hexStringToByteArray(messageBody); try { this.wr = this.socket.getOutputStream(); wr.write(arraybyte); wr.flush(); } catch (IOException e) { e.printStackTrace(); BVLog.error(5,"IOException Exception for the FEP Socket while sending message-->"+e.getMessage()); BVLog.printStackTrace(5, e); }catch(Exception ee){ ee.printStackTrace(); } } – Roarke

*/ public String getServerResponse(int len){ int i = 1; String getfep = null; //Get response try { this.in = socket.getInputStream(); int dataInt = this.in.read(); byte[] buf = new byte[4096]; int bytesRead; while( (bytesRead = in.read(buf)) != -1 ) { System.out.print(","+i+"--"+bytesRead); i++; } System.out.print("End Of loop"); } catch (IOException e) { BVLog.error(5,"IOException Exception for the FEP Socket while getting message-->"+e.getMessage()); BVLog.printStackTrace(5, e); } return getfep.toString(); } – Roarke

@Heiko Rupp: no, the semantics are identical. When you get to the end of the stream the read returns -1. Period. – Poacher

EJP no, they are not. For a File you always know the size and thus you know when there is no more data. For Sockets, you can not know if the remote is going to send you more if the remote does not explicitly send the size in advance or if it does not close the socket. – Emersen

@HeikoRupp For a File you always know the size if it isn't increasing or the metadata has been flushed between the last append and the last time you checked it, but that doesn't change the fact that the read semantics of are the same for files as they are for sockets, contrary to your express statement. – Poacher