How to get all data from NetworkStream

Question

I am trying to read all data present in the buffer of the Machine connected through TCP/IP but i don't know why i am not getting all data ,some data is getting Missed. Here is the code that i am using ..

using (NetworkStream stream = client.GetStream())
{
    byte[] data = new byte[1024];
    int numBytesRead = stream.Read(data, 0, data.Length);
    if (numBytesRead > 0)
    {
       string str= Encoding.ASCII.GetString(data, 0, numBytesRead);
    }
}

Please tell me what i am missing to get all the data from the machine. Thanks in advance..

Littoral

any chance this can have anything to do with the fact that you're explicitly reading at most 1024 bytes? – Paoting

@Paoting Ok So how can i read all the data? – Littoral

What is exactly missing? Begin of the stream, end, or random bytes? Are you able recognize end of the stream from the byte sequence? Do you know expected length of the data? – Eyrie

@Eyrie It is missing random bytes.And data length is not known it can be anything? – Littoral

No, it cannot be anything. You need to define a protocol if you don't want to wait until the client closes the connection. Otherwise - keep reading until the client closes the connection. – Shockproof

@Shockproof My machine has the feature to store the data into the buffer if there is no client listening/connected to the machine.So my requirement is if i connects to the machine in that case whatever/all data is there in the buffer of the machine,my code should be able to read it... – Littoral

I'm not sure what you're saying there or how that is relevant to what I said in my comment. – Shockproof

@Shockproof I just wanted to ask how i can use my posted code to get all the data from the buffer of the machine – Littoral

And I'm not sure whether you know what that means and that actually is what you want. If by "the buffer of the machine" you mean "the data received until now", which may or may not be all data the other party sent you, then yes, you can use NetworkStream.DataAvailable. – Shockproof

@shubham Hegdey are you sure data are missing? The problem can be in Encoding.ASCII.GetString as well. Are you transferring text? Is the text encoded as ASCII (only 128 characters)? – Eyrie

@Eyrie Yes it is .Please check the last answer given by GeorgeChond.Is his code more meaningfull in comparison to mine – Littoral

@shubham Hegdey GeorgeChond answer doesn't answer me my questions: Is the text encoded as ASCII (only 128 characters)? The conversion from bytes to string can be the problematic part. – Eyrie

Answer 1

The problem with your code is that you will not get all the data if the data size is bigger than the buffer size (1024 bytes in your case) so you have to Read the stream inside the loop. Then you can Write all the data inside a MemoryStream until the end of the NetworkStream.

      string str;
      using (NetworkStream stream = client.GetStream())
      {
            byte[] data = new byte[1024];
            using (MemoryStream ms = new MemoryStream())
            {

                int numBytesRead ;
                while ((numBytesRead = stream.Read(data, 0, data.Length)) > 0)
                {
                    ms.Write(data, 0, numBytesRead);


                }
               str = Encoding.ASCII.GetString(ms.ToArray(), 0, (int)ms.Length);
            }
        }

Eldreeda

No, in str you will get the whole string which is stored in the NetworkStream the time you read it. – Eldreeda

You mean to say that all the data available in the buffer .Am i right? – Littoral

Also a possible solution with using MemoryStream inside NetworkStream :) +1 If you want an approach that uses your code and extends it check the other answer aswell. Try to debug the code you use and understand what is happening at each step, this might help you. – Rationale

I am getting error at ` while (numBytesRead = stream.Read(data, 0, data.Length) > 0)` can not convert bool to int and buffer does not exists in the current context and at the last line of the code Encoding.ASCII.GetString(ms.ToArray(), 0, ms.Length); – Littoral

Well, I forgot to add parenthesis in the while that's why you getting that error (I edited my answer) and I am not using the buffer in the Encoding part. There is no buffer in the code at all. – Eldreeda

Again i am getting error at ` ms.Write(buffer, 0, numBytesRead); ` the name buffer does not exists in the current context – Littoral

Oops, sorry about that it's ms.Write(data, 0, numBytesRead); – Eldreeda

Ok Thnaks .but now also getting error at Encoding.ASCII.GetString(ms.ToArray(), 0, ms.Length); The best overloaded method has some invalid arguments..Please check – Littoral

Try Encoding.ASCII.GetString(ms.ToArray(), 0, (int)ms.Length) – Eldreeda

I have one query as what does it means byte[] data = new byte[1024]; 1024 in the above code.Does it mean the present in one line? – Littoral

Sorry, I didn't get that. – Eldreeda

Does byte[] data = new byte[1024]; means the byte data for one line of text to get from the machine because everytime i am seeing that it is returning line by line data – Littoral

Nothing to do with the lines of text. I means that in that array 1024 bytes of data are temporarily stored in order to be saved in the MemoryStream. For example in this array a lot of lines of text can be stored. – Eldreeda

Can't we just use networkStream.CopyTo(memoryStream);? (stream.CopyTo(ms);) – Tiptop

Be careful. This solution CLOSE connection after using section ends. – Carrero

For anyone else reading this for string data why not just use the following: StreamReader reader = new StreamReader(stream); string data = reader.ReadToEnd(); – Psychobiology

Any idea why I get the next error? System.IO.IOException: Unable to read data from the transport connection: Operation on non-blocking socket would block. – Ullyot

@DanielReyhanian does that help? #46307589 – Eldreeda

@GeorgeChond It was a problem with my server side. Cheers. – Ullyot

Answer 2

This example from MSDN: NetworkStream.DataAvailable shows how you can use that property to do so:

// Examples for CanRead, Read, and DataAvailable. 
// Check to see if this NetworkStream is readable. 
if(myNetworkStream.CanRead)
{
    byte[] myReadBuffer = new byte[1024];
    StringBuilder myCompleteMessage = new StringBuilder();
    int numberOfBytesRead = 0;

    // Incoming message may be larger than the buffer size. 
    do{
         numberOfBytesRead = myNetworkStream.Read(myReadBuffer, 0, myReadBuffer.Length);

         myCompleteMessage.AppendFormat("{0}", Encoding.ASCII.GetString(myReadBuffer, 0, numberOfBytesRead));

    }
    while(myNetworkStream.DataAvailable);

    // Print out the received message to the console.
    Console.WriteLine("You received the following message : " +
                                 myCompleteMessage);
}
else
{
     Console.WriteLine("Sorry.  You cannot read from this NetworkStream.");
}

Milliner

And what does this example do? Is NetworkStream.DataAvailable reliable for OP's purpose? When does it become false? – Shockproof

I becomes false when no more data is in the socket, and why would not be reliable? – Milliner

No. NetworkStream.DataAvailable being false just indicates there's no data in the receiver's receive buffer. It doesn't mean that the sender is done sending, so it can't be used to implement a protocol and it won't solve OP's problem. – Shockproof

Read yourself, you are saying I'm right, and he asked to read all the data in the stream. You are going one step beyond and I agree with you, he needs to implement a protocol, but that is not the question here, he ask how to read all the data in the stream. – Milliner

I highly doubt that is what OP wants, but you're right. – Shockproof

Me too, he will make a new question, and better to crack himself trying to understand, we all went through that once :D – Milliner

Answer 3

Try this code:

using (NetworkStream stream = client.GetStream())
    {
         while (!stream.DataAvailable)
         {
             Thread.Sleep(20);
         }
        
         if (stream.DataAvailable && stream.CanRead)
         {
              Byte[] data = new Byte[1024];
              List<byte> allData = new List<byte>();
        
              do
              {
                    int numBytesRead = stream.Read(data,0,data.Length);
    
                    if (numBytesRead == data.Length)
                    {
                         allData.AddRange(data);
                    }
                    else if (numBytesRead > 0)
                    {
                         allData.AddRange(data.Take(numBytesRead));
                    }                                    
               } while (stream.DataAvailable);
          }
    }
      

Hope this helps, it should prevent that you miss any data sended to you.

Rationale

Answer 4

The synchronous method sometimes does not display the request body. Using the asynchronous method always does.

string request = default(string);
StringBuilder sb = new StringBuilder();

byte[] buffer = new  byte[client.ReceiveBufferSize];
int bytesCount;

if (client.GetStream().CanRead)
{
    do
    {
        bytesCount = client.GetStream().ReadAsync(buffer, 0, buffer.Length).Result;
        sb.Append(Encoding.UTF8.GetString(buffer, 0, bytesCount));
    }
    while(client.GetStream().DataAvailable);

    request = sb.ToString();
}

Accroach

this is a good solution too. – Popple

Answer 5

Try this:

 private string GetResponse(NetworkStream stream)
    {
        byte[] data = new byte[1024];
        using (MemoryStream memoryStream = new MemoryStream())
        {
            do
            {
                stream.Read(data, 0, data.Length);
                memoryStream.Write(data, 0, data.Length);
            } while (stream.DataAvailable);

            return Encoding.ASCII.GetString(memoryStream.ToArray(), 0, (int)memoryStream.Length);
        }
    }

Whitefish

I would replace line 8 & 9 by var readCount = stream.Read(data, 0, data.Length); and memoryStream.Write(data, 0, readCount); respectively. This way one doesn't get trailing 0 bytes which translate into "\0" null chars in the output string. – Tumbledown

Answer 6

TCP itself does not have any ways to define "end of data" condition. This is responsibility of application level portocol.

For instance see HTTP request description:

A client request (consisting in this case of the request line and only one header field) is followed by a blank line, so that the request ends with a double newline

So, for request end of data is determined by two newline sequences. And for response:

Content-Type specifies the Internet media type of the data conveyed by the HTTP message, while Content-Length indicates its length in bytes.

The response content size is specified in header before data. So, it's up to you how to encode amount of data transferred at once - it can be just first 2 or 4 bytes in the beginning of the data holding total size to read or more complex ways if needed.

Hyo

Answer 7

for my scenario, the message itself was telling the length of subsequent message. here is the code

 int lengthOfMessage=1024;
 string message = "";
 using (MemoryStream ms = new MemoryStream())
 {
      int numBytesRead;
      while ((numBytesRead = memStream.Read(MessageBytes, 0, lengthOfMessage)) > 0)
      {
            lengthOfMessage = lengthOfMessage - numBytesRead;
            ms.Write(MessageBytes, 0, numBytesRead);
      }
      message = Encoding.ASCII.GetString(ms.ToArray(), 0, (int)ms.Length);
 }

Inspector

Answer 8

@George Chondrompilas answer is correct but instead of writing it by yourself you can use CopyTo function which does the same :

https://mcmap.net/q/743145/-will-c-networkstream-read-wait-until-the-specified-amount-of-data-is-read

Pinkie