Working with byte arrays in C#

Question

I have a byte array that represents a complete TCP/IP packet. For clarification, the byte array is ordered like this:

(IP Header - 20 bytes)(TCP Header - 20 bytes)(Payload - X bytes)

I have a Parse function that accepts a byte array and returns a TCPHeader object. It looks like this:

TCPHeader Parse( byte[] buffer );

Given the original byte array, here is the way I'm calling this function right now.

byte[] tcpbuffer = new byte[ 20 ];
System.Buffer.BlockCopy( packet, 20, tcpbuffer, 0, 20 );
TCPHeader tcp = Parse( tcpbuffer );

Is there a convenient way to pass the TCP byte array, i.e., bytes 20-39 of the complete TCP/IP packet, to the Parse function without extracting it to a new byte array first?

In C++, I could do the following:

TCPHeader tcp = Parse( &packet[ 20 ] );

Is there anything similar in C#? I want to avoid the creation and subsequent garbage collection of the temporary byte array if possible.

Barrybarrymore

Save yourself the time/effort and use a pre-existing network capture/parsing framework like SharpPcap or Pcap.Net. Writing a TCP header parser is akin to writing a perl script that parses HTML. There are already many many different styles of that wheel to be found in the wild. – Toponymy

Possible duplicate: #1056423 – Wack

Answer 1

A common practice you can see in the .NET framework, and that I recommend using here, is specifying the offset and length. So make your Parse function also accept the offset in the passed array, and the number of elements to use.

Of course, the same rules apply as if you were to pass a pointer like in C++ - the array shouldn't be modified or else it may result in undefined behavior if you are not sure when exactly the data will be used. But this is no problem if you are no longer going to be modifying the array.

Duprey

Although this solves the issue, the asker said "Is there a CONVENIENT way to pass the TCP byte array...?". The answer from @casperOne seems a better fit for this question. – Flores

Answer 2

I would pass an ArraySegment<byte> in this case.

You would change your Parse method to this:

// Changed TCPHeader to TcpHeader to adhere to public naming conventions.
TcpHeader Parse(ArraySegment<byte> buffer)

And then you would change the call to this:

// Create the array segment.
ArraySegment<byte> seg = new ArraySegment<byte>(packet, 20, 20);

// Call parse.
TcpHeader header = Parse(seg);

Using the ArraySegment<T> will not copy the array, and it will do the bounds checking for you in the constructor (so that you don't specify incorrect bounds). Then you change your Parse method to work with the bounds specified in the segment, and you should be ok.

You can even create a convenience overload that will accept the full byte array:

// Accepts full array.
TcpHeader Parse(byte[] buffer)
{
    // Call the overload.
    return Parse(new ArraySegment<byte>(buffer));
}

// Changed TCPHeader to TcpHeader to adhere to public naming conventions.
TcpHeader Parse(ArraySegment<byte> buffer)

Pashto

ArraySegment<byte> seg = new ArraySegment<byte>(packet, 20, packet.Length-1); – Waldner

ooops! ArraySegment<byte> b2 = new ArraySegment<byte>(b1, 20, b1.Length-20); – Waldner

But... doesn't this create a new class for the GC to collect, which the asker wanted to avoid? – Unlicensed

@Unlicensed The OP wanted to prevent copying the segment of the byte array; ArraySegment is a wrapper around the array, it doesn't perform a copy. It basically gives you a view over the array that doesn't allow you to work outside of those bounds. – Pashto

Curious to know why this answer is not preferred. Are there performance implications compared with the answer above? – Wack

I would add that as ArraySegment is a structure and not a class, it probably won't be bothersome for the GC. – Bloodletting

Answer 3

If an IEnumerable<byte> is acceptable as an input rather than byte[], and you're using C# 3.0, then you could write:

tcpbuffer.Skip(20).Take(20);

Note that this still allocates enumerator instances under the covers, so you don't escape allocation altogether, and so for a small number of bytes it may actually be slower than allocating a new array and copying the bytes into it.

I wouldn't worry too much about allocation and GC of small temporary arrays to be honest though. The .NET garbage collected environment is extremely efficient at this type of allocation pattern, particularly if the arrays are short lived, so unless you've profiled it and found GC to be a problem then I'd write it in the most intuitive way and fix up performance issues when you know you have them.

Guinn

Thanks, Greg. In truth, I haven't profiled it. But common sense says allocating the new array and copying the 20 bytes is less efficient than simply using the array as is. Given the number of packets, I need to be as efficient as possible. Plus, it looks 'neater' without the allocation and copy. – Barrybarrymore

The array copy done in the question is faster than using Linq for this purpose. It does not solve the problem of creating a copy of the array anyway. – Unlicensed

However, I completely agree that small array copies like this are unlikely to cause problems. After all, TCP packets are relatively limited in size and quantity. I've only ever had problems with small array allocations in a program which literally created billions of copies of a tiny array, but unless the question is about a TCP logger at an ISP, I suspect that's not the case here. – Unlicensed

Answer 4

If you really need these kind of control, you gotta look at unsafe feature of C#. It allows you to have a pointer and pin it so that GC doesn't move it:

fixed(byte* b = &bytes[20]) {
}

However this practice is not suggested for working with managed only code if there are no performance issues. You could pass the offset and length as in Stream class.

Bunghole

Answer 5

If you can change the parse() method, change it to accept the offset where the processing should begin. TCPHeader Parse( byte[] buffer , int offset);

Bugaboo

Answer 6

You could use LINQ to do something like:

tcpbuffer.Skip(20).Take(20);

But System.Buffer.BlockCopy / System.Array.Copy are probably more efficient.

Retrogradation

Answer 7

This is how I solved it coming from being a c programmer to a c# programmer. I like to use MemoryStream to convert it to a stream and then BinaryReader to break apart the binary block of data. Had to add the two helper functions to convert from network order to little endian. Also for building a byte[] to send see Is there a way cast an object back to it original type without specifing every case? which has a function that allow for converting from an array of objects to a byte[].

  Hashtable parse(byte[] buf, int offset )
  {

     Hashtable tcpheader = new Hashtable();

     if(buf.Length < (20+offset)) return tcpheader;

     System.IO.MemoryStream stm = new System.IO.MemoryStream( buf, offset, buf.Length-offset );
     System.IO.BinaryReader rdr = new System.IO.BinaryReader( stm );

     tcpheader["SourcePort"]    = ReadUInt16BigEndian(rdr);
     tcpheader["DestPort"]      = ReadUInt16BigEndian(rdr);
     tcpheader["SeqNum"]        = ReadUInt32BigEndian(rdr);
     tcpheader["AckNum"]        = ReadUInt32BigEndian(rdr);
     tcpheader["Offset"]        = rdr.ReadByte() >> 4;
     tcpheader["Flags"]         = rdr.ReadByte() & 0x3f;
     tcpheader["Window"]        = ReadUInt16BigEndian(rdr);
     tcpheader["Checksum"]      = ReadUInt16BigEndian(rdr);
     tcpheader["UrgentPointer"] = ReadUInt16BigEndian(rdr);

     // ignoring tcp options in header might be dangerous

     return tcpheader;
  } 

  UInt16 ReadUInt16BigEndian(BinaryReader rdr)
  {
     UInt16 res = (UInt16)(rdr.ReadByte());
     res <<= 8;
     res |= rdr.ReadByte();
     return(res);
  }

  UInt32 ReadUInt32BigEndian(BinaryReader rdr)
  {
     UInt32 res = (UInt32)(rdr.ReadByte());
     res <<= 8;
     res |= rdr.ReadByte();
     res <<= 8;
     res |= rdr.ReadByte();
     res <<= 8;
     res |= rdr.ReadByte();
     return(res);
  }

Sorcim

That is certainly a simple, elegant way to do it. I've defined classes for the IP, TCP, and UDP headers. Internally, I use the BitConverter functions to extract the values and IPAddress.NetworkToHostOrder to swap the bytes. I may run some tests to see which approach is more efficient. – Barrybarrymore

you might want to look at stackoverflow.com/questions/2871 if performance is what you are after and switch from classes to structs. I would also leave everything in network order and only convert when you need it. – Sorcim

Answer 8

I don't think you can do something like that in C#. You could either make the Parse() function use an offset, or create 3 byte arrays to begin with; one for the IP Header, one for the TCP Header and one for the Payload.

Unappealable

A better solution IMO would be to use ArraySegment<T> which does the bounds checking for you, so that you don't have to duplicate it everywhere. – Pashto

Answer 9

There is no way using verifiable code to do this. If your Parse method can deal with having an IEnumerable<byte> then you can use a LINQ expression

TCPHeader tcp = Parse(packet.Skip(20));

Metapsychology

Answer 10

Why not flip the problem and create classes that overlay the buffer to pull bits out?

// member variables
IPHeader ipHeader = new IPHeader();
TCPHeader tcpHeader = new TCPHeader();

// passing in the buffer, an offset and a length allows you
// to move the header over the buffer
ipHeader.SetBuffer( buffer, 0, 20 );

if( ipHeader.Protocol == TCP )
{
    tcpHeader.SetBuffer( buffer, ipHeader.ProtocolOffset, 20 );
}

Palaeobotany

Answer 11

Some people who answered

tcpbuffer.Skip(20).Take(20);

did it wrong. This is excellent solution, but the code should look like:

packet.Skip(20).Take(20);

You should use Skip and Take methods on your main packet, and tcpbuffer should not be exist in the code you posted. Also you don't have to use then System.Buffer.BlockCopy.

JaredPar was almost correct, but he forgot the Take method

TCPHeader tcp = Parse(packet.Skip(20));

But he didn't get wrong with tcpbuffer. Your last line of your posted code should look like:

TCPHeader tcp = Parse(packet.Skip(20).Take(20));

But if you want to use System.Buffer.BlockCopy anyway instead Skip and Take, because maybe it is better in performance as Steven Robbins answered : "But System.Buffer.BlockCopy / System.Array.Copy are probably more efficient", or your Parse function cannot deal with IEnumerable<byte>, or you are more used to System.Buffer.Block in your posted question, then I would recommend to simply just make tcpbuffer not local variable, but private or protected or public or internal and static or not field (in other words it should be defined and created outside method where your posted code is executed). Thus tcpbuffer will be created only once, and his values (bytes) will be set every time you pass the code you posted at System.Buffer.BlockCopy line.

This way your code can look like:

class Program
{
    //Your defined fields, properties, methods, constructors, delegates, events and etc.
    private byte[] tcpbuffer = new byte[20];
    Your unposted method title(arguments/parameters...)
    {
    //Your unposted code before your posted code
    //byte[] tcpbuffer = new byte[ 20 ]; No need anymore! this line can be removed.
    System.Buffer.BlockCopy( packet, 20, this.tcpbuffer, 0, 20 );
    TCPHeader tcp = Parse( this.tcpbuffer );
    //Your unposted code after your posted code
    }
    //Your defined fields, properties, methods, constructors, delegates, events and etc.
}

or simply only the necessary part:

private byte[] tcpbuffer = new byte[20];
...
{
...
        //byte[] tcpbuffer = new byte[ 20 ]; No need anymore! This line can be removed.
        System.Buffer.BlockCopy( packet, 20, this.tcpbuffer, 0, 20 );
        TCPHeader tcp = Parse( this.tcpbuffer );
...
}

If you did:

private byte[] tcpbuffer;

instead, then you must on your constructor/s add the line:

this.tcpbuffer = new byte[20];

or

tcpbuffer = new byte[20];

You know that you don't have to type this. before tcpbuffer, it is optional, but if you defined it static, then you cannot do that. Instead you'll have to type the class name and then the dot '.', or leave it (just type the name of the field and that's it all).

Checkered