What are the use-cases for IsLittleEndian in BitConverter class?

Question

I was so happy when I discovered IsLittleEndian field in BitConverter. I thought of course it should be there and I should be able to specify whatever endian I like. Well, my happiness didn’t last long. Spent some time till found out that there is no way to set the field. The field is readonly, and it is only set to true in static constructor:

static BitConverter()
{
    IsLittleEndian = true;
}

It is funny that the field is actually used in the code. For example ToInt32 method implementation looks like this:

if (IsLittleEndian)
{
     return (((numRef[0] | (numRef[1] << 8)) | (numRef[2] << 0x10)) | (numRef[3] << 0x18));
}
return ((((numRef[0] << 0x18) | (numRef[1] << 0x10)) | (numRef[2] << 8)) | numRef[3]);

So seems like the ToInt32 is perfectly capable to handle both little and big endians.

My question is: how come there is very useful piece of code that is already implemented and sitting there in the FCL, but there is no way to use it (unless you start messing with reflection of course)? Is it just because some developers didn’t meet the deadline and left the job half-done? Even if so, why the code is not available, but the field is? I hope there is a good reason for this.

I want to make myself clear. I don't need a solution on how to handle big-endian values. I do have a solution. The solution is actually shown in my question.

Everywhere

Answer 1

The answer lies in looking at the reference source for the BitConverter class.

The relevant extract is:

        // This field indicates the "endianess" of the architecture.
        // The value is set to true if the architecture is
        // little endian; false if it is big endian.
#if BIGENDIAN
        public static readonly bool IsLittleEndian /* = false */;
#else
        public static readonly bool IsLittleEndian = true;
#endif

The flag is hard-wired by the preprocessor directive because the endian-ness of the architecture for which a particular version of the framework is compiled will not change.

Winograd

Answer 2

Unfortunately the IsLittleEndian field is just there to inform you. But Jon Skeets MiscUtil library has a nice EndianBitConverter, that supports little and big endian. There are also endian-aware BinaryWriter/-Reader classes.

Here is the link: http://www.yoda.arachsys.com/csharp/miscutil/

Edit: sorry but I don't have a better explanation. I think that this should have been included in the framework and I guess the code is currently there, so that it is easy to port the Converter to another architecture.

But exposing that functionality is a tad more complex than just making the field public. The converter is static, so changing the flag effectively changes global state and in a multi-threaded scenario this would be disastrous. The way to go is probably to provide two BitConverter objects that you can instantiate and use locally (that's what MiscUtil does). This requires additional classes and/or interfaces, so maybe it was a deadline issue and just got dropped for the time being. Let's hope it is added some time later.

Herrenvolk

This is not exactly an answer to my question. You just restated the fact that I already stated in my question that IsLittleEndian is not available for setting. Also you are suggesting a solution, but I have one and I didn't ask for a solution. Please read the question. – Everywhere

I read your edit after answering and I wasn't entirely sure if you wanted a solution (but I thought mentioning MiscUtil wouldn't hurt). I've added some thoughts of mine to the answer. – Herrenvolk

Answer 3

First let's establish that the class BitConverter is specifically designed for bit conversion for the local processor only. That is why the IsLittleEndian is read-only. As a result, it does not support converting to or from big-endian if the local processor is little-endian and vice versa.

Although I do not know the reasoning for omitting support for general endianness, the most logical reason to me is performance. A class that is widely used throughout the framework for its intended purpose (conversion to and from the native processor's endianness), should be as performant as possible. By limiting the generality of the class, its performance is improved by restricting the cases that must be handled. By only supporting little-endian it is likely measurable faster.

OK, so now we get to the crux of the question. Why would the authors include code to handle both little-endian and big-endian if the overall design of the class only intends to support one?

Again, we can only speculate. But the answer likely rests in two observations:

• the disassembled code that refers to IsLittleEndian is an unimportant case performance-wise
• writing portable code, if it doesn't affect performance, is a good software engineering

The reason that that the code you copied from the ToInt32 method is unimportant, is because it is only used for unaligned memory. The 99% code-path is a direct unsafe "memcpy" of the bits.

Even to the extent that converting from unaligned memory does occur, the code that handles it is an order of magnitude less efficient that the raw method. So an extra condition doesn't really hurt performance.

The net result is:

• The BitConverter class is as efficient as possible for its limited purpose
• The source-code for BitConverter is nevertheless portable to big-endian processor architectures

Caucasian

Answer 4

According to the MSDN documentation IsLittleEndian it's meerly to inform you (your program or the BitConverter Class) of whether or not the architecture is Little or Big Endian. I don't that there is an intended use outside of that.

Wilkie

Despite what documentation says, Reflector shows that IsLittleEndian is always true, and there is no way to make it false. I need to make it false. – Everywhere

it's always false because your computer is 32bit. – Monoxide

@David Wick - seriously? did you just make up this explanation? :) – Everywhere

@David Wick: Endianness is unrelated to the number of bits. Endianness depends on the processor type. x86 and x64 processors are Little Endian processors. – Ignorant

yeah, i got that now. skimmed wikipedia a little too quickly. – Monoxide

Answer 5

it's set internally based on the achitecture type.

from the docs:

"Different computer architectures store data using different byte orders. "Big-endian" means the most significant byte is on the left end of a word. "Little-endian" means the most significant byte is on the right end of a word."

Edit:

This was a design decision made by the c# team. The function is capable of converting from both types because it can be used on both types of systems. it is up to you as the developer to convert it otherwise.

"...All methods of BitConverter accept or return byte arrays in SYSTEM endian order..."

int i = BitConverter.ToInt32(byte[] inputdata);
(manipulate i)
return BitConverter.GetBytes(i);

"In the case where the data never leaves the confines of your application, this actually works fine."

For more information read this article.

Monoxide

@David Wick - do you seriously think that I didn't read documentation? Also I liked your comment you would never manually have a use for this field. Not that I would or wouldn't, I do need to use it. I receive a byte stream generated by hardware device and I need to convert data to a list of int. I can't use BitConverter for it, because the date are big endian. – Everywhere

convert the data to little-edian first with a byte swap. – Monoxide

@David Wick - I know how to handle big-endian. My question actually shows the code to handle big-endian. My question is not about how to do this. My question is why BitConverter has the code to do this, but I can't use this code. – Everywhere

how is it suppose to know if a byte stream is big edian or little edian? it's just a stream of bytes. they're making the assumption that the data came from the same machine. it's up to you as the developer to convert it otherwise. – Monoxide

@David Wick - I don't expect it to guess what kind of endian is my stream. I never asked for that. I know that my stream is big-endian. I do know how to parse it. I just basically copy/pasted the code from BitConverter class implementation. So if you look at the code, BitConverter is capable of handling big-endian, but there is no way to use this code. – Everywhere

the use is that it will work for both x64 and x86 machines. this is clearly a design decision made by the c# team not to allow you to set the "edian-ness" and force you to convert external data if necessary. the docs are pretty clear, this field is for determining what edian the system uses. read this for more details. – Monoxide

and it looks like im incorrect in stating x64/x86 as being big/little respectively. i made that assumption based on the documenations example. in any case, you get the idea. – Monoxide

Answer 6

See http://snipplr.com/view/15179/adapt-systembitconverter-to-handle-big-endian-network-byte-ordering-in-order-to-create-number-types-from-bytes-and-viceversa/ if you want an implementation which allows you to set the endian-ness.

Calebcaledonia

Answer 7

I'm pretty sure they set it to true with no possibility of false because all versions of Windows are little-endian.

Now, the issue of them doing if (IsLittleEndian) in a class that never sets IsLittleEndian to anything other than true is most likely a just-in-case scenario. This way if there ever does need to be a compile of the .NET BCL for big-endian, a simple #if/#else around that one assignment will suffice, rather than having to also write new code.

I bet Mono sets it false for some operating systems and architectures.

Edit: And I was right. Mono does the following. Technically none of the other code would need additions, except that they had to write everything differently in Mono due to copyright issues.

public static readonly bool IsLittleEndian = AmILittleEndian ();

static unsafe bool AmILittleEndian ()
{
  // binary representations of 1.0:
  // big endian: 3f f0 00 00 00 00 00 00
  // little endian: 00 00 00 00 00 00 f0 3f
  // arm fpa little endian: 00 00 f0 3f 00 00 00 00
  double d = 1.0;
  byte *b = (byte*)&d;
  return (b [0] == 0);
}

Ecotone

Yes, I understand that this is hardcoded, but my question is why there is actually piece of code that accounts for both true and false. – Everywhere

I also e-mailed the BCL team. – Ecotone

did you hear back from BCL team? – Everywhere

@Alex Aza: Not yet. I had actually thought of e-mailing Eric Lippert first, who is likely to have an answer but focuses on a different area. So I e-mailed the appropriate team first. I'm inclined to wait more than 1 week, though. – Ecotone