Setting extra bits in a bool makes it true and false at the same time

Question

If I get a bool variable and set its second bit to 1, then variable evaluates to true and false at the same time. Compile the following code with gcc6.3 with -g option, (gcc-v6.3.0/Linux/RHEL6.0-2016-x86_64/bin/g++ -g main.cpp -o mytest_d) and run the executable. You get the following.

How can T be equal to true and false at the same time?

       value   bits 
       -----   ---- 
    T:   1     0001
after bit change
    T:   3     0011
T is true
T is false

This can happen when you call a function in a different language (say fortran) where true and false definition is different than C++. For fortran if any bits are not 0 then the value is true, if all bits are zero then the value is false.

#include <iostream>
#include <bitset>

using namespace std;

void set_bits_to_1(void* val){
  char *x = static_cast<char *>(val);

  for (int i = 0; i<2; i++ ){
    *x |= (1UL << i);
  }
}

int main(int argc,char *argv[])
{

  bool T = 3;

  cout <<"       value   bits " <<endl;
  cout <<"       -----   ---- " <<endl;
  cout <<"    T:   "<< T <<"     "<< bitset<4>(T)<<endl;

  set_bits_to_1(&T);


  bitset<4> bit_T = bitset<4>(T);
  cout <<"after bit change"<<endl;
  cout <<"    T:   "<< T <<"     "<< bit_T<<endl;

  if (T ){
    cout <<"T is true" <<endl;
  }

  if ( T == false){
    cout <<"T is false" <<endl;
  }


}

/////////////////////////////////// // Fortran function that is not compatible with C++ when compiled with ifort.

       logical*1 function return_true()
         implicit none

         return_true = 1;

       end function return_true

Asclepius

With undefined behavior, anything is possible :) – Eydie

not quite a duplicate of Does the C++ standard allow for an uninitialized bool to crash a program? where my answer explains that the x86-64 System V ABI specifies that bool is a 0 or 1, and thus the compiler is allowed to assume this when emitting code. – Cloven

There's a tension between the mathematics of booleans and their computer science representations. Mathematically, booleans have two values, so a single bit. The problem is that in c++, bools have to be addressable, but individual bits are not addressable. The standard requires implementations to make all boolean operations result in a zero or a one. Anything else is a noncompliant implementation. The standard also requires programmers to follow this rule. In particular, intentionally setting bits so that a bool has a value this is neither zero nor one is undefined behavior. – Mediant

Here is minimal example. GCC up to 8.3 behaves the same as in the post. With 9.1 it's different, but still is surprising. (C++ is fun!) – Shafting

TAKE AWAY: Don't use bool/logical types if using multiple languages in a program. Each language has their own standard about bits of bool. Use int or other types that have same definition in all languages (if that is even possible) – Asclepius

"Doctor, it hurts when I do this." – Seth

@DennisWilliamson If "this" is "walking", it's reasonable to want an explanation rather than a "don't walk then" response from your doctor. – Colonize

Answer 1

In C++ the bit representation (and even the size) of a bool is implementation defined; generally it's implemented as a char-sized type taking 1 or 0 as possible values.

If you set its value to anything different from the allowed ones (in this specific case by aliasing a bool through a char and modifying its bit representation), you are breaking the rules of the language, so anything can happen. In particular, it's explicitly specified in the standard that a "broken" bool may behave as both true and false (or neither true nor false) at the same time:

Using a bool value in ways described by this International Standard as “undefined,” such as by examining the value of an uninitialized automatic object, might cause it to behave as if it is neither true nor false

(C++11, [basic.fundamental], note 47)

---

In this particular case, you can see how it ended up in this bizarre situation: the first if gets compiled to

    movzx   eax, BYTE PTR [rbp-33]
    test    al, al
    je      .L22

which loads T in eax (with zero extension), and skips the print if it's all zero; the next if instead is

    movzx   eax, BYTE PTR [rbp-33]
    xor     eax, 1
    test    al, al
    je      .L23

The test if(T == false) is transformed to if(T^1), which flips just the low bit. This would be ok for a valid bool, but for your "broken" one it doesn't cut it.

Notice that this bizarre sequence is only generated at low optimization levels; at higher levels this is generally going to boil down to a zero/nonzero check, and a sequence like yours is likely to become a single test/conditional branch. You will get bizarre behavior anyway in other contexts, e.g. when summing bool values to other integers:

int foo(bool b, int i) {
    return i + b;
}

becomes

foo(bool, int):
        movzx   edi, dil
        lea     eax, [rdi+rsi]
        ret

where dil is "trusted" to be 0/1.

---

If your program is all C++, then the solution is simple: don't break bool values this way, avoid messing with their bit representation and everything will go well; in particular, even if you assign from an integer to a bool the compiler will emit the necessary code to make sure that the resulting value is a valid bool, so your bool T = 3 is indeed safe, and T will end up with a true in its guts.

If instead you need to interoperate with code written in other languages that may not share the same idea of what a bool is, just avoid bool for "boundary" code, and marshal it as an appropriately-sized integer. It will work in conditionals & co. just as fine.

---

Update about the Fortran/interoperability side of the issue

Disclaimer all I know of Fortran is what I read this morning on standard documents, and that I have some punched cards with Fortran listings that I use as bookmarks, so go easy on me.

First of all, this kind of language interoperability stuff isn't part of the language standards, but of the platform ABI. As we are talking about Linux x86-64, the relevant document is the System V x86-64 ABI.

First of all, nowhere is specified that the C _Bool type (which is defined to be the same as C++ bool at 3.1.2 note †) has any kind of compatibility with Fortran LOGICAL; in particular, at 9.2.2 table 9.2 specifies that "plain" LOGICAL is mapped to signed int. About TYPE*N types it says that

The “TYPE*N” notation specifies that variables or aggregate members of type TYPE shall occupy N bytes of storage.

(ibid.)

There's no equivalent type explicitly specified for LOGICAL*1, and it's understandable: it's not even standard; indeed if you try to compile a Fortran program containing a LOGICAL*1 in Fortran 95 compliant mode you get warnings about it, both by ifort

./example.f90(2): warning #6916: Fortran 95 does not allow this length specification.   [1]

    logical*1, intent(in) :: x

------------^

and by gfort

./example.f90:2:13:
     logical*1, intent(in) :: x
             1
Error: GNU Extension: Nonstandard type declaration LOGICAL*1 at (1)

so the waters are already muddled; so, combining the two rules above, I'd go for signed char to be safe.

However: the ABI also specifies:

The values for type LOGICAL are .TRUE. implemented as 1 and .FALSE. implemented as 0.

So, if you have a program that stores anything besides 1 and 0 in a LOGICAL value, you are already out of spec on the Fortran side! You say:

A fortran logical*1 has same representation as bool, but in fortran if bits are 00000011 it is true, in C++ it is undefined.

This last statement is not true, the Fortran standard is representation-agnostic, and the ABI explicitly says the contrary. Indeed you can see this in action easily by checking the output of gfort for LOGICAL comparison:

integer function logical_compare(x, y)
    logical, intent(in) :: x
    logical, intent(in) :: y
    if (x .eqv. y) then
        logical_compare = 12
    else
        logical_compare = 24
    end if
end function logical_compare

becomes

logical_compare_:
        mov     eax, DWORD PTR [rsi]
        mov     edx, 24
        cmp     DWORD PTR [rdi], eax
        mov     eax, 12
        cmovne  eax, edx
        ret

You'll notice that there's a straight cmp between the two values, without normalizing them first (unlike ifort, that is more conservative in this regard).

Even more interesting: regardless of what the ABI says, ifort by default uses a nonstandard representation for LOGICAL; this is explained in the -fpscomp logicals switch documentation, which also specifies some interesting details about LOGICAL and cross-language compatibility:

Specifies that integers with a non-zero value are treated as true, integers with a zero value are treated as false. The literal constant .TRUE. has an integer value of 1, and the literal constant .FALSE. has an integer value of 0. This representation is used by Intel Fortran releases before Version 8.0 and by Fortran PowerStation.

The default is fpscomp nologicals, which specifies that odd integer values (low bit one) are treated as true and even integer values (low bit zero) are treated as false.

The literal constant .TRUE. has an integer value of -1, and the literal constant .FALSE. has an integer value of 0. This representation is used by Compaq Visual Fortran. The internal representation of LOGICAL values is not specified by the Fortran standard. Programs which use integer values in LOGICAL contexts, or which pass LOGICAL values to procedures written in other languages, are non-portable and may not execute correctly. Intel recommends that you avoid coding practices that depend on the internal representation of LOGICAL values.

(emphasis added)

Now, the internal representation of a LOGICAL normally shouldn't a problem, as, from what I gather, if you play "by the rules" and don't cross language boundaries you aren't going to notice. For a standard compliant program there's no "straight conversion" between INTEGER and LOGICAL; the only way I see you can shove an INTEGER into a LOGICAL seem to be TRANSFER, which is intrinsically non-portable and give no real guarantees, or the non-standard INTEGER <-> LOGICAL conversion on assignment.

The latter one is documented by gfort to always result in nonzero -> .TRUE., zero -> .FALSE., and you can see that in all cases code is generated to make this happen (even though it's convoluted code in case of ifort with the legacy representation), so you cannot seem to shove an arbitrary integer into a LOGICAL in this way.

logical*1 function integer_to_logical(x)
    integer, intent(in) :: x
    integer_to_logical = x
    return
end function integer_to_logical

integer_to_logical_:
        mov     eax, DWORD PTR [rdi]
        test    eax, eax
        setne   al
        ret

The reverse conversion for a LOGICAL*1 is a straight integer zero-extension (gfort), so, to be honoring the contract in the documentation linked above, it's clearly expecting the LOGICAL value to be 0 or 1.

But in general, the situation for these conversions is a bit of a mess, so I'd just stay away from them.

---

So, long story short: avoid putting INTEGER data into LOGICAL values, as it is bad even in Fortran, and make sure to use the correct compiler flag to get the ABI-compliant representation for booleans, and interoperability with C/C++ should be fine. But to be extra safe, I'd just use plain char on the C++ side.

Finally, from what I gather from the documentation, in ifort there is some builtin support for interoperability with C, including booleans; you may try to leverage it.

Diogenes

Thanks for detailed explanation. I think C++ standard definition is very dangerous and leaves a lot of room for error when used with other languages. Fortran defines bool to be true if any bit is 1 and false if all are zero. If bool has 8 bits standard needs to clearly define what happens when other bits are set. You cannot assume that people will never touch other bits. People develop software with many languages combined and you need to safely cover these cases. Undefined behavior because some bits are set to one is simply not an acceptable standard in my view. I call this stupid standard. – Asclepius

If you will use this standard then require bool to be 1 bit not 8 bits – Asclepius

This is like having an 8 lanes highway, all lanes are open, but if you use any lane other than the first you will have a guaranteed accident. – Asclepius

bool effectively is one bit, it's just not implemented that way under the hood. How many bits are used to represent is an implementation detail (part of the ABI), not one that is defined by the language. I do not understand why this is a problem in the real world, although it does make for a great Stack Overflow Q&A. I do lots of interop between C++ and code in other languages, and I've never had a problem. @Asclepius – Gouveia

Try using gcc with intel fortran compiler. Have a fortran function that returns a logical*1 type variable that is a 8 bit bool and assign it to a C++ bool and see what happens :) – Asclepius

I'm not a Fortran expert by any stretch of the imagination, but I don't see why that would be a problem. As far as I can tell, a LOGICAL*1 type variable has compatible semantics with a C++ bool: non-zero values are "true", while zero values are "false". I believe that Fortran actually represents "true" by setting all bits, whereas a C++ implementation will typically only bother to set the lowest-order bit, but that doesn't matter, as long as "false" is defined as "all bits zero". Alternatively, there are ways to make Intel Fortran fully compliant with C; see logical(c_bool). @Asclepius – Gouveia

@Cody Gray , A fortran logical*1 has same representation as bool, but in fortran if bits are 00000011 it is true, in C++ it is undefined. That is the problem. You cannot have fortran return a logical*1, and assign that to a C++ bool. C++ bool becomes undefined, see my code above. (I am manipulating bits to mimic what fortran does and how C++ treats it) – Asclepius

@Asclepius IT looks like you don't know how to properly interop between fortran and C++. Don't blame your ignorance on the C++ standard. Standards exists precisely because without them any implementation could do anything they wont and interoperability would be impossible. – Run

BTW this kind of interoperability isn't any of the C++ or Fortran standard business, but of the platform ABI. In case of x86-64 Linux, it's uclibc.org/docs/psABI-x86_64.pdf; it specifies that plain LOGICAL is mapped to signed int (so, it's 4 byte, like Win32 BOOL), while there's no explicit mapping for LOGICAL*1. From the fact that "The "TYPE*N" notation specifies that variables or aggregate members of type TYPE shall occupy N bytes of storage", it should follow that the type to be mapped should probably be signed char. – Diogenes

Besides, it also says that LOGICAL should have only 1 and 0 as possible values, which makes me wonder how you obtained the broken data even on the Fortran side: from what I gather, LOGICAL should work similarly to C++ bool: the actual content is always 0 or 1, it's on assignment that nonzero integer values are forced to become 1. But again, this is just what I seem to gather from a quick read of the spec, so I may be wrong. – Diogenes

This seems hinted by the last message of this post: even in Fortran you are not expected to mess with LOGICAL bits setting them to any value you like. – Diogenes

@BY408: starting from this, I expanded a bit my answer, you may find it useful to be aware of some extra culprits about LOGICAL, interoperatbility and even portability inside Fortran itself. – Diogenes

Fortran function added above. – Asclepius

re the quote about bool fiddling (unfortunately a moderator nuked our prior comments on the subject for no reason): github.com/cplusplus/draft/commit/… #sadface – Colonize

@LightnessRacesinOrbit: once that we had a note that didn't add confusion, they decide to remove it! :-( – Diogenes

@MatteoItalia IKR! – Colonize

Answer 2

This is what happens when you violate your contract with both the language and the compiler.

You probably heard somewhere that "zero is false", and "non-zero is true". That holds when you stick to the language's parameters, statically converting an int to bool or vice versa.

It does not hold when you start messing with bit representations. In that case, you break your contract, and enter the realm of (at the very least) implementation-defined behaviour.

Simply don't do that.

It's not up to you how a bool is stored in memory. It's up to the compiler. If you want to change a bool's value, either assign true/false, or assign an integer and use the proper conversion mechanisms provided by C++.

---

The C++ standard used to actually give a specific call-out to how using bool in this manner is naughty and bad and evil ("Using a bool value in ways described by this document as 'undefined',such as by examining the value of an uninitialized automatic object, might cause it to behave as if it is neither true nor false."), though it was removed in C++20 for editorial reasons.

Colonize

It's up to the compiler. It's up to "the implementation" in C++ terms. On most platforms (including x86-64 GNU/Linux), compilers all follow an ABI (x86-64 System V) which is a separate document from the compiler. It's not up to the compiler how a bool is stored in memory, that's specified by the ABI (other than private bool objects that nothing outside the function can ever see; then the as-if rule is in full force). "Up to the compiler" is a useful simplification, but it's not really true, especially for compilers other than GCC (because GCC devs designed the x86-64 System V ABI.) – Cloven

@PeterCordes That's correct. As you say, it's a useful simplification, and completely warranted in this context IMO. – Colonize

My answer on the uninitialized-bool UB question you linked covers all that, though :) +1 don't mess with the object-representation of bool. – Cloven

I won't mess up with bits in my code, but this problem happens because I had a fortran function that returns a logical type (which is the same size as bool) and assigned that return value to a C++ bool. Then C++ started to act as it does above. I added that bit manipulation to mimic what fortran returned as true to save you the hassle of needing two compilers to reproduce the problem. What fortran assign to a variable as true is not true when you return that and assign it to a C++ bool. And it is not intuitive why standard will not accept non-zero bits as true. – Asclepius

@BY408, if you need a type to handle values other than 1 and 0, then you shouldn't use a bool. Use an unsigned char or int or other number variable. A unsigned char used in an if() statement behaves the way you want (zero is false, nonzero is true). – Tenrec

@Asclepius Such a rule would make code necessarily slower, for no real gain. Matteo shows how the implementations that the standard permits can be super fast! Thanks, standard! – Colonize

@Asclepius can you show the code of this Fortran function? – Diogenes

@LightnessRacesinOrbit: If the Standard had specified that each read of a bool that has had a non-zero even number into a bool may independently treat it as yielding any integer value and specified that implementations need not allow code to take the address of __bool objects, that would have made the type compatible with many implementations' pre-existing bit types, and would have allowed compilers to generate optimal code in many circumstances where they presently cannot. – Commitment

@supercat: I don't follow. Can you rephrase? – Colonize

@LightnessRacesinOrbit: Suppose unsigned f(unsigned) is external function of whose return value is known by the programmer to always be 0 or 1 when passed an argument of 2. Given bool b = f(2); a C99 compiler would be required to generate machine code to check whether the value is zero and force it to 1 if not. If the Standard had been written with optimization in mind, it could simply store the return value of f directly. Further, a fair number of compilers prior to C99 had a bit type which behaved like a single-bit bitfield with the same truncation behavior as other unsigned types. – Commitment

@Commitment That seems like an edge case when compared to every single int-to-bool conversion ever. – Colonize

@LightnessRacesinOrbit: The same issue would apply in any situation where the programmer knows that an integer-type value will be 0 or 1 in all cases where the value in the bool would matter, but a compiler would have no way of knowing that. – Commitment

@Commitment There are intrinsics you can use for edge cases like that to signal that you know something the compiler doesn't – Colonize