Convert FP32 to Bfloat16 in C++

Question

How can I convert from float (1bit sign, 8bit exp, 23bit mantissa) to Bfloat16 (1bit sign, 8bit exp, 7bit mantissa) in C++?

Caudal

frexp can be used to break a float down into components. Assembling it back into whatever structure you call Bfloat16 is left as an exercise for the reader. – Nikolas

I imagine you want to do this efficiently, since the only reason for such a small floating point format is when you have a very large number of them. I also imagine it needs to do proper rounding. – Designate

@IgorTandetnik but that would be expensive. Bfloat16 is designed as the top half of float so that you can truncate it easily – Compulsion

Answer 1

As demonstrated in the answer by Botje it is sufficient to copy the upper half of the float value since the bit patterns are the same. The way it is done in that answer violates the rules about strict aliasing in C++. The way around that is to use memcpy to copy the bits.

static inline tensorflow::bfloat16 FloatToBFloat16(float float_val)
{
    tensorflow::bfloat16 retval;
#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
    memcpy(&retval, &float_val, sizeof retval);
#else
    memcpy(&retval, reinterpret_cast<char *>(&float_val) + sizeof float_val - sizeof retval, sizeof retval);
#endif
    return retval;
}

If it's necessary to round the result rather than truncating it, you can multiply by a magic value to push some of those lower bits into the upper bits.

float_val *= 1.001957f;

Designate

Answer 2

memcpy wouldn't compile for me in the little endian case for some reason. This is my solution. I have it as a struct here so that I can easily access the data and run through different ranges of values to confirm that it works properly.

struct bfloat16{
   unsigned short int data;
   public:
   bfloat16(){
      data = 0;
   }
   //cast to float
   operator float(){
      unsigned int proc = data<<16;
      return *reinterpret_cast<float*>(&proc);
   }
   //cast to bfloat16
   bfloat16& operator =(float float_val){
      data = (*reinterpret_cast<unsigned int *>(&float_val))>>16;
      return *this;
   }
};

//an example that enumerates all the possible values between 1.0f and 300.0f
using namespace std;

int main(){
   bfloat16 x;
   for(x = 1.0f; x < 300.0f; x.data++){
      cout<<x.data<<" "<<x<<endl;
   }
   
   return 0;
}

Incantatory

Works great :). Do check this answer, which also include operator >> overload for cin: https://mcmap.net/q/343753/-why-is-there-no-2-byte-float-and-does-an-implementation-already-exist – Furlana

using reinterpret_cast for type punning invokes undefined behavior?. You need to use bit_cast instead: Why was std::bit_cast added, if reinterpret_cast could do the same? – Compulsion

Answer 3

From the Tensorflow implementation:

static inline tensorflow::bfloat16 FloatToBFloat16(float float_val) {
#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
    return *reinterpret_cast<tensorflow::bfloat16*>(
        reinterpret_cast<uint16_t*>(&float_val));
#else
    return *reinterpret_cast<tensorflow::bfloat16*>(
        &(reinterpret_cast<uint16_t*>(&float_val)[1]));
#endif
}

Tien

I think that implementation is flawed because it violates aliasing. If you copy the two bytes as individual bytes (e.g. unsigned char's), it'll be right. – Bluebonnet

@AlexeyFrunze : why is de-referencing violating aliasing ? It seems like it should be fine since we never update/modify or write to float_val , if we are just reading and never writing to the different pointer type , is it still a violation or UB ? – Riendeau

@Riendeau Doesn't matter. You can legally peek/poke an object through a pointer to a compatible type, a pointer to the (un)signed variant of the same type, or a pointer to a char. float and uint16_t is not an allowed pair here (unless uint16_t is a type name that stands for unsigned char). – Bluebonnet

@AlexeyFrunze : got it . thanks . Could you please give an example why such a rule was decided. Understand that this is what standard says but do not understand the rationale and what can possibly go wrong , any code snippet or example would help – Riendeau

@Riendeau For one, different types may have different alignments and using a badly aligned pointer may not work how one may expect it to (it may not work at all, just crash your program). For another, this rule gives the compiler the ability to know that certain pointers in the code never point to the same object and the compiler can safely make optimizations (like caching the object in a register, knowing it won't mysteriously change through some other pointer). – Bluebonnet

@AlexeyFrunze agreed about alignment part. But the second part , how does that get violated if we are doing such reinterpret_cast to a read only memory location , If my program never writes or updates that memory location , and this is purely a read only storage , is it ok to apply reinterpret cast (assuming i make sure the memory are aligned). – Riendeau

@Riendeau The language standard does not always state the rationale. Try asking its authors or looking for their discussions and things like the language defect reports. I know, it's not always clear and logical but that's what it is. – Bluebonnet

@Riendeau Is reinterpret_cast type punning actually undefined behavior? – Compulsion