Bit field padding in C

Question

Continuing my experiments in C, I wanted to see how bit fields are placed in memory. I'm working on Intel 64 bit machine. Here is my piece of code:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
int main(int argc, char **argv) {
       struct box_props {
         unsigned int opaque       : 1;
         unsigned int fill_color   : 3;
         unsigned int              : 4; 
         unsigned int show_border  : 1;
         unsigned int border_color : 3;
         unsigned int border_style : 2;
         unsigned int              : 2; 
       };
       
       struct box_props s;
       memset(&s, 0, 32);
       s.opaque = 1;
       s.fill_color = 7;
       s.show_border = 1;
       s.border_color = 7;
       s.border_style = 3;
    
       printf("sizeof box_porps: %d sizeof unsigned int: %d\n", sizeof(struct box_props), sizeof(unsigned int));
       char *ptr = (char *)&s;
       for (int i=0; i < sizeof(struct box_props); i++) {
          printf("%x = %x\n", ptr + i, *(ptr + i));
       }

       return 0;
}

and here is an output:

sizeof box_porps: 4 sizeof unsigned int: 4
5be6e2f0 = f
5be6e2f1 = 3f
5be6e2f2 = 0
5be6e2f3 = 0

And here is the question: why does struct box_props have size 4 - can't it just be 2 bytes? How is the padding done in that case? I'm a bit (nomen omen) confused with it.

Ikon

I think this is because you are using type int, try using short int if you want size 2.try and send a feedback – Propjet

memset(&s, 0, 32); Ouch! You ask memset to zero out 32 bytes, but have only four. You're just unlucky that didn't destroy some vital part of the stack and caused your programme to crash violently. – Spoony

Answer 1

Even though the total requirement is just 2 Bytes (1+3+4+1+3+2+2) in this case, the size of the data type used (unsigned int) is 4 bytes. So the allocated memory is also 4 Bytes. If you want just 2 Bytes allocated use unsigned short as your data type and run the program again.

Hoggish

Answer 2

From the ISO C standard:

An implementation may allocate any addressable storage unit large enough to hold a bit- field. (and later) There may be unnamed padding at the end of a structure or union.

So there's no requirement to always choose the smallest possible chunk of memory for the struct. And since 32-bit words are probably the native size for your compiler, that's what it chooses.

Cioban

The normal behavior I see from compilers nowadays is to allocate things according to the type of the bitfield. So uint32_t foo:4; will allocate foo four bits from a storage location of type uint32_t; unless four bits remain from an immediately-preceding bitfield of the same type, it will allocate another uint32_t. I dislike those rules, but they are what they are. – Uninstructed

Answer 3

The placement of bit-fields in memory depends not only on how the compiler decides to assign the various fields within the structure but also on the endian-ness of the machine on which you are running. Lets take these one-by-one. The assignment of the fields within the compiler can be controlled by specifying the size of the field (as @DDD) points out, but also through another mechanism. You can tell the compiler to either pack your structure or leave it better suited for how the compiler may want to optimize for the machine architecture for which you are compiling. Packing is specified using the packed type attribute. Thus, if you specify the struct as:

struct __attribute__ ((__packed__)) box_props {
    ...
} 

you may well see a different layout in memory. Note that you won't see the layout being different by examing the structure components - its the layout in memory that could change. Packing a structure is crucially important when communicating with something else such as an IO device which expects specific bits at specific places.

The second issue with bit field structures is their layout dependence on endian-ness. The layout of the struct in memory (or any data for that matter) depends on whether you are running on a big-endian (POWER) or little-endian (e.g., x86) machine. Some systems (e.g., embedded PowerPC systems are bi-endian).

In general, bit fields make it very hard to port code because you are futzing with the layout of data in memory.

Hope this helps!

Roentgenotherapy

Answer 4

For some reason I do not fathom, the implementers of the C standard decided that specifying a numberic type along with a bitfield should allocate space sufficient to hold that numeric type unless the previous field was a bitfield, allocated out of the same type, which had enough space left over to handle the next field.

For your particular example, on a machine with 16 bit unsigned shorts, you should change the declarations in your bitfield to unsigned shorts. As it happens, unsigned char would also work, and yield the same results, but that is not always the case. If optimally-packed bitfields would straddle char boundaries but not short boundaries, then declaring bitfields as unsigned char would require padding to avoid such straddling.

Although some processors would have no trouble generating code for bitfields which straddle storage-unit boundaries, the present C standard would forbid packing them that way (again, for reasons I do not fathom). On a machine with typical 8/16/32/64-bit data types, for example, a compiler could not allow a programmer to specify a 3-byte structure containing eight three-byte fields, since the fields would have to straddle byte boundaries. I could understand the spec not requiring compilers to handle fields that straddle boundaries, or requiring that bitfields be laid out in some particular fashion (I'd regard them as infinitely more useful if one could specify that a particular bitfield should e.g. use bits 4-7 of some location), but the present standard seems to give the worst of both worlds.

In any case, the only way to use bitfields efficiently is to figure out where storage unit boundaries are, and choose types for the bitfields suitably.

PS--It's interesting to note that while I recall compilers used to disallow volatile declarations for structures containing bitfields (since the sequence of operations when writing a bitfield may not be well defined), under the new rules the semantics could be well defined (I don't know if the spec actually requires them). For example, given:

typedef struct {
  uint64_t b0:8,b1:8,b2:8,b3:8, b4:8,b5:8,b6:8,b7:8;
  uint64_t b8:8,b9:8,bA:8,bB:8, bC:8,bD:8,bE:8,bF:8;
} FOO;
extern volatile FOO bar;

the statement bar.b3 = 123; will read the first 64 bits from bar, and then write the first 64 bits of bar with an updated value. If bar were not volatile, a compiler might replace that sequence with a simple 8-bit write, but bar could be something like a hardware register which can only be written in 32-bit or 64-bit chunks.

If I had my druthers, it would be possible to define bitfields using something like:

typedef struct {
  uint32_t {
    baudRate:13=0, dataFormat:3,
    enableRxStartInt: 1=28, enableRxDoneInt: 1, enableTxReadyInt: 1, enableTxEmptyInt: 1;};
  };
} UART_CONTROL;

indicating that baudRate is 13 bits starting at bit 0 (the LSB), dataFormat is 3 bits starting after baudRate, enableRxStartInt is bit 28, etc. Such a syntax would allow many types of data packing and unpacking to be written in portable fashion, and would allow many I/O register manipulations to be done in a compiler-agnostic fashion (though such code would obviously be hardware-specific).

Uninstructed

“the implementers of the C standard decided that specifying a numberic type along with a bitfield should allocate space sufficient to hold that numeric type […]” What clause of the C standard is that? C99 6.7.2.1:3 says “the width of a bit-field shall be an integer constant expression with a nonnegative value that does not exceed the width of an object of the type that would be specified were the colon and expression omitted” which is a very different constraint than the one you are describing. – Somewhere

Also 6.7.2.1:10: “An implementation may allocate any addressable storage unit large enough to hold a bit-field” – Somewhere

@PascalCuoq: In some older C compilers, I believe the only aspect of a bitfield's type the compiler would care about was whether it was signed or unsigned, but on the newer compilers I've seen any particular bitfield will always be allocated as part of a storage location of its indicated type. For example, if one has a volatile struct with an 8-bit bitfield of type uint64_t, even if it's byte-aligned, the generated code will read the 64-bit storage unit containing the bitfield, change 8 bits of it, and write all 64 bits back. – Uninstructed

And 6.7.2.1 (11) says: "If insufficient space remains, whether a bit-field that does not fit is put into the next unit or overlaps adjacent units is implementation-defined.", thus straddling is explicitly allowed. gcc and clang straddle bit-fields and use smaller units of storage than would be necessary for the named type if you tell them to by using __attribute__((packed)) at least on some architectures. – Spoony

@DanielFischer: Ah, thanks. I hadn't seen that part. BTW, what do you think of my syntax idea? – Uninstructed

That depends on what you consider your syntax idea. I don't think = bit_position is a particularly good way to specify it (though I don't have a better idea). If you mean the general idea of being able to specify the exact layout of bit-field sequences, that would, I believe, be quite useful sometimes. But it ain't gonna happen, the committee is quite loath to restrict implementations thus strongly, as far as I can tell. – Spoony

@DanielFischer: I was thinking of it being like enum, which assigns either the value specified with =, or the next value if none is specified. As for the idea of explicit struct layout "restricting implementations", the spec for C has always required that CHAR_BITS be at least 8, and even systems where CHAR_BITS is greater than 8 often have to communicate over 8-bit links. It will thus often be necessary to convert data into a known format. With standards as they are, I think using a function to pack or extract things is cleaner than using a union, but... – Uninstructed

...such an approach won't allow compilers much room to optimize things. If one wants to write a value which is known to be in the range 0 to 4095, to bits 14-19 of a 32-bit field, saying field = (field & ~0x000FC000) | (value << 14); may be the most efficient portable way to do it, but on processors with bit-field-manipulation instructions a bit-field store might be more efficient. Note that with the code written as above a compiler could not perform the optimization unless it could prove value would never exceed 4095. – Uninstructed

Answer 5

Topic starter's example is very hard... I want to add my very simple example.

#include <stdio.h>
#include <stdint.h>

typedef union{
    uint32_t value;
    struct //__attribute__((packed))
    {
        uint8_t data_1       : 7;
        uint8_t data_2       : 2;
    }field;
}reg_t;

int main()
{
    reg_t data= {};
    data.field.data_1= 0;
    data.field.data_2= 1;
    printf("0x%X\n", data.value);
    return 0;
}

Result is 0x100

#include <stdio.h>
#include <stdint.h>

typedef union{
    uint32_t value;
    struct __attribute__((packed))
    {
        uint8_t data_1       : 7;
        uint8_t data_2       : 2;
    }field;
}reg_t;

int main()
{
    reg_t data= {};
    data.field.data_1= 0;
    data.field.data_2= 1;
    printf("0x%X\n", data.value);
    return 0;
}

Result is 0x80.

And don't forget that bit-field is endian-ness of the machine, my X86_64 is little-endian so in my PC's memory byte order is not human-readable. This is picture where result is 0x100.

Cwm

Answer 6

As others said, the size of the structure depends upon many factors, e.g. the size of the structure elements and the packing used.

If the compiler lets you, setting a packed attribute would force using the smallest size available in the current architecture for the structure; if no packing is explicitly set, a default size will be used for packing, that happens to be 4 bytes in your setup.

So even if 2 bytes would be enough to accommodate your bits, the structure will be reserved 4 bytes because this is the value used for the default packing.

Please note that your memset call uses a fixed value of 32 for the length: this is not correct.

Following an example, that on my setup would change the size of the box_props structure from 4 to 2 bytes depending upon the absence/presence of the packed attribute:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
int main(int argc, char **argv) {
        struct box_props {
          unsigned int opaque       : 1;
          unsigned int fill_color   : 3;
          unsigned int              : 4;
          unsigned int show_border  : 1;
          unsigned int border_color : 3;
          unsigned int border_style : 2;
          unsigned int              : 2;
        }__attribute((packed));

       struct box_props s;
       memset(&s, 0, sizeof(s));
       s.opaque = 1;
       s.fill_color = 7;
       s.show_border = 1;
       s.border_color = 7;
       s.border_style = 3;

       printf("sizeof box_porps: %zu\nsizeof unsigned int: %zu\n", sizeof(struct box_props), sizeof(unsigned int));
       char *ptr = (char *)&s;
       for (int i=0; i < sizeof(struct box_props); i++) {
          printf("[%p] = 0x%02x\n", ptr + i, *(ptr + i));
       }

       return 0;
}

Output:

sizeof box_porps: 2
sizeof unsigned int: 4
[0x7ffc750a75f6] = 0x0f
[0x7ffc750a75f7] = 0x3f

Beauty