Is gcc's __attribute__((packed)) / #pragma pack unsafe?
Asked Answered
H

5

202

In C, the compiler will lay out members of a struct in the order in which they're declared, with possible padding bytes inserted between members, or after the last member, to ensure that each member is aligned properly.

gcc provides a language extension, __attribute__((packed)), which tells the compiler not to insert padding, allowing struct members to be misaligned. For example, if the system normally requires all int objects to have 4-byte alignment, __attribute__((packed)) can cause int struct members to be allocated at odd offsets.

Quoting the gcc documentation:

The `packed' attribute specifies that a variable or structure field should have the smallest possible alignment--one byte for a variable, and one bit for a field, unless you specify a larger value with the `aligned' attribute.

Obviously the use of this extension can result in smaller data requirements but slower code, as the compiler must (on some platforms) generate code to access a misaligned member a byte at a time.

But are there any cases where this is unsafe? Does the compiler always generate correct (though slower) code to access misaligned members of packed structs? Is it even possible for it to do so in all cases?

Hilleary answered 19/12, 2011 at 22:28 Comment(1)
The gcc bug report is now marked as FIXED with the addition of a warning on the pointer assignment (and an option to disable the warning). Details in my answer.Hilleary
H
186

Yes, __attribute__((packed)) is potentially unsafe on some systems. The symptom probably won't show up on an x86, which just makes the problem more insidious; testing on x86 systems won't reveal the problem. (On the x86, misaligned accesses are handled in hardware; if you dereference an int* pointer that points to an odd address, it will be a little slower than if it were properly aligned, but you'll get the correct result.)

On some other systems, such as SPARC, attempting to access a misaligned int object causes a bus error, crashing the program.

There have also been systems where a misaligned access quietly ignores the low-order bits of the address, causing it to access the wrong chunk of memory.

Consider the following program:

#include <stdio.h>
#include <stddef.h>
int main(void)
{
    struct foo {
        char c;
        int x;
    } __attribute__((packed));
    struct foo arr[2] = { { 'a', 10 }, {'b', 20 } };
    int *p0 = &arr[0].x;
    int *p1 = &arr[1].x;
    printf("sizeof(struct foo)      = %d\n", (int)sizeof(struct foo));
    printf("offsetof(struct foo, c) = %d\n", (int)offsetof(struct foo, c));
    printf("offsetof(struct foo, x) = %d\n", (int)offsetof(struct foo, x));
    printf("arr[0].x = %d\n", arr[0].x);
    printf("arr[1].x = %d\n", arr[1].x);
    printf("p0 = %p\n", (void*)p0);
    printf("p1 = %p\n", (void*)p1);
    printf("*p0 = %d\n", *p0);
    printf("*p1 = %d\n", *p1);
    return 0;
}

On x86 Ubuntu with gcc 4.5.2, it produces the following output:

sizeof(struct foo)      = 5
offsetof(struct foo, c) = 0
offsetof(struct foo, x) = 1
arr[0].x = 10
arr[1].x = 20
p0 = 0xbffc104f
p1 = 0xbffc1054
*p0 = 10
*p1 = 20

On SPARC Solaris 9 with gcc 4.5.1, it produces the following:

sizeof(struct foo)      = 5
offsetof(struct foo, c) = 0
offsetof(struct foo, x) = 1
arr[0].x = 10
arr[1].x = 20
p0 = ffbff317
p1 = ffbff31c
Bus error

In both cases, the program is compiled with no extra options, just gcc packed.c -o packed.

(A program that uses a single struct rather than array doesn't reliably exhibit the problem, since the compiler can allocate the struct on an odd address so the x member is properly aligned. With an array of two struct foo objects, at least one or the other will have a misaligned x member.)

(In this case, p0 points to a misaligned address, because it points to a packed int member following a char member. p1 happens to be correctly aligned, since it points to the same member in the second element of the array, so there are two char objects preceding it -- and on SPARC Solaris the array arr appears to be allocated at an address that is even, but not a multiple of 4.)

When referring to the member x of a struct foo by name, the compiler knows that x is potentially misaligned, and will generate additional code to access it correctly.

Once the address of arr[0].x or arr[1].x has been stored in a pointer object, neither the compiler nor the running program knows that it points to a misaligned int object. It just assumes that it's properly aligned, resulting (on some systems) in a bus error or similar other failure.

Fixing this in gcc would, I believe, be impractical. A general solution would require, for each attempt to dereference a pointer to any type with non-trivial alignment requirements either (a) proving at compile time that the pointer doesn't point to a misaligned member of a packed struct, or (b) generating bulkier and slower code that can handle either aligned or misaligned objects.

I've submitted a gcc bug report. As I said, I don't believe it's practical to fix it, but the documentation should mention it (it currently doesn't).

UPDATE: As of 2018-12-20, this bug is marked as FIXED. The patch will appear in gcc 9 with the addition of a new -Waddress-of-packed-member option, enabled by default.

When address of packed member of struct or union is taken, it may result in an unaligned pointer value. This patch adds -Waddress-of-packed-member to check alignment at pointer assignment and warn unaligned address as well as unaligned pointer

I've just built that version of gcc from source. For the above program, it produces these diagnostics:

c.c: In function ‘main’:
c.c:10:15: warning: taking address of packed member of ‘struct foo’ may result in an unaligned pointer value [-Waddress-of-packed-member]
   10 |     int *p0 = &arr[0].x;
      |               ^~~~~~~~~
c.c:11:15: warning: taking address of packed member of ‘struct foo’ may result in an unaligned pointer value [-Waddress-of-packed-member]
   11 |     int *p1 = &arr[1].x;
      |               ^~~~~~~~~
Hilleary answered 19/12, 2011 at 22:29 Comment(14)
misaligned struct elements on ARM does weird stuff: Some accesses cause faults, others cause the retrieved data to be rearranged counter-intuitively or incorporate adjacent unexpected data.Cristobal
It seems that packing itself is safe, but how the packed members are used can be unsafe. Older ARM-based CPUs didn't support unaligned memory accesses either, newer versions do but I know Symbian OS still disallows unaligned accesses when running on these newer versions (the support is turned off).Koffler
Another way to fix it within gcc would be to use the type system: require that pointers to members of packed structs can only be assigned to pointers that are themselves marked as packed (ie. potentially unaligned). But really: packed structs, just say no.Spinthariscope
@Keith Thompson Even though you don't seem to have a fully intact understanding, I have to upvote you just for filing the bug! :) This is certainly a warning I would want to see enabled either by default of one of the all/extra groups. Personally, I can even see an experienced programming running into this by mistake if they aren't paying attention or forget that they're dealing with a packed struct.Dispersive
I would like to add that if you have this problem on a newer ARM processor, you can "fix" it by disabling the alignment checking feature of the MMU by clearing the A bit in the SCTLR c15 register: infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0388f/… This is useful for manipulating file system and USB data structures that are byte aligned but have 16 & 32 bit membersFreer
The correct way of handling this by compiler is to copy given unaligned value byte-by-byte into an aligned temporary or a register, and only then using it as the larger type value. Attempt to access it directly as a larger type is a compiler bug.Pettigrew
@SF.: Sure, and gcc already does that (or the equivalent) when accessing a misaligned member by name. But the sample program stores the address of a misaligned int member in an int* pointer. It's difficult for the compiler to detect the misalignment when that pointer is dereferenced, unless it adds checks on every dereference, which would slow down code that doesn't use packed structures.Hilleary
@KeithThompson: Oh, right. I wonder, if there's a way to tell it "use that pointer cautiously, it may be misaligned", something akin to 'volatile' but alignment-wise, not value-wise. int* __attribute__((aligned(1))) variable; seems silly to me.Pettigrew
@SF.: Consider an external function that takes an int* argument. That function has, and should have, no awareness that it might receive a misaligned pointer. The simplest solution would be to treat members of packed structures like bit fields, disallowing taking their addresses (which would also mean that an array member of a packed struct couldn't be indexed). Or gcc could provide a new kind of pointer type, incompatible with int*, that could point to misaligned objects; &obj.x would yield a misaligned pointer that couldn't be directly passed to something expecting an int*.Hilleary
@JakeV: I've added a paragraph discussing that. Now that I look at this again, it's a bit odd that arr is allocated at an address that's a multiple of 2 but not of 4. On Solaris, if I add a declaration short ignored; preceding the declaration of arr, arr is allocated at an address that's a multiple of 4. There's no requirement for it to be 4-byte aligned, so it's not very odd, just mildly surprising.Hilleary
@KeithThompson: If the Standard recognized that foo[x] is equivalent to *(foo+x) in cases where both are allowed, but allowed them to have different valid usage cases, that would provide an obvious way to allow arrays in a packed structure, arrays of things smaller than char (e.g. unsigned foo[3]:4;), and would clarify what is necessary to treat an NxM array as a linear array of (MN) elements (use pointer-arithmetic syntax rather than element-access syntax).Idleman
Note that the issue has been fixed in clang: bugs.llvm.org/show_bug.cgi?id=22821Bewhiskered
There are x86 instructions that expect alignment, in SSE I think. Using these with unaligned addresses, one gets a trap. Pointers to members of packed struct should be consider unsafe on x86 too.Counterproposal
Well, here's one that works on x86 and GCC: https://mcmap.net/q/14290/-c-undefined-behavior-strict-aliasing-rule-or-incorrect-alignment-duplicateCounterproposal
D
79

As ams said above, don't take a pointer to a member of a struct that's packed. This is simply playing with fire. When you say __attribute__((__packed__)) or #pragma pack(1), what you're really saying is "Hey gcc, I really know what I'm doing." When it turns out that you do not, you can't rightly blame the compiler.

Perhaps we can blame the compiler for it's complacency though. While gcc does have a -Wcast-align option, it isn't enabled by default nor with -Wall or -Wextra. This is apparently due to gcc developers considering this type of code to be a brain-dead "abomination" unworthy of addressing -- understandable disdain, but it doesn't help when an inexperienced programmer bumbles into it.

Consider the following:

struct  __attribute__((__packed__)) my_struct {
    char c;
    int i;
};

struct my_struct a = {'a', 123};
struct my_struct *b = &a;
int c = a.i;
int d = b->i;
int *e __attribute__((aligned(1))) = &a.i;
int *f = &a.i;

Here, the type of a is a packed struct (as defined above). Similarly, b is a pointer to a packed struct. The type of of the expression a.i is (basically) an int l-value with 1 byte alignment. c and d are both normal ints. When reading a.i, the compiler generates code for unaligned access. When you read b->i, b's type still knows it's packed, so no problem their either. e is a pointer to a one-byte-aligned int, so the compiler knows how to dereference that correctly as well. But when you make the assignment f = &a.i, you are storing the value of an unaligned int pointer in an aligned int pointer variable -- that's where you went wrong. And I agree, gcc should have this warning enabled by default (not even in -Wall or -Wextra).

Dispersive answered 8/4, 2013 at 4:30 Comment(6)
+1 for explaining how to use pointers with unaligned structs!Mercie
@Soumya Thanks for the points! :) Keep in mind however that __attribute__((aligned(1))) is a gcc extension and isn't portable. To my knowledge, the only really portable way to do unaligned access in C (with any compiler / hardware combination) is with a byte-wise memory copy (memcpy or similar). Some hardware does not even have instructions for unaligned access. My expertise is with arm and x86 which can do both, although the unaligned access is slower. So if you ever need to do this with high performance, you'll need to sniff the hardware and use arch-specific tricks.Dispersive
@Soumya Sadly, __attribute__((aligned(x))) now appears to be ignored when used for pointers. :( I don't yet have the full details of this, but using __builtin_assume_aligned(ptr, align) seems to get gcc to generate the correct code. When I a more concise answer (and hopefully a bug report) I'll update my answer.Dispersive
@DanielSantos: A quality compiler I use (Keil) recognizes "packed" qualifiers for pointers; if a structure is declared "packed", taking the address of a uint32_t member will yield a uint32_t packed*; trying to read from such a pointer on e.g. a Cortex-M0 will IIRC call a subroutine which will take ~7x as long as a normal read if the pointer is unaligned or ~3x as long if it's aligned, but will behave predictably in either case [in-line code would take 5x as long whether aligned or unaligned].Idleman
Here's a crash on x86-64 on rather innocent-looking code...Counterproposal
For an example of how ARM GCC handles unaligned access: godbolt.org/z/PceW9fTheomania
H
55

It's perfectly safe as long as you always access the values through the struct via the . (dot) or -> notation.

What's not safe is taking the pointer of unaligned data and then accessing it without taking that into account.

Also, even though each item in the struct is known to be unaligned, it's known to be unaligned in a particular way, so the struct as a whole must be aligned as the compiler expects or there'll be trouble (on some platforms, or in future if a new way is invented to optimise unaligned accesses).

Horseflesh answered 20/12, 2011 at 10:53 Comment(3)
Hmm, I wonder what happens if you put one packed struct inside another packed struct where the alignment would be different? Interesting question, but it shouldn't change the answer.Horseflesh
GCC won't always align the structure itself either. For example: struct foo { int x; char c; } __attribute__((packed)); struct bar { char c; struct foo f; }; I found that bar::f::x will not be necessarily aligned, at least on certain flavors of MIPS.Ferne
@antonm: Yes, a struct within a packed struct may well be unaligned, but, again, the compiler knows what the alignment of each field is, and it's perfectly safe as long as you don't try to use pointers into the struct. You should imagine a struct within a struct as one flat series of fields, with the extra name just for readability.Horseflesh
F
11

Using this attribute is definitely unsafe.

One particular thing it breaks is the ability of a union which contains two or more structs to write one member and read another if the structs have a common initial sequence of members. Section 6.5.2.3 of the C11 standard states:

6 One special guarantee is made in order to simplify the use of unions: if a union contains several structures that share a common initial sequence (see below), and if the union object currently contains one of these structures, it is permitted to inspect the common initial part of any of them anywhere that a declaration of the completed type of the union is visible. Tw o structures share a common initial sequence if corresponding members have compatible types (and, for bit-fields, the same widths) for a sequence of one or more initial members.

...

9 EXAMPLE 3 The following is a valid fragment:

union {
    struct {
        int    alltypes;
    }n;
    struct {
        int    type;
        int    intnode;
    } ni;
    struct {
        int    type;
        double doublenode;
    } nf;
}u;
u.nf.type = 1;
u.nf.doublenode = 3.14;
/*
...
*/
if (u.n.alltypes == 1)
if (sin(u.nf.doublenode) == 0.0)
/*
...
*/

When __attribute__((packed)) is introduced it breaks this. The following example was run on Ubuntu 16.04 x64 using gcc 5.4.0 with optimizations disabled:

#include <stdio.h>
#include <stdlib.h>

struct s1
{
    short a;
    int b;
} __attribute__((packed));

struct s2
{
    short a;
    int b;
};

union su {
    struct s1 x;
    struct s2 y;
};

int main()
{
    union su s;
    s.x.a = 0x1234;
    s.x.b = 0x56789abc;

    printf("sizeof s1 = %zu, sizeof s2 = %zu\n", sizeof(struct s1), sizeof(struct s2));
    printf("s.y.a=%hx, s.y.b=%x\n", s.y.a, s.y.b);
    return 0;
}

Output:

sizeof s1 = 6, sizeof s2 = 8
s.y.a=1234, s.y.b=5678

Even though struct s1 and struct s2 have a "common initial sequence", the packing applied to the former means that the corresponding members don't live at the same byte offset. The result is the value written to member x.b is not the same as the value read from member y.b, even though the standard says they should be the same.

Funch answered 2/4, 2019 at 18:43 Comment(1)
One might argue that if you pack one of the structs and not the other, then you're not going to expect them to have consistent layouts. But yes, this is another standard requirement that it can violate.Hilleary
T
1

(The following is a very artificial example cooked up to illustrate.) One major use of packed structs is where you have a stream of data (say 256 bytes) to which you wish to supply meaning. If I take a smaller example, suppose I have a program running on my Arduino which sends via serial a packet of 16 bytes which have the following meaning:

0: message type (1 byte)
1: target address, MSB
2: target address, LSB
3: data (chars)
...
F: checksum (1 byte)

Then I can declare something like

typedef struct {
  uint8_t msgType;
  uint16_t targetAddr; // may have to bswap
  uint8_t data[12];
  uint8_t checksum;
} __attribute__((packed)) myStruct;

and then I can refer to the targetAddr bytes via aStruct.targetAddr rather than fiddling with pointer arithmetic.

Now with alignment stuff happening, taking a void* pointer in memory to the received data and casting it to a myStruct* will not work unless the compiler treats the struct as packed (that is, it stores data in the order specified and uses exactly 16 bytes for this example). There are performance penalties for unaligned reads, so using packed structs for data your program is actively working with is not necessarily a good idea. But when your program is supplied with a list of bytes, packed structs make it easier to write programs which access the contents.

Otherwise you end up using C++ and writing a class with accessor methods and stuff that does pointer arithmetic behind the scenes. In short, packed structs are for dealing efficiently with packed data, and packed data may be what your program is given to work with. For the most part, you code should read values out of the structure, work with them, and write them back when done. All else should be done outside the packed structure. Part of the problem is the low level stuff that C tries to hide from the programmer, and the hoop jumping that is needed if such things really do matter to the programmer. (You almost need a different 'data layout' construct in the language so that you can say 'this thing is 48 bytes long, foo refers to the data 13 bytes in, and should be interpreted thus'; and a separate structured data construct, where you say 'I want a structure containing two ints, called alice and bob, and a float called carol, and I don't care how you implement it' -- in C both these use cases are shoehorned into the struct construct.)

Truth answered 16/8, 2015 at 14:45 Comment(1)
Unless I'm missing something, this doesn't answer the question. You argue that structure packing is convenient (which it is), but you don't address the question of whether it's safe. Also, you assert that performance penalties for unaligned reads; that's true for x86, but not for all systems, as I demonstrated in my answer.Hilleary

© 2022 - 2024 — McMap. All rights reserved.