Is accessing members through offsetof well defined?

Question

When doing pointer arithmetic with offsetof, is it well defined behavior to take the address of a struct, add the offset of a member to it, and then dereference that address to get to the underlying member?

Consider the following example:

#include <stddef.h>
#include <stdio.h>

typedef struct {
    const char* a;
    const char* b;
} A;

int main() {
    A test[3] = {
        {.a = "Hello", .b = "there."},
        {.a = "How are", .b = "you?"},
        {.a = "I\'m", .b = "fine."}};

    for (size_t i = 0; i < 3; ++i) {
        char* ptr = (char*) &test[i];
        ptr += offsetof(A, b);
        printf("%s\n", *(char**)ptr);
    }
}

This should print "there.", "you?" and "fine." on three consecutive lines, which it currently does with both clang and gcc, as you can verify yourself on wandbox. However, I am unsure whether any of these pointer casts and arithmetic violate some rule which would cause the behavior to become undefined.

Godesberg

I'm very curious about why you want to do something like this? Is it plain curiosity? Or is there some underlying problem that you want to solve this way? If the latter, then perhaps you should ask about that instead? – Agan

@Someprogrammerdude Mostly curiosity. The idea just popped up when writing code, and it could indeed mean a (very small) optimization, but there is no real problem that I am trying to solve here. I am specifically interested in this offsetof use. – Godesberg

When you mention "optimization" I have alarm bells going of in my head. Don't do premature optimizations, instead write simple, readable, and most importantly maintainable code first and foremost. Then remember that "good enough" often is good enough. And only if the performance of your program isn't "good enough" for your requirements you measure, profile and benchmark to find the bottlenecks, and fix only the worst of those (with plenty of comments and documentation). – Agan

You violate the strict aliasing rule. – Huxley

@Someprogrammerdude I'm well aware of the danger of premature optimization. Maybe I didn't express myself clearly enough, but I do not actually intend to use this unless required by performance deficits etc. This is just the context in which I came up with this idea. I asked this question because I'm genuinely curious about whether this would actually be possible or not, not because I intend to use it. – Godesberg

@BenSteffan I'm not even sure how doing this manually would be any kind of optimization in any case. Using direct member access with test[i].b will do the same pointer arithmetic as necessary, but is guaranteed to be correct behavior while pointer hacks are not. And since it's the common case, it's the one that compiler designers would focus optimization on, whereas they might not have as effectively optimized pointer hacks. One could imagine a case where test[i].b is compiled to a single load instruction while the manual arithmetic is done as individual steps followed by a load. – Thralldom

@zstewart: If multiple structures will have certain fields in the same places, and one needs a function that can work with all such structures interchangeably, I'm not sure how one could write the code without using either offsetof or a compiler that actually processes the Common Initial Sequence guarantees in usable fashion. – Mclaughlin

@Mclaughlin oh, interesting. I'd gotten the wrong impression about the ability to cast between structs with common prefixes because I'd though Python was doing it -- and they were but it looks like they aren't anymore. – Thralldom

@zstewart: The ability to cast between structs with common prefixes is allowed by the Standard if a complete union type declaration containing both types is visible when the struct is accessed, but the authors of gcc claim that the rule says that it's only required to honor the CIS rule when all lvalue accesses are performed through the union type, even though that isn't what the rule says and would make the rule useless. – Mclaughlin

Answer 1

As far as I can tell, it is well-defined behavior. But only because you access the data through a char type. If you had used some other pointer type to access the struct, it would have been a "strict aliasing violation".

Strictly speaking, it is not well-defined to access an array out-of-bounds, but it is well-defined to use a character type pointer to grab any byte out of a struct. By using offsetof you guarantee that this byte is not a padding byte (which could have meant that you would get an indeterminate value).

Note however, that casting away the const qualifier does result in poorly-defined behavior.

EDIT

Similarly, the cast (char**)ptr is an invalid pointer conversion - this alone is undefined behavior as it violates strict aliasing. The variable ptr itself was declared as a char*, so you can't lie to the compiler and say "hey, this is actually a char**", because it is not. This is regardless of what ptr points at.

I believe that the correct code with no poorly-defined behavior would be this:

#include <stddef.h>
#include <stdio.h>
#include <string.h>

typedef struct {
    const char* a;
    const char* b;
} A;

int main() {
    A test[3] = {
        {.a = "Hello", .b = "there."},
        {.a = "How are", .b = "you?"},
        {.a = "I\'m", .b = "fine."}};

    for (size_t i = 0; i < 3; ++i) {
        const char* ptr = (const char*) &test[i];
        ptr += offsetof(A, b);

        /* Extract the const char* from the address that ptr points at,
           and store it inside ptr itself: */
        memmove(&ptr, ptr, sizeof(const char*)); 
        printf("%s\n", ptr);
    }
}

Rivera

"Not sure why you did that cast, since it is superfluous." -- The cast makes perfect sense. ptr has type char * and points to the first byte of a pointer value. It needs to be cast to a pointer-to-pointer type to access the actual pointer value. You might argue that it should be const char ** rather than char **, but there definitely needs to be a cast there. – Lamphere

@hvd Fair enough, it is not superfluous, it is plain UB. Converting a char* to a char** is an invalid pointer conversion, even if the effective type of the pointed-at data happened to be a pointer. – Rivera

Converting a char * to a char ** is supposed to be valid when the value pointed to really is a char *. Otherwise, the whole offsetof macro would be pretty difficult to use correctly (as @cmaster points out, it's still possible). Where does the standard say it is an invalid pointer conversion? 6.3.2.3p7 at least says that given char *p;, char *q = (char *) &p; is valid, and after that, (char **) q == &p must hold. – Lamphere

@Rivera So, you think that replacing the cast (char**)ptr with the equivalent char* string; memcpy(&string, ptr, sizeof(string)); would make the code snippet well-defined? – Assemblyman

@hvd 6.3.2.3/7 specifies what conversion is possible as far as the pointer syntax goes - C allows all manner of wild pointer casts. Whether you can actually access that pointed-at data or not is determined by effective type/strict aliasing, 6.5/6 and 6.5/7. And well, the effective type of the object is const char*. So I suppose the only issue is the const after all. If ptr is a pointer to such an object, then the proper type to use would be char *const *. While char** is an incompatible type. – Rivera

@cmaster Apart from the const issue, then yes that code would also be well-defined. – Rivera

Hmm, given some more thought... The issue I wasn't certain about is ptr itself, not the pointed at data. ptr is clearly a char* in the original code, you can't tell the compiler that ptr is actually a char**. A char** cannot alias a char* - this is the actual issue! Not the pointed at data. I'll update the answer. – Rivera

@Rivera Both, really. 6.3.2.3p7 is required to determine whether the converted pointer points to the intended object, 6.5 is then required to determine whether the access to that object is allowed by the aliasing rules. As for your update, "so you can't lie to the compiler and say "hey, this is actually a char**"" -- the code isn't doing that. (char***)&ptr would be doing that, and that would indeed be invalid, but (char**)ptr only converts the value stored in ptr to char**, it doesn't pretend that ptr is a char**. – Lamphere

@hvd ptr being a char*, can only point at characters, not at other pointers. Of course there's a pointer b sitting right there in the binary, but the compiler is free to assume that the line(char**)ptr is not used to access the pointer b, since a char** cannot be used to access a char*. Regardless of the const issue. – Rivera

@hvd You can verify that this is truly UB by writing to b with this char** method on various compilers and then print the value of b afterwards. The value might not have been updated, because of the strict aliasing violation. Most likely compiler to misbehave is gcc, because it optimizes such code aggressively. – Rivera

@Rivera "ptr being a char*, can only point at characters, not at other pointers." -- There's a special exception for character types. They may be used to point to and access any object. That's what 6.3.2.3p7 covers in its last two sentences, and what the special exception in 6.5p7 ("a character type") is for. And that's well supported even in GCC. – Lamphere

@hvd Yes, I am well aware. You can use it to read out the binary representation of any data type (in which case you'll actually want unsigned char or uint8_t). But that's not how it was used here! To do that you'd have to create some non-portable monstrosity like const char* p = ptr[0] << 24 | ptr[1] << 16, which assumes that you are aware of the specific pointer representation and its endianess. – Rivera

@Rivera Or, as 6.3.2.3p7 allows in some cases, just cast the char * value back to a pointer to the proper type, const char ** in this case, and dereference that. Which is what the OP is doing. You had just claimed it is invalid because char * cannot point to pointer objects, only to characters, but you seem to acknowledge now that both pointing to other objects and accessing other objects is allowed. – Lamphere

@hvd The proper type of ptr in the question is char* and nothing else, period. As I already said, it doesn't matter what it points at, because the compiler will assume that the pointed-at data, what ever it might be, will not be accessed through a char**. The strict aliasing rule has an exception for the purpose of serialization of binary data byte by byte, and that was the example I just gave since you thought that it applied here - but it has nothing at all to do with the OP's case. – Rivera

@Rivera No, the compiler will not assume that, because there is no rule anywhere in the standard that allows the compiler to assume that. In the OP's case, there is no access of the b member using an lvalue of char, because ptr is never dereferenced directly, and even if the b member were dereferenced directly, the exception for character types always applies, because the standard does not and cannot make exceptions depending on what's in the programmer's mind at the time, it cannot possibly only apply when the purpose is serialisation. – Lamphere

@Lundin: Seeing what compilers do is not an effective way of testing whether something is defined by the Standard. Knowing that a compiler handles certain constructs bogus fashion may be useful, but that doesn't imply that the Standard doesn't define those constructs. – Mclaughlin

Answer 2

Given

struct foo {int x, y;} s;
void write_int(int *p, int value) { *p = value; }

nothing in the Standard would distinguish between:

write_int(&s.y, 12); //Just to get 6 characters

and

write_int((int*)(((char*)&s)+offsetof(struct foo,y)), 12);

The Standard could be read in such a way as to imply that both of the above violate the lvalue-type rules since it does not specify that the stored value of a structure can be accessed using an lvalue of a member type, requiring that code wanting to access as structure member be written as:

void write_int(int *p, int value) { memcpy(p, value, sizeof value); }

I personally think that's preposterous; if &s.y can't be used to access an lvalue of type int, why does the & operator yield an int*?

On the other hand, I also don't think it matters what the Standard says. Neither clang nor gcc can be relied upon to correctly handle code that does anything "interesting" with pointers, even in cases that are unambiguously defined by the Standard, except when invoked with -fno-strict-aliasing. Compilers that make any bona fide effort to avoid any incorrect aliasing "optimizations" in cases which would be defined under at least some plausible readings of the Standard will have no trouble handling code that uses offsetof in cases where all accesses that will be done using the pointer (or other pointers derived from it) precede the next access to the object via other means.

Mclaughlin

Would you recommend to always use -fno-strict-aliasing? It seems to me to be the only sane way left. And do you know if all current compilers offer a similar flag? If not, what other options are there? – Retirement

@GermanNerd: People seeking to sell compilers design them to fulfill customers' needs. It's possible that some specialized compilers might not offer options to disable obtuse "optimizations" that break commonplace constructs, but any quality compiler that is designed to be suitable for low-level programming will offer a mode that will handle the constructs that gcc/clang optimizers can't. Getting anything useful in the Standard would require a consensus among three factions: those who want to allow aggressive optimizations by implementations intended for purposes where they would be fine,... – Mclaughlin

...those who want to be able to use low-level constructs in cases that require them, and those who are opposed to "fragmenting the language" by recognizing that implementations intended for some purposes should offer semantics that could not be practically supported by those intended for some other purposes. IMHO, the Standard would be most useful if the third group could be excluded, but a Standard which excludes one of the others, and is recognized as making no effort to satisfy that groups' needs, would be better than what we have now. – Mclaughlin

Thank you for your elaborations. I personally think that introducing restrictions on the language to facilitate optimizations is disastrous, completely against the spirit of C. Not to even mention that the SAR is basically inviting programmers to fall into carefully prepared traps...of the worst kind, which might exhibit behaviours that are traceable only if one understands a specific(!) compilers optimization. That's supposed to be portable? To be more specific: If I allocate some memory (malloc), I expect that memory to be MINE, to do with it as I see fit. It can be very useful... – Retirement

@GermanNerd: N1570 6.5p7 would be fine except for one word: "by". If that were replaced by "...with a visible relation to...", a footnote made clear that the ability to recognize relations was a Quality of Implementation issue, the rule would be fine. I share your feeling about vandals stealing the language. – Mclaughlin

Let us continue this discussion in chat. – Retirement