Opaque types allocatable on stack in C

Question

When designing a C interface, it is common to let into the public interface (.h) only what needs to be known by the user program.

Hence for example, the inner components of structures should remain hidden if the user program does not need to know them. This is indeed good practice, as the content and behavior of the struct could change in the future, without affecting the interface.

A great way to achieve that objective is to use incomplete types.

typedef struct foo opaqueType;

Now an interface using only pointers to opaqueType can be built, without the user program ever needing to know the inner working of struct foo.

But sometimes, it can be required to allocate such structure statically, typically on stack, for performance and memory fragmentation issues. Obviously, with above construction, opaqueType is incomplete, so its size is unknown, so it cannot be statically allocated.

A work around is to allocate a "shell type", such as :

typedef struct { int faketable[8]; } opaqueType;

Above construction enforces a size and an alignment, but doesn't go farther into describing what the structure really contains. So it matches the objective of keeping the type "opaque".

It mostly works. But in one circumstance (GCC 4.4), the compiler complains that it breaks strict-aliasing, and it generates buggy binary.

Now, I've read a ton of things about strict aliasing, so I guess I understand now what it means.

The question is : is there a way to define an opaque type which can nonetheless be allocated on stack, and without breaking strict aliasing rule ?

Note that I've attempted the union method described in this excellent article but it still generates the same warning.

Note also that visual, clang and gcc 4.6 and later don't complain and work fine with this construction.

[Edit] Information complement :

According to tests, the problem only happens in the following circumstances :

• Private and public type different. I'm casting the public type to private inside the .c file. It doesn't matter apparently if they are part of the same union. It doesn't matter if the public type contains char.
• If all operations on private type are just reads, there's no problem. Only writes cause problems.
• I also suspect that only functions which are automatically inlined get into trouble.
• Problem only happens on gcc 4.4 at -O3 setting. -O2 is fine.

Finally, my target is C90. Maybe C99 if there really is no choice.

Besides

If all the other compilers, including a newer version of gcc work, this seems to be an issue with gcc 4.4. – Hakenkreuz

@rost0031: Not sure why other compilers did not complain, possibly some optimization issue, but IMHO this is a violation. – Separative

I believe this is indeed a violation of the rule to the letter, hence why it leads to problem when it is applied "blindly" (which seems to be gcc 4.4). I also believe that in above circumstances, the aliasing situation is just plain trivial. Therefore, any compiler can easily detect it, and act appropriately (hence gcc 4.6+, clang, visual, likely others). The question targets the first kind of compilers. – Besides

Don't use int for the fake type; use char. Doesn't that avoid the strict aliassing issues? – Wittenburg

I've been looking for a good solution for this for 20 years and I've concluded that I'm looking for a technical solution to a social problem. I now don't do anything clever, just expose the proper structs and sabotage anyone who touches them. By for example randomly renaming members and moving stuff around in backwards compatible bugfix releases. – Professor

For an excellent article/tutorial on incomplete datatypes, encapsulation, data-hiding, dynamic linkage/late binding and object oriented approaches to dynamic data-structures, see Object Oriented Programming in ANSI-C. While it is written at, and requires, a fairly in-depth knowledge of C, it is well worth the effort requierd to digest the material. It covers a number of topics not included in most C books or tutorials and directly covers the opaque (incomplete) datatypes at issue here. – Guinea

@jonathan : int is required for the alignment. (It could be size_t, short or long long depending on necessary alignment). char does not provide such guarantee, the structure could start at any position in memory. But maybe Rodrigo answer (a union) is the right one. – Besides

@Professor Not sure whether to be awed or appalled. – Rarefied

Answer 1

You can force the alignment with max_align_t and you can avoid the strict aliasing issues using an array of char since char is explicitly allowed to alias any other type.

Something along the lines of:

#include <stdint.h>
struct opaque
{
    union
    {
        max_align_t a;
        char b[32]; // or whatever size you need.
    } u;
};

If you want to support compiler that do not have the max_align_t, or if you know the alignment requirements of the real type, then you can use any other type for the a union member.

UPDATE: If you are targetting C11, then you may also use alignas():

#include <stdint.h>
#include <stdalign.h>
struct opaque
{
    alignas(max_align_t) char b[32];
};

Of course, you can replace the max_align_t with whatever type you think appropriate. Or even an integer.

UPDATE #2:

Then, the use of this type in the library would be something along the lines of:

void public_function(struct opaque *po)
{
    struct private *pp = (struct private *)po->b;
    //use pp->...
}

This way, since you are type-punning a pointer to char you are not breaking the strict aliasing rules.

Aground

It looks like a good idea. It probably needs testing, as my understanding of the char* exception to strict aliasing rule is that it only works one way. See : cellperformance.beyond3d.com/articles/2006/06/… – Besides

@Cyan: I think your link talks about doing for example int x; *(float*)(char*)&x = 0;. That still breaks strict aliasing, the char* does not help. But the code in my answer is different, and I believe it is correct. – Aground

Since the max_align_t member still takes up space, this may violate strict aliasing, unless you use the C11 construct. – Pereyra

I just made a test using union proposal ; Unfortunately, it doesn't work. Or rather : the compiler (gcc 4.4) still generates warnings (the same ones as before) and a faulty binary (segfault). :( – Besides

@Kevin: I don't see why the space matters. What would matter is if you assign to the a member. But if you use only the b you should be safe. – Aground

@Cyan: Maybe you used opaque o; private *p = (private*)&o? That will break strict aliasing rules, I think. Use instead: opaque o; private *p = (private*)o.u.b;. If it still fails, maybe you can post a minimal example. – Aground

You haven't actually shown how is this used. Assuming the API will reinterpret struct opaque as struct private, it will break strict aliasing. – Shoon

@2501: The idea is to cast the pointer to opaque::b, not to opaque. Since it is an array of char they are explicitly allowed to alias any other type and you are safe. See the update #2 in my answer. – Aground

char can alias any type, but not vice-versa. struct private * is no way allowed to alias either char or an array of char. This is a common misconception, please look it up. – Shoon

Answer 2

What you desire is some kind of equivalent of the C++ private access control in C. As you know, no such equivalent exists. The approach you give is approximately what I would do. However, I would make the opaqueType opaque to the inner components implementing the type, so I would be forced to cast it to the real type within the inner components. The forced cast should not generate the warning you are mentioning.

Although cumbersome to use, you can define an interface that provides "stack allocated" memory to an opaque type without exposing a sized structure. The idea is that the implementation code is in charge of the stack allocation, and the user passes in a callback function to get a pointer to the allocated type.

typedef struct opaqueType_raii_callback opqaueType_raii_callback;
struct opaqueType_raii_callback {
    void (*func)(opqaueType_raii_callback *, opqaueType *);
};
extern void opaqueType_raii (opaqueType_raii_callback *);
extern void opaqueType_raii_v (opaqueType_raii_callback *, size_t);


void opaqueType_raii (opaqueType_raii_callback *cb) {
    opaqueType_raii_v(cb, 1);
}

void opqaueType_raii_v (opaqueType_raii_callback *cb, size_t n) {
    opaqueType x[n];
    cb->func(cb, x);
}

The definitions above look a bit esoteric, but it is the way I normally implement a callback interface.

struct foo_callback_data {
    opaqueType_raii_callback cb;
    int my_data;
    /* other data ... */
};

void foo_callback_function (opaqueType_raii_callback *cb, opaqueType *x) {
    struct foo_callback_data *data = (void *)cb;
    /* use x ... */
}

void foo () {
    struct foo_callback_data data;
    data.cb.func = foo_callback_function;
    opaqueType_raii(&data.cb);
}

Kish

hard to follow ... I will probably have to come back here and read it again a few times ... – Besides

@Cyan: The stack looks like this (longest-lived scope first): foreign code which calls into your API, opaqueType_raii() (optional, foreign code can just call the next function directly), opaqueType_raii_v(), and finally the function pointer attached to the opaqueType_raii_callback structure (which was set by foreign code). The opaqueType_raii_v() stack frame contains the opaque object, and passes a pointer to it as an argument to the callback. IMHO this is using a sledgehammer to squash a fly, but it should work. – Pereyra

@Kevin: No global/static state or trampoline is required. Please look at the example usage. – Kish

@jxh: Oh, I see what you're doing. That's ugly as hell, and vaguely resembles an aliasing violation. – Pereyra

Would it kill you to just shove an extra void* member into the struct? – Pereyra

@Kevin: This is how a lot of existing C code implements dynamic dispatching. To me, it is just the standard way to do things. C standard guarantees the pointer to the first member of a structure is equal to the pointer to the containing structure. – Kish

Sure, but it'd be a lot easier to read (relatively speaking; this is still way overkill) with that extra void* member. – Pereyra

@Kevin: It actually does not really improve readability for someone that is already used to dynamic dispatching. Plus, the consequence is that another pointer has to be followed, and an additional data cache miss is likely the result. – Kish

Let us continue this discussion in chat. – Pereyra

Answer 3

For me this seems to be something which just shouldn't be done.

The point of having an opaque pointer is to hide the implementation details. The type and alignment of memory where the actual structure is allocated, or whether the library manages additional data beyond what's pointed to are also implementation details.

Of course not that you couldn't document that one or another thing was possible, but the C language uses this approach (strict aliasing), which you can only more or less hack around by Rodrigo's answer (using max_align_t). By the rule you can't know by the interface what kind of constraints the particular compiler would impose on the actual structure within the implementation (for some esoteric microcontrollers, even the type of memory may matter), so I don't think this can be done reliably in a truly cross platform manner.

Feat