What are the real benefits of flexible array member?

Question

After reading some posts related to flexible array member, I am still not fully understand why we need such a feature.

Possible Duplicate:
Flexible array members in C - bad?
Is this a Flexible Array Struct Members in C as well?

(Blame me if I didn't solve my problem from the possible duplicate questions above)

What is the real difference between the following two implementations:

struct h1 {
    size_t len;
    unsigned char *data;
};

struct h2 {
    size_t len;
    unsigned char data[];
};

I know the size of h2 is as if the flexible array member (data) were omitted, that is, sizeof(h2) == sizeof(size_t). And I also know that the flexible array member can only appear as the last element of a structure, so the original implementation can be more flexible in the position of data.

My real problem is that why C99 add this feature? Simply because sizeof(h2) doesn't contain the real size of data? I am sure that I must miss some more important points for this feature. Please point it out for me.

Prevot

Answer 1

The two structs in your post don't have the same structure at all. h1 has a integer and a pointer to char. h2 has an integer, and an array of characters inline (number of elements determined at runtime, possibly none).

Said differently, in h2 the character data is inside the struct. In h1 it has to be somewhere outside.

This makes a lot of difference. For instance, if you use h1 you need to take care of allocating/freeing the payload (in addition to the struct itself). With h2, only one allocation/free is necessary, everything is packaged together.

One case where using h2 might make sense is if you're communicating with something that expects messages in the form of {length,data} pairs. You allocate an instance of h2 by requesting sizeof(h2)+how many payload chars you want, fill it up, and then you can transfer the whole thing in a single write (taking care about endianess and such of course). If you had used h1, you'd need two write calls (unless you want to send the memory address of the data, which usually doesn't make any sense).

So this feature exists because it's handy. And various (sometimes non-portable) tricks where used before that to simulate this feature. Adding it to the standard makes sense.

Ciera

Thanks for your reply! According to your answer, what really cools is that it avoids the extra alloc/free work related to the payload. So another question is that why it chooses to omit the size of data when I call sizeof(h2) since it must know the actual size of data in h2? This is more related about the design, can you help me? :-) – Prevot

"... since it must know ..." - who's "it"? The compiler doesn't know the size of data. Only the programmer does. The compiler might be smart enough to infer the size of data after you've allocated memory for an instance of h2 in some circumstances, but generally it won't. – Ciera

Suppose I allocate struct h2 this way: struct h2 *p; p = (struct h2 *) malloc(sizeof(struct h2) + len * sizeof(char)); p->len = len; Then when I want to free it, I don't need to care about the payload: free(p); However, someone must know the size of data since it needs to free the extra payload. Since it knows the size, why sizeof(h2) can't tell me the actually size including the payload? – Prevot

(Don't cast the return of malloc in C.) The runtime system uses whatever tricks it wants to know how much to release when you call free on a pointer. That's not the compiler's job, the compiler doesn't track that. It's the malloc implementation that does. – Ciera

@YanzheChen: In many implementations, malloc() will sometimes return a pointer to a block which is larger than requested. In such cases, the system might not keep track of how much memory was requested from malloc(). While a function to inquire about the actual allocated size of a block may be helpful, having such a function report a value that doesn't match the argument given to malloc could be confusing. – Avaria

Answer 2

The main reason the Committee introduced flexible array members is to implement the famous struct hack. See the below quote from the C99 Rationale, especially the part I add the emphasis.

Rationale for International Standard — Programming Languages — C §6.7.2.1 Structure and union specifiers

There is a common idiom known as the “struct hack” for creating a structure containing a variable-size array:

struct s
{
int n_items;
/* possibly other fields */
int items[1];
};

struct s *p;
size_t n, i;
/* code that sets n omitted */
p = malloc(sizeof(struct s) + (n - 1) * sizeof(int));
/* code to check for failure omitted */
p->n_items = n;
/* example usage */
for (i = 0; i < p->n_items; i++)
    p->items[i] = i;

The validity of this construct has always been questionable. In the response to one Defect Report, the Committee decided that it was undefined behavior because the array p->items contains only one item, irrespective of whether the space exists. An alternative construct was suggested: make the array size larger than the largest possible case (for example, using int items[INT_MAX];), but this approach is also undefined for other reasons.

The Committee felt that, although there was no way to implement the “struct hack” in C89, it was nonetheless a useful facility. Therefore the new feature of “flexible array members” was introduced. Apart from the empty brackets, and the removal of the “-1” in the malloc call, this is used in the same way as the struct hack, but is now explicitly valid code.

There are a few restrictions on flexible array members that ensure that code using them makes sense. For example, there must be at least one other member, and the flexible array must occur last. Similarly, structures containing flexible arrays can't occur in other structures or in arrays. Finally, sizeof applied to the structure ignores the array but counts any padding before it. This makes the malloc call as simple as possible.

Clypeate

I've encountered some old compilers which would accept declarations for zero-element arrays, and which--given such declarations--would behave in exactly the fashion appropriate for flexible array members. I suspect many other compilers would do so if one were to hack out the code which specifically forbids zero-element arrays. I wonder if there would have been any problem with having both type[] and type[0] as synonyms for each other? – Avaria

@Avaria The difference is, zero length arrays are never part of standard C, some compilers support it as extension, e.g, GCC. But flexible length arrays are part of standard C99. The usage of zero length arrays are therefore up to the compilers that support it. – Clypeate

My point is that when FLA's were added to C99, some existing implementations (whether "rightly" or "wrongly") already supported zero-length arrays; having zero-length arrays as an alternative way of writing FLA's would have allowed code to be interoperable with either those compilers or new ones, though I suppose one could define a macro FLA to be either empty for C99 or 0 for one of those older compilers, and then declare char theArray[FLA];. Actually, thinking about it, zero-length arrays could potentially have an alternative use... – Avaria

...for data structures which, depending upon compile-time constants, might or might not need to contain certain data [e.g. char first_block[INFO_SIZE > 255 ? 256 : INFO_SIZE]; char *additional_blocks[INFO_SIZE/256];], where being able to have the array shrink to zero elements could be cleaner than having to add more special-case behavior for INFO_SIZE values above or below 256 [though adding that special-case behavior could eliminate unnecessary array-access code in cases where INFO_SIZE is less than 256]. – Avaria

@Avaria With the assumption that the implementation supports such usage, it makes sense to me. Just another note, even with variable length arrays, the size cannot be zero according to the standard. Zero length arrays are still non-standard compiler extension. – Clypeate

Answer 3

I don't know if this is considered as an important point, but GCC docs points this out:

GCC allows static initialization of flexible array members. This is equivalent to defining a new structure containing the original structure followed by an array of sufficient size to contain the data. E.g. in the following, f1 is constructed as if it were declared like f2.

struct f1 {
    int x; int y[];
} f1 = { 1, { 2, 3, 4 } };

struct f2 {
    struct f1 f1; int data[3];
} f2 = { { 1 }, { 2, 3, 4 } };

(taken from http://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html)

Sou

This is interesting, but I do not think it is the reason flexible array members were added to the standard. So it is not an answer to the question. It could be provided as a comment. – Rabelais