What is the format of the x86_64 va_list structure?

Question

Anyone have a reference for the representation of va_list in the x86_64 ABI (the one used on Linux)? I'm trying to debug some code where the stack or arguments seem corrupt and it would really help to understand what I'm supposed to be seeing...

Succinylsulfathiazole

Answer 1

The x86-64 System V ABi doc may help. It's a reference, albeit lightweight.

The Variable Argument List reference starts on page 54, then it goes on, page 56-57 documents va_list:

The va_list Type

The va_list type is an array containing a single element of one structure containing the necessary information to implement the va_arg macro. The C definition of va_list type is given in figure 3.34.

Figure 3.34: va_list Type Declaration

typedef struct {
   unsigned int gp_offset;
   unsigned int fp_offset;
   void *overflow_arg_area;
   void *reg_save_area;
} va_list[1];

The va_start Macro

The va_start macro initializes the structure as follows:

reg_save_area The element points to the start of the register save area.

overflow_arg_area This pointer is used to fetch arguments passed on the stack. It is initialized with the address of the first argument passed on the stack, if any, and then always updated to point to the start of the next argument on the stack.

gp_offset The element holds the offset in bytes from reg_save_area to the place where the next available general purpose argument register is saved. In case all argument registers have been exhausted, it is set to the value 48 (6 * 8).

fp_offset The element holds the offset in bytes from reg_save_area to the place where the next available floating point argument register is saved. In case all argument registers have been exhausted, it is set to the value 304 (6 * 8 + 16 * 16).

Airsickness

I’m pretty sure these “floating point” registers are actually SSE registers, and that there are only 8 of them. – Cahra

@DaveAbrahams x86_64 has 16 general and SSE registers. – Linders

The size of the save area is 176 bytes, that is (6 * 8 + 8 * 16) instead of (6 * 8 + 16 * 16). Yes, the processor has 16 SSE registers however only first 8 registers are used to pass arguments. Beside that thanks for the information, it was really helpful. – Piaffe

Answer 2

It turns out the problem was gcc's making va_list an array type. My function was of the signature:

void foo(va_list ap);

and I wanted to pass a pointer to ap to another function, so I did:

void foo(va_list ap)
{
    bar(&ap);
}

Unfortunately, array types decay to pointer types in function argument lists, so rather than passing a pointer to the original structure, I was passing a pointer to a pointer.

To work around the problem, I changed the code to:

void foo(va_list ap)
{
    va_list ap2;
    va_copy(ap2, ap);
    bar(&ap2);
    va_end(ap2);
}

This is the only portable solution I could come up with, that accounts for both the possibility that va_list is an array type and the possibility that it's not.

Succinylsulfathiazole

Couldn't you have just made foo take an argument of type va_list *? – Flair

@caf: foo has a fixed signature I can't change. And even if not, v* functions always take a va_list argument, not a va_list * argument. This is standard convention and it would be annoying to users of the function to violate it. – Succinylsulfathiazole

Why did you need to pass a pointer to ap2 in the call to bar() ? – Waste

This code is of course stripped down, but the point is that bar uses additional information to determine the type of the next argument and "pops" it off the argument list. If you passed ap by value instead of passing a pointer to it, any further use of ap after bar returns, except va_end, would result in undefined behavior. – Succinylsulfathiazole

@R I'm not sure I see how passing it by pointer would help - your case sounds like it's exactly what va_copy is for. – Foxed

@bdonlan: bar is called more than once by foo, and each call to bar must see the effects of the previous one. This is explicitly UB (per ISO C) if you pass the va_list; you're required to pass a pointer to va_list for this usage. – Succinylsulfathiazole

even though this should work in practice, I think it technically still is UB, as va_copy() is used with an argument which does not have type va_list: if va_copy() is not a function (which is explicitly allowed), then no parameter adjustments willl be performed(!); however, one could argue that as the standard gives a prototype for va_copy() and state that it might be implemented as a function, argument adjustments are implicitly assumed... – Tented

I don't follow. Are you saying the argument does not have type va_list because va_list is an array type, and that this makes using va_copy undefined? What "parameter adjustments" are you talking about? Decay of array types to pointers has nothing to do with them being passed to a function... – Succinylsulfathiazole

@Christoph: Looking back, I see what you were saying, but I don't think it's an issue. The obvious intended usage of va_copy is to use it on a va_list you obtained as an argument. If you had obtained it with va_start, you could just call va_start again to get a copy and there would be no need for va_copy. – Succinylsulfathiazole

I don't get what the va_copy changes. You still get a va_list, so why is it all of a sudden alright to pass the pointer to the copy? EDIT: I got the basic idea now... julio.meroh.net/2011/09/using-vacopy-to-safely-pass-ap.html – Rhinal

Yet, apparently, C99 tells us that there should be no issue with passing a pointer . – Rhinal

@polynomial_donut: The blog post you linked to is bogus, as noted in the update at the top of it. It's always fine to pass a va_list you received as an argument to another function taking a va_list (as long as you don't use it after that except to va_end). It's also fine to pass a pointer to a va_list.... – Succinylsulfathiazole

... The issue is that, while declaring an object va_list ap; gives you an object of type va_list so that &ap has type pointer-to-va_list, receiving an argument declared va_list ap does not necessarily mean ap has type va_list. If va_list is an array type, the array decay rules for function arguments apply, and then ap has type pointer-to-pointer-to-__typeof__(*ap), which is neither type-compatible nor the right value to pass to a function expecting pointer-to-va_list (it's an extra level of indirection). va_copy to temp is the only way to fix this. – Succinylsulfathiazole

@R.. 'I got the idea now' was referring to the example that an offset variable to a stack pointer could be involved. That example wasn't bogus in explaining why and when the difference could matter - or at least it sounded like a reasonable explanation to me. On the other hand, passing a pointer to a va_list is ok, if the C99 tells us that, or not (?). At least I'd like to assume clang or gcc have implemented C99 according to the full specs (?). – Rhinal

@R.. I also have to admit I don't yet see what the problem is purely with the argument decaying other than throwing a sizeof at it and expecting the size of the entire data object - but this wouldn't work even if you didn't pass a pointer to the array type and dereference it (and just passed the array itself) (?) – Rhinal

@polynomial_donut: The "an offset variable to a stack pointer could be involved" line of reasoning was entirely bogus. The language allows passing va_list to another function, so such an implementation that used relative offsets from a base that only the function calling va_start could know would be non-conforming. – Succinylsulfathiazole

@polynomial_donut: In the case of a function argument declared va_list ap where va_list is defined with array type, ap does not have type va_list. It has type __typeof__(*ap)*. Arrays are not pointers. Thus &ap doesn't have type va_list * (pointer to whole-array, whose value is the same as pointer to the initial member but whose type differs), which it would need to have in order to pass it to a function expecting va_list *. Instead &ap has type __typeof__(*ap)**, a pointer-to-pointer type whose value has nothing to do with the address of the va_list object. – Succinylsulfathiazole

@R.. first off, thanks for your patience and answers :) From what you've written and reading up on this a little bit, what happens can be boiled down to: 1. On entry into foo, ap is converted into typeof(*ap) * (according to C99) 2. Then, bar is called on &ap, which is of type typeof(*ap) ** 3. Assuming the received argument inside bar is named ap_bar: If ap_bar is dereferenced, the result is an actual array, so machine code will operate differently on it when using [] or * than if it were a __typeof__(*ap) ** (which it really is in your example). – Rhinal

Also, ap_bar * is subject to the same argument conversions (array into pointer) as mentioned in point 1; this conversion can then produce improper results when applied to a variable of type __typeof__(*ap) *. – Rhinal

@polynomial_donut: I think it would be instructive for you to work out an example with typedef int foo[1]; and functions taking this type foo, or a pointer to it, as an argument. – Succinylsulfathiazole

Answer 3

In i386 architecture, the va_list is a pointer type. However, in AMD64 architecture, it is an array type. What is the difference? Actually, if you apply an & operation to a pointer type, you will get the address of this pointer variable. But no matter how many times you apply & operation to an array type, the value is the same, and is equal to the address of this array.

So, what should you do in AMD64? The easiest way to pass variable of va_list in a function is just passing it with no * or & operator.

For example:

void foo(const char *fmt, ...) {
    va_list ap;
    int cnt;
    va_start(ap, fmt);
    bar(fmt, ap);
    va_end(ap);
    return cnt;
}
void bar(const char *fmt, va_list ap) {
    va_arg(ap, int);
    //do something
    test(ap);
}
void test(va_list ap) {
    va_arg(ap, int);
    //do something
}

It just works! And you don't need to worry about how many arguments you have got.

Amargo

This does not answer the question. – Succinylsulfathiazole