GCC ignores cdecl?

Question

I'm using gcc on Linux x86. My program exports a pointer to a C function to LLVM JIT functions. The calling convention is cdecl. It runs well on MingW on Windows. But strange things happens on linux x86 platform. The disassembly of the exported C function is like this :

push   ebp
mov    ebp,esp
push   ebx
sub    esp,0x34
mov    eax,0xfffffffc
mov    eax,DWORD PTR gs:[eax]
mov    eax,DWORD PTR [eax+0x1c]
mov    eax,DWORD PTR [eax]
mov    eax,DWORD PTR [eax+0x28]
mov    edx,DWORD PTR [ebp+0xc]
shl    edx,0x4
add    eax,edx
mov    DWORD PTR [ebp-0xc],eax
mov    edx,DWORD PTR ds:0x8e49940
mov    ebx,DWORD PTR [ebp+0x8]
lea    eax,[ebp-0x20]
mov    ecx,DWORD PTR [ebp-0xc]
mov    DWORD PTR [esp+0xc],ecx
mov    ecx,DWORD PTR [ebp+0x10]
mov    DWORD PTR [esp+0x8],ecx
mov    DWORD PTR [esp+0x4],edx
mov    DWORD PTR [esp],eax
call   0x8090f6f <SoCreateArray(DVM_VirtualMachine_tag*, int, DVM_TypeSpecifier_tag*)>
sub    esp,0x4
mov    eax,DWORD PTR [ebp-0x20]
mov    edx,DWORD PTR [ebp-0x1c]
mov    DWORD PTR [ebx],eax
mov    DWORD PTR [ebx+0x4],edx
mov    eax,DWORD PTR [ebp+0x8]
mov    ebx,DWORD PTR [ebp-0x4]
leave
ret    0x4

And the C source code is here:

DVM_ObjectRef SoNewArray(BINT ty,BINT dim)
{
    DVM_TypeSpecifier *type
        = &curthread->current_executable->executable->type_specifier[ty];
    DVM_ObjectRef barray;
    barray = SoCreateArray(curdvm, dim, type);
    return barray;
}

Notice that the final instruction of the disassembly code is "ret 0x4", which means the function it self cleans the stack and it is not a cdecl function! What's more, even if I declare the C function like this:

DVM_ObjectRef SoNewArray(BINT ty,BINT dim) attribute((cdecl));

the resule is the same. Maybe GCC optimizes my code, and automatically use stdcall, ignoring the calling convention?

My GCC command is

gcc -Wall -fexceptions -Wfatal-errors -g

Undine

I have also found that this program also pushes eax (mov DWORD PTR [esp],eax) before "call" instruction. If I compile a simple "hello world" C program, it doesn't pushes eax before "call"s – Undine

So in the end, does GCC ignore __cdecl? Still unanswered. – Oscar

Answer 1

https://www.zhihu.com/question/38726507

Here is an answer from "Zhihu", a Chinese "Stackoverflow". Thanks for RednaxelaFX @ zhihu. I figured out that actually the reason for this problem is that the function returns a structure rather than a basic type. See the function declaration:

DVM_ObjectRef SoNewArray(BINT ty,BINT dim)

DVM_ObjectRef is a structure. When x86 gcc processes a function like above, it actually generates:

DVM_ObjectRef SoNewArray(DVM_ObjectRef * ret,BINT ty,BINT dim)

See more details on x86 calling conventions:

http://www.angelcode.com/dev/callconv/callconv.html

Undine

zhihu is Chinese Stack Exchange, but stack overflow – Williwaw

Answer 2

My gcc 5.4.0 on 64-bit linux requires __attribute__((__cdecl__)) :

#ifdef __GNUC__
#define _cdecl __attribute__((__cdecl__))
#endif

//sometime later
int _cdecl AddItemVar(void *AContext,void *AFun,long ACfg1,...);

The code originally comes from __BORLANDC__, but also compiles with __WATCOMC__ and __MSVC__ . Compilation produces the following warning:

../../LIB/DevOS.C/DevOS.h:242:64: warning: ‘cdecl’ attribute ignored [-Wattributes]

Encrata