Retrieve arguments of a x64 masm assembly procedure

Question

I have a function with the signature :

extern "C" int foo(int a, int b, int c, int d, int e);

which is in fact written in assembly.

With ml(32 bits), using standard calling convention you can pretty much write

.code
foo PROC a: DWORD, b: DWORD ,c: DWORD, d: DWORD, e: DWORD

     mov eax, d
     mov ebx, e

and start using those labels to access your arguments

With ml64 (64 bits) the fastcall is the only convention available. I have no trouble accessing the first arguments stored in the registers, but issues to access the ones in the stack (e in this example): I tried

.code
foo PROC a: DWORD, b: DWORD ,c: DWORD, d: DWORD, e: DWORD

and

.code
foo PROC  e: DWORD

but the value in e is garbage.

I found that if I use the stack address directly I find the value.

.code
foo PROC  e: DWORD

     mov eax, r9                  ; d
     mov ebx, DWORD PTR[rbp + 48] ; e

Is there another way?

Gastroenterology

Answer 1

Documentation explains everything... In Windows, the first four integer parameters are passed in registers RCX, RDX, R8, R9 and floating point in XMM0, XMM1, XMM2, XMM3, anything more than four parameters are passed on the stack above the shadow space. For Unix type OS's it is a bit different.

So, your example is correct - mov ebx, DWORD PTR[rbp + 48] ; e

Shadow space = 32 + saved rbp = 40 + 5th parameter = 48

Carlos

Answer 2

given

extern "C" int foo(int a, int b, int c, int d, int e);

I found out that visual studio 2010 doesn't save the base pointer RBP if

.code
foo PROC

but save the base pointer if

.code
foo PROC  e: DWORD

Later versions (vs2015) don't allow the second code.

There is an optional optimization in x64 systems where RBP is not used (found out the hard way). It says :

The conventional use of %rbp as a frame pointer for the stack frame may be avoided by using %rsp (the stack pointer) to index into the stack frame. This technique saves two instructions in the prologue and epilogue and makes one additional general-purpose register (%rbp) available.

So it is possible that either foo PROC e: DWORD doesnt compile (vs2015), or foo PROC crashes because RBP is null.

The correct way to retrieve stack arguments is to use the RSP stack pointer given that

RBP = RSP + 8 * num_saved_reg

Where num_saved_reg is the number of registers specified in the PROC directive. So when rbp is not saved (otherwise add 8)

PROC -> DWORD PTR[rsp + 40]
PROC use RDI -> DWORD PTR[rsp + 40 + 8]
PROC use RDI RSI RBX -> DWORD PTR[rsp + 40 + 24]

Gastroenterology

The problem with your answer comes when you call into another function; this will clobber the stack pointer and you will get a stack exception. It should work if you never call into another function, but don't quote me on that, and it's safest just to save and restore the stack pointer. – Ophthalmoscopy