Custom SEH handler with /SAFESEH

Question

I am currently trying to create a compiler in C++, which generates machine code at run time. However, I am currently trying to enable safe exception handling (compiling with /SAFESEH). My custom exception handler is working in the debug mode, but when I run the same code in release mode, my process is just terminated.

I am quite certain the problem is that I fail to register my custom exception handler as such, because when I compile my code with /SAFESEH:NO, everything works fine even in release mode.

My custom exception handler is written in my other C++-code, and I have tried to register it as an exception handler by adding a .asm-file to my project with the content:

.386
.model flat
_MyExceptionHandler@16 proto
.safeseh _MyExceptionHandler@16
end

as described here. The asm-file was then assembled with the /safeseh option (among others).

My handler function currently has the following declaration:

extern "C" EXCEPTION_DISPOSITION __stdcall MyExceptionHandler(struct
_EXCEPTION_RECORD *ExceptionRecord, void * EstablisherFrame, struct 
_CONTEXT *ContextRecord, void * DispatcherContext);

What would the correct way to register this function as an exception handler be?

Thanks for any suggestions!

Unclose

Answer 1

I finally found a page describing the problem: here. However, the code sample in the page did not work without modification.

The problem seems to be that registering external procedures as exception handler does not work as intended, so you have to register a local assembly procedure as an exception handler instead.

Based on the example given on the previously mentioned page, this is what I ended up with:

.386
.model flat, stdcall
option casemap :none

extern MyExceptionHandler@16:near

MyExceptionHandlerAsm proto
.SAFESEH MyExceptionHandlerAsm

.code
MyExceptionHandlerAsm proc
    jmp MyExceptionHandler@16
MyExceptionHandlerAsm endp
end

This seems to work, but it is most probably not the most elegant solution. For example: to avoid linking errors when referencing MyExceptionHandlerAsm from C/C++, I had to declare it as:

extern "C" int __stdcall MyExceptionHandlerAsm();

Which would crash if anyone tried to call MyExceptionHandlerAsm from C/C++, since the number of parameters does not match the ones of MyExceptionHandler.

Unclose

Does this work on x64 as well? If I compile this using a 32 bit VC compiler, and run it on a 64 bit windows, will the customized exception handler be invoked? – Heddy

As long as you compile it for x86, Windows will run 32 bit applications on both 32-bit and 64-bit Windows. This is what I do all the time. However, I have not tested compiling the code using a 64 bit compiler. – Unclose

Answer 2

From the MSDN documentation it looks like your ASM is correct. Did you add the /safeseh option to ml.exe ?

Deglutition

Thanks for the answer! Yes, I added /safeseh to ml.exe, otherwise I was given a warning. I will note it above as well. – Unclose

Answer 3

The signature of the ExceptionHandler function is actually:

typedef EXCEPTION_DISPOSITION (*PEXCEPTION_ROUTINE) (
    __in struct _EXCEPTION_RECORD *ExceptionRecord,
    __in PVOID EstablisherFrame,
    __inout struct _CONTEXT *ContextRecord,
    __inout PVOID DispatcherContext
    );

See this MSDN page. The same exception routine is used for 32-bit windows, so ignore the fact that the doc mentions x64.

Bennir