Delphi - Obtain Full Stack Trace on OSX

Question

I have an application which can log a stacktrace, which can be later used for debugging.

On Windows, I've gotten by using the excellent JCLDebug unit provided by the JEDI project.

Now that my application is running on OSX, I've hit a bit of a hitch - I don't know how to obtain the correct stacktrace when an exception occurs.

I have got the basics down -

1) I can get a stacktrace using 'backtrace' (found in libSystem.dylib)

2) The resulting backtrace can be converted into line numbers using the .map file provided by Delphi's linker

The issue I'm left with is - I don't know where to call backtrace from. I know that Delphi uses Mach exceptions (on a separate thread), and that I cannot use posix signals, but that's all that I've managed to sort out.

I can get a backtrace in the 'try...except' block, but unfortunately, by that point the stack has already wound down.

How can I install a proper exception logger which will run right after the exception occurs?

Update:

As per 'Honza R's suggestion, I've taken a look at the 'GetExceptionStackInfoProc' procedure.

This function does get me 'inside' of the exception handling process, but unfortunately leaves me with some of the same issues I had previously.

First of all - on desktop platforms, this function 'GetExceptionStackInfoProc' is just a function pointer, which you can assign with your own exception info handler. So out of the box, Delphi doesn't provide any stack information provider.

If I assign a function to 'GetExceptionStackInfoProc' and then run a 'backtrace' inside of it, I receive a stacktrace, but that trace is relative to the exception handler, not the thread which caused the exception.

'GetExceptionStackInfoProc' does contain a pointer to a 'TExceptionRecord', but there's very limited documentation available on this.

I might be going beyond my depth, but how can I get a stacktrace from the correct thread? Would it be possible for me to inject my own 'backtrace' function into the exception handler and then return to the standard exception handler from there?

Update 2

Some more details. One thing to clear up - this question is about exceptions that are handled by MACH messages, not software exceptions that are handled entirely within the RTL.

Embarcadero has laid out some comments along with these functions -

    System.Internal.MachExceptions.pas -> catch_exception_raise_state_identity

    {
     Now we set up the thread state for the faulting thread so that when we
     return, control will be passed to the exception dispatcher on that thread,
     and this POSIX thread will continue watching for Mach exception messages.
     See the documentation at <code>DispatchMachException()</code> for more
     detail on the parameters loaded in EAX, EDX, and ECX.
    }

    System.Internal.ExcUtils.pas -> SignalConverter

    {
      Here's the tricky part.  We arrived here directly by virtue of our
      signal handler tweaking the execution context with our address.  That
      means there's no return address on the stack.  The unwinder needs to
      have a return address so that it can unwind past this function when
      we raise the Delphi exception.  We will use the faulting instruction
      pointer as a fake return address.  Because of the fencepost conditions
      in the Delphi unwinder, we need to have an address that is strictly
      greater than the actual faulting instruction, so we increment that
      address by one.  This may be in the middle of an instruction, but we
      don't care, because we will never be returning to that address.
      Finally, the way that we get this address onto the stack is important.
      The compiler will generate unwind information for SignalConverter that
      will attempt to undo any stack modifications that are made by this
      function when unwinding past it.  In this particular case, we don't want
      that to happen, so we use some assembly language tricks to get around
      the compiler noticing the stack modification.
    }

Which seem to be responsible for the issue I'm having.

When I do a stacktrace after this exception system has handed control over to the RTL, it looks like this - (bearing in mind, the stack unwinder has been superseded by a backtrace routine. The backtrace will hand control over to the unwinder once it is completed)

0: MyExceptionBacktracer
1: initunwinder in System.pas
2: RaiseSignalException in System.Internal.ExcUtils.pas 

Since RaiseSignalException is called by SignalConverter, I'm led to believe that the backtrace function provided by libc is not compatible with the modifications made to the stack. So, it's incapable of reading the stack beyond that point, but the stack is still present underneath.

Does anyone know what to do about that (or whether my hypothesis is correct)?

Update 3

I've finally managed to get proper stacktraces on OSX. Huge thanks to both Honza and Sebastian. By combining both of their techniques, I found something that works.

For anyone else who could benefit from this, here's the basic source. Bear in mind that I'm not quite sure if it's 100% correct, if you can suggest improvements, go ahead. This technique hooks onto an exception right before Delphi unwinds the stack on the faulting thread, and compensates for any stack frame corruption that might have taken place beforehand.

unit MyExceptionHandler;

interface

implementation

uses
  SysUtils;

var
  PrevRaiseException: function(Exc: Pointer): LongBool; cdecl;

function backtrace2(base : NativeUInt; buffer : PPointer; size : Integer) : Integer;
var SPMin   : NativeUInt;
begin
  SPMin:=base;
  Result:=0;
  while (size > 0) and (base >= SPMin) and (base <> 0) do begin

    buffer^:=PPointer(base + 4)^;
    base:=PNativeInt(base)^;

    //uncomment to test stacktrace
    //WriteLn(inttohex(NativeUInt(buffer^), 8));

    Inc(Result);
    Inc(buffer);
    Dec(size);

  end;
  if (size > 0) then buffer^:=nil;
end;

procedure UnInstallExceptionHandler; forward;

var
  InRaiseException: Boolean;

function RaiseException(Exc: Pointer): LongBool; cdecl;
var b : NativeUInt;
    c : Integer;
    buff : array[0..7] of Pointer;
begin
  InRaiseException := True;

  asm
    mov b, ebp
  end;

  c:=backtrace2(b - $4 {this is the compiler dependent value}, @buff, Length(buff));
  //... do whatever you want to do with the stacktrace

  Result := PrevRaiseException(Exc);
  InRaiseException := False;
end;

procedure InstallExceptionHandler;
var
  U: TUnwinder;
begin
  GetUnwinder(U);
  Assert(Assigned(U.RaiseException));
  PrevRaiseException := U.RaiseException;
  U.RaiseException := RaiseException;
  SetUnwinder(U);
end;

procedure UnInstallExceptionHandler;
var
  U: TUnwinder;
begin
  GetUnwinder(U);
  U.RaiseException := PrevRaiseException;
  SetUnwinder(U);
end;

initialization
  InstallExceptionHandler;
end.

Blanc

Answer 1

You can use GetExceptionStackInfoProc, CleanUpStackInfoProc and GetStackInfoStringProc in Exception class you need to save stack trace in GetExceptionStackInfoProc and then retrieve it with GetStackInfoStringProc which will get called by RTL if you use StackTrace property of the Exception. Maybe you could also take look at https://bitbucket.org/shadow_cs/delphi-arm-backtrace which demonstrates this on Android.

To do this properly on Mac OS X the libc backtrace function cannot be used because Delphi will corrupt stack frame when calling the GetExceptionStackInfoProc from Exception.RaisingException. Own implementation must be used that is capable of walking the stack from different base address which can be corrected by hand.

Your GetExceptionStackInfoProc would then look like this (I used XE5 for this example the value added to EBP bellow may differ based on which compiler you use and this example was only tested on Mac OS X, Windows implementation may or may not differ):

var b : NativeUInt;
    c : Integer;
    buff : array[0..7] of Pointer;
begin
  asm
    mov b, ebp
  end;
  c:=backtrace2(b - $14 {this is the compiler dependent value}, @buff, Length(buff));
  //... do whatever you want to do with the stacktrace
end;

And the backtrace2 function would look like this (note that stop conditions and other validations are missing in the implementation to ensure that AVs are not caused during stack walking):

function backtrace2(base : NativeUInt; buffer : PPointer; size : Integer) : Integer;
var SPMin   : NativeUInt;
begin
  SPMin:=base;
  Result:=0;
  while (size > 0) and (base >= SPMin) and (base <> 0) do begin
    buffer^:=PPointer(base + 4)^;
    base:=PNativeInt(base)^;
    Inc(Result);

    Inc(buffer);
    Dec(size);
  end;
  if (size > 0) then buffer^:=nil;
end;

Retainer

Awesome. I'll test this out when I get a chance, and check off your answer when I get it working! – Blanc

If you're trying to use this to catch AVs you this may not work (I didn't test that on Andorid) but since this uses a signal which would most likely terminate your iOS app. But I guess you should be just fine with 'ordinary' Exceptions. – Retainer

I've tried this out, and updated the original question. It's a start but I'm still unsure how to sort this one out. – Blanc

There are even more problems about this GetExceptionStackInfoProc does contain pointer to TExceptionRecord but not all fields of that record are initialized (especially on ARM). I also believe that the thread is indeed the same one but there appears to be some stack corruption (inconsistency) which make the backtrace function not to work properly, as you can see even Delphi Call stack is not complete. I'm talking about Mac OS on x86 here. – Retainer

I updated the answer to be more specific to your problem, I will also push these changes to bitbucket after a bit more testing. – Retainer

Ah, I've been working on this all day, also realizing that the backtrace function will not be useable. I'll try working through your solution. I also edited the question to point out where the stack corruption occurred. – Blanc

It took a little bit of messing around, but I've had some luck with this technique. – Blanc

Is there something you could add that led to your final solution? Did you have to modify backtrace2 function further or were there other mach exception issues you had to solve? – Retainer

I'll update the question soon, to explain my solution. One important thing was that I used a combination of Sebastian's answer along with yours. (ie: I did not use GetExceptionStackInfoProc) – Blanc

I posted the source that's working for me into the original question. – Blanc

Answer 2

You could hook yourself into the Exception Unwinder. Then you can call backtrace where the exception happens. Here's an example. The unit SBMapFiles is what I use for reading the mapfiles. It is not required to get the exception call stack.

unit MyExceptionHandler;

interface

implementation

uses
  Posix.Base, SysUtils, SBMapFiles;

function backtrace(result: PNativeUInt; size: Integer): Integer; cdecl; external libc name '_backtrace';
function _NSGetExecutablePath(buf: PAnsiChar; BufSize: PCardinal): Integer; cdecl; external libc name '__NSGetExecutablePath';

var
  PrevRaiseException: function(Exc: Pointer): LongBool; cdecl;
  MapFile: TSBMapFile;

const
  MaxDepth = 20;
  SkipFrames = 3;

procedure ShowCurrentStack;
var
  StackLog: PNativeUInt; //array[0..10] of Pointer;
  Cnt: Integer;
  I: Integer;
begin
  {$POINTERMATH ON}
  GetMem(StackLog, SizeOf(Pointer) * MaxDepth);
  try
    Cnt := backtrace(StackLog, MaxDepth);

    for I := SkipFrames to Cnt - 1 do
    begin
      if StackLog[I] = $BE00EF00 then
      begin
        WriteLn('---');
        Break;
      end;
      WriteLn(IntToHex(StackLog[I], 8), ' ', MapFile.GetFunctionName(StackLog[I]));
    end;

   finally
    FreeMem(StackLog);
   end;
  {$POINTERMATH OFF}
end;

procedure InstallExceptionHandler; forward;
procedure UnInstallExceptionHandler; forward;

var
  InRaiseException: Boolean;

function RaiseException(Exc: Pointer): LongBool; cdecl;
begin
  InRaiseException := True;
  ShowCurrentStack;

  Result := PrevRaiseException(Exc);
  InRaiseException := False;
end;

procedure InstallExceptionHandler;
var
  U: TUnwinder;
begin
  GetUnwinder(U);
  Assert(Assigned(U.RaiseException));
  PrevRaiseException := U.RaiseException;
  U.RaiseException := RaiseException;
  SetUnwinder(U);
end;

procedure UnInstallExceptionHandler;
var
  U: TUnwinder;
begin
  GetUnwinder(U);
  U.RaiseException := PrevRaiseException;
  SetUnwinder(U);
end;

procedure LoadMapFile;
var
  FileName: array[0..255] of AnsiChar;
  Len: Integer;
begin
  if MapFile = nil then
  begin
    MapFile := TSBMapFile.Create;
    Len := Length(FileName);
    _NSGetExecutablePath(@FileName[0], @Len);
    if FileExists(ChangeFileExt(FileName, '.map')) then
      MapFile.LoadFromFile(ChangeFileExt(FileName, '.map'));
  end;
end;

initialization
  LoadMapFile;
  InstallExceptionHandler;
end.

Intercalary

I put some time into this one tonight - as much as I appreciate the effort - I'm getting the same results as a previous solution. I'll keep working on it a bit, but it seems like the stacktrace is nearly identical to what I had before. – Blanc

Also, I did some hunting, and apparently the unwinder that we're placing the stacktrace prior to is provided in a .dylib, not in source. So no looking at how that works. Was really hoping there would be some clues in there. – Blanc

I have the following function: procedure TForm1.Button3Click(Sender: TObject); begin raise EProgrammerNotFound.Create('Error Message'); end; And I get the following as stack trace: 005A59EB Unit1.TForm1.Button3Click 00356F46 FMX.Controls.TControl.Click 00237D0D FMX.StdCtrls.TCustomButton.Click 0040383F FMX.Forms.TCommonCustomForm.MouseUp 00506DFA FMX.Platform.Mac.TPlatformCocoa.MouseEvent 00506FFD FMX.Platform.Mac.TPlatformCocoa.MouseEvent 0050D4BC FMX.Platform.Mac.TFMXViewBase.mouseUp 002D123A Macapi.ObjectiveC.DispatchT... Could it be that your real problem is reading the map file? – Intercalary

I've been able to read the .map fine, but the only thing that I get is 'RaiseException' and 'ShowCurrentStack' on the top of the stack! Out of curiosity, which version of Delphi are you using? I created a completely blank project and added your source just to check it again... no luck yet! – Blanc

I used Delphi XE5 Update 2 and Delphi XE6 for testing. OSX Version = 10.9.2 – Intercalary

After reading through System.Internal.ExcUtils, I'm starting to think that I'm on the correct thread after all, but Delphi has been doing a lot of tricky adjustments to the stack. I'll test on XE5 Update 2 tomorrow and see if I can duplicate your results. (I tested XE4 thus far) – Blanc

I'm not fully convinced about the way you interpret your map file, unless you changed SkipFrames from 3 to 0. This is for skipping the 3 top entries on the stack ShowCurrentStack, RaiseException and SysRaiseException. So these should not really appear. Some time ago I wrote a utility that merges a OSX crash report and a Delphi map file. I updated the Windows version of this tool so you can enter an address like $00cb54f4 instead of the path to the crash report. That way you can verify if you get the same address from the map file as I do. link – Intercalary

(I did switch SkipFrames to 0) – Blanc

I've been able to duplicate your results, if I place raise Exception.Create('test'); into my code. Unfortunately, this example is only using the RTL, and doesn't actually trigger a MACH exception. If I simulate an exception which won't get handled by the RTL directly (ie: a dylib failing), then I won't be able to collect a proper stacktrace. – Blanc

If I may ask, where does the "SBMapFiles" unit come from? I've not been able to find any info, but it sounds like something I could use! – Nonchalance

SBMapFiles is a unit I wrote a while ago. It's for reading map files and getting function names from addresses. I haven't published it. You can see it in action in delphi.zierer.info/CrashReportMerger.zip (sorry, no source). That tool takes a Delphi map file and an OSX crash report and makes something more useful out of it. – Intercalary