Delphi obtain stack trace after exception

Question

I'm trying to figure out how to obtain a stack trace after an exception is thrown in Delphi. However, when I try to read the stack in the Application.OnException event using the function below, the stack already seems to be flushed and replaced by the throwing procedures.

function GetStackReport: AnsiString;
var
    retaddr, walker: ^pointer;
begin

    // ...

    // History of stack, ignore esp frame
    asm
        mov walker, ebp
    end;

    // assume return address is present above ebp
    while Cardinal(walker^) <> 0 do begin
        retaddr := walker;
        Inc(retaddr);
        result := result + AddressInfo(Cardinal(retaddr^));
        walker := walker^;
    end;
end;

Here's what kind of results I'm getting:

001A63E3: TApplication.HandleException (Forms)
00129072: StdWndProc (Classes)
001A60B0: TApplication.ProcessMessage (Forms)

That's obviously not what I'm looking for, although it's correct. I'd like to retrieve the stack as it was just before the exception was thrown, or in other words the contents before (after would do too) the OnException call.

Is there any way to do that?

I am aware that I'm reinventing the wheel, because the folks over at madExcept/Eurekalog/jclDebug have already done this, but I'd like to know how it's done.

Bonne

How good are you at reading assembler? ;-) – Moia

I never understood why the RTL doesn't have this built in... – Incivility

@WarrenP: reasonably good, or at least good enough to be able to fix this I thought, but I'm out of luck it seems. :( – Bonne

"I'd like to know how it's done." - read the sources. JCL sources are open, same for mORMot and some free loggers and profilers. There are a number of FLOSS products incorporating unwinding stack. You just can read their code and learn from it. – Algology

And if you need to do something and don't have time to hash it out yourself, you could contact madshi (author of MadExcept) and ask him to engineer a solution. That's what I would do. This stuff is HARD. – Moia

Answer 1

It is not possible to manually obtain a viable stack trace from inside the OnException event. As you have already noticed, the stack at the time of the error is already gone by the time that event is triggered. What you are looking for requires obtaining the stack trace at the time the exception is raised. Third-party exception loggers, like MadExcept, EurekaLog, etc handle those details for you by hooking into key functions and core exception handlers inside of the RTL itself.

In recent Delphi versions, the SysUtils.Exception class does have public StackTrace and StackInfo properties now, which would be useful in the OnException event except for the fact that Embarcadero has chosen NOT to implement those properties natively for unknown reasons. It requires third-party exception loggers to assign handlers to various callbacks exposed by the Exception class to generate stack trace data for the properties. But if you have JclDebug installed, for instance, then you could provide your own callback handlers in your own code that use JCL's stack tracing functions to generate the stack data for the properties.

Depressive

The reasons are not unknown at all. The fields were added as a service to help the third party tools. The fields provide a common place for them to place their information. Emba didn't populate them because that's the job of the third party libraries. – Brummell

The reason might be known to those following the newsgroups or other sources. But the online help does not mention it: docwiki.embarcadero.com/Libraries/XE3/en/… – Atlas

Embacadero is best suited to implement stack tracing since it is the one making the compiler in the first place. Prior to exposing the callbacks, third party hooking was the only option. With the callbacks in place, not providing a native solution is a cop-out IMHO. Embarcadero could and should, but they seem to have chosen not to. – Depressive

At the very least, I would like to see Embarcadero add a new unit to the RTL that users could optionally add to their uses clause if desired and it would assign its own handlers to the callbacks and generate basic call stack information. For those users who don't want to buy/install third party loggers. – Depressive

I think they should just give a pile of cash to Madshi and build in MadExcept. (Of course that would tick off the loyal fans of EurekaLog, but as in all holy wars, it seems I am right, and you EurekaLog fans are just wrong.) ;-) – Moia

@WarrenP I would not like to see that. Because then Emba would be in charge of developing it. And I think it's better off with Mathias at the helm. – Brummell

@David Heffernan : 'that's the job of the third party libraries'. Is that the same approach they took with the Delphi Help? It's the job of the users? 'Embarcadero has no further information about this topic...' If Embarcadero doesn't, who is supposed to have the information? Inexcusable. Same here - as Remy said "They have chosen not to" - that is, they have chosen to cut corners and leave us without information - as usual.. – Rabah

Only in the world of Embarcadero could providing an exception stack trace be considered the job of third party libraries. – Epiphora

@WarrenP latest madexcept was pretty bad at discerning which code is part of the stack trace. We could not use it in production as it added little to no business value, littering stack trace with vcl, sysutils, etc while ignoring our own functions and procedures. – Nonu

@Remy: if you want EMBT to provide an own unit with a basic implementation of these callbacks then please create a QP request and add the number here. I'd vote for it. – Uplift

@Uplift a QP already exists (from 2 years ago): RSP-13347 – Depressive

@Remy: thanks for pointing me to it, I voted for it and commented it. – Uplift

Answer 2

I'd like to retrieve the stack as it was just before the exception was thrown, or in other words the contents before (after would do too) the OnException call.

Actually, you don't want the stack before the OnException call. That's what you've already got. You want the stack at the point at which the exception was raised. And that requires the stack tracing to happen ASAP after the raise. It's too late in the OnException call because the exception has propagated all the way to the top-level handler.

madExcept works by hooking all the RTL functions that handle exceptions. And it hooks the lowest level functions. This takes some serious effort to bring about. With these routines hooked the code can capture stack traces and so on. Note that the hooking is version specific and requires reverse engineering of the RTL.

What's more the stack walking is very much more advanced than your basic code. I don't mean that in a derogatory way, it's just that stack walking on x86 is a tricky business and the madExcept code is very well honed.

That's the basic idea. If you want to learn more then you can obtain the source code of JclDebug for free. Or buy madExcept and get its source.

Brummell

Answer 3

I agree with David, and I understand where you are coming from as dependencies on 3rd party are always risky "can't afford to lose it, can't afford to have it"

(I like to share code; I wrote this using Delphi 12, your implementation may differ)

There is nothing wrong with madExcept, but it's not the only one. Have a look at another Paid tool EurekaLog:

Here is the relevant documentation: https://www.eurekalog.com/help/eurekalog/index.php?how_to_convert_call_stack_to_text.php

Or a free version via GetIt Jedi, or get the code in GitHub. Here is how you manually get the call stack:

procedure LogException(const E: Exception);
var
  msg: string;
  StackTrace: string;
begin
   StackTrace := '';
  if not IsNullOrEmpty(E.StackTrace) then
    StackTrace := E.StackTrace
  else
  begin
    {$IFDEF EUREKALOG}
       {  Eurecalog is a bit of a puzzle but one needs following using statement
          for the code to compile:
          IsEurekaLogInstalled() -> ExceptionLog7 or EBase
          IsEurekaLogActive() -> ExceptionLog7 or EBase
          GetTracer(int) ->  ECallStack
          CallStackToString(statce,header) -> ECallStack
          TCompactStackFormatter -> ECallStack
          TracerFramesEurekaLogV7 -> const with the value of 6 in EStackTracing
          https://www.eurekalog.com/help/eurekalog/how_to_convert_call_stack_to_text.php
       }
      if IsEurekaLogInstalled() and  IsEurekaLogActive() and IsNullOrEmpty(StackTrace) then
      begin
        var callStack := GetTracer(TracerFramesEurekaLogV7);
        var formatter:= TCompactStackFormatter.Create;
        try
          CallStack.Build(CallStack.GetCurrentInstruction);
          //use the stack and covert it to a simple string
          StackTrace := CallStackToString(callStack,'Callastack:',formatter);

        finally
          FreeAndNil(callStack);
          FreeAndNil(formatter);
        end;
      end;
   {$ENDIF}
   {$IFDEF JclDebugEnabled }
     if IsNullOrEmpty(StackTrace) then
     begin
       var list := TStringList.Create;
       try
          JclLastExceptStackListToStrings(list,True, True, True, False);
          StackTrace := list.Text;
       finally
          FreeAndNil(list);
       end;
     end;
     {$ENDIF}

  end;

  msg := Format('%s: %s on %s', [E.ClassName, E.Message,StackTrace]);
  Log(msg, etException);//Quick Logger https://blogs.embarcadero.com/quick-logger-is-a-powerful-enterprise-grade-asynchronous-logger-for-delphi/
end;
 

I define if I use JEDI or EurekaLog using a compiler directive in the source of a .dpr file, you could do it in the project options UI as well:

{$DEFINE JclDebugEnabled}

The same goes for EurekaLog; however, the EurekaLog wizard does it for you:

I did not write the code for re-use, but one could take the parts and make it more integration-friendly.

Here is a nice link to help those who like to understand jclDebug https://blog.dummzeuch.de/2014/03/08/using-jcldebug/ as jclDebug needs to be enabled; from the link:

initialization
  // Enable raw mode (default mode uses stack frames which aren't always generated by the compiler)
  Include(JclStackTrackingOptions, stRawMode);
  // Disable stack tracking in dynamically loaded modules (it makes stack tracking code a bit faster)
  Include(JclStackTrackingOptions, stStaticModuleList);
  // Initialize Exception tracking
  JclStartExceptionTracking;
finalization
  JclStopExceptionTracking;
end.

Preferment