How to extract stack traces from minidumps?

Question

I've got a whole bunch of minidumps which were recorded during the runtime of an application through MiniDumpWriteDump. The minidumps were created on a machine with a different OS version than my development machine.

Now I'm trying to write a program to extract stack traces from the minidumps, using dbghelp.dll. I'm walking the MINIDUMP_MODULE_LIST and call SymLoadModule64, but this fails to download the pdbs (kernel32 etc.) from the public symbol server. If I add "C:\Windows\System32" to the symbol path it finds the dlls and downloads the symbols, but of course they don't match the dlls from the minidump, so the results are useless.

So how do I tell dbghelp.dll to download and use the proper pdbs?

[edit]

I forgot to state that SymLoadModule64 only takes a filename and no version/checksum information, so obviously with SymLoadModule64 alone it's impossible for dbghelp to figure out which pdb to download.

The information is actually available in the MINIDUMP_MODULE_LIST but I don't know how to pass it back to the dbghelp API.

There is SymLoadModuleEx which takes additional parameters, but I have no idea if that's what I need or what I should pass for the additional parameters.

[edit]

No luck so far, though I've noticed there's also dbgeng.dll distributed together with dbghelp.dll in the debugging SDK. MSDN looks quite well documented and says it's the same engine as windbg uses. Maybe I can use that to extract the stack traces.

If anyone can point me to some introduction to using dbgeng.dll to process minidumps that would probably help too, as the MSDN documents only the individual components but not how they work together.

Solley

You've possibly made you minidump too mini. Tinker with the DumpType argument. Make sure that the Debug + Windows + Modules list displays accurate DLL path, version and time stamps. – Mukerji

No, that's not a problem, I can load the minidumps in WinDbg just fine and it downloads the PDBs correctly. It's just that I want to automate the stack retrieval instead of inspecting the dumps manually in WinDbg. – Solley

If you want to go the hack route, you could just pass commands down to ntsd and capture the output – Klansman

@Paul Betts - That's not really what I want, as I hoped to extend the automated minidump analysis later, once I got the stackdump extraction working. Though if there's no other solution I'll have to look into it and just take what I can get. – Solley

Answer 1

Just in case anyone else wants to automate extracting stack traces from dumps, here's what I ended up doing:

Like I mentioned in the update it's possible to use dbgeng.dll instead of dbghelp.dll, which seems to be the same engine WinDbg uses. After some trial and error here's how to get a good stack trace with the same symbol loading mechanism as WinDbg.

• call DebugCreate to get an instance of the debug engine
• query for IDebugClient4, IDebugControl4, IDebugSymbols3
• use IDebugSymbols3.SetSymbolOptions to configure how symbols are loaded (see MSDN for the options WinDbg uses)
• use IDebugSymbols3.SetSymbolPath to set the symbol path like you would do in WinDbg
• use IDebugClient4.OpenDumpFileWide to open the dump
• use IDebugControl4.WaitForEvent to wait until the dump is loaded
• use IDebugSymbols3.SetScopeFromStoredEvent to select the exception stored in the dump
• use IDebugControl4.GetStackTrace to fetch the last few stack frames
• use IDebugClient4.SetOutputCallbacks to register a listener receiving the decoded stack trace
• use IDebugControl4.OutputStackTrace to process the stack frames
• use IDebugClient4.SetOutputCallbacks to unregister the callback
• release the interfaces

The call to WaitForEvent seems to be important because without it the following calls fail to extract the stack trace.

Also there still seems to be some memory leak in there, can't tell if it's me not cleaning up properly or something internal to dbgeng.dll, but I can just restart the process every 20 dumps or so, so I didn't investigate more.

Solley

Answer 2

An easy way to automate the analysis of multiple minidump files is to use the scripts written by John Robbins in his article "Automating Analyzing Tons Of Minidump Files With WinDBG And PowerShell" (you can grab the code on GitHub).

This is easy to tweak to have it perform whatever WinDbg commands you'd like it to, if the default setup is not sufficient.

Incoming