Debugging/bypassing BSOD without source code

Question

Hello and good day to you.

Need a bit of assitance here:

Situation:
I have an obscure DirectX 9 application (name and application details are irrelevant to the question) that causes blue screen of death on all nvidia cards (GeForce 8400GS and up) since certain driver version. I believe that the problem is indirectly caused by DirectX 9 call or a flag that triggers driver bug.

Goal:
I'd like to track down offending flag/function call (for fun, this isn't my job/homework) and bypass error condition by writing proxy dll. I already have a finished proxy dll that provides wrappers for IDirect3D9, IDirect3DDevice9, IDirect3DVertexBuffer9 and IDirect3DIndexBuffer9 and provides basic logging/tracing of Direct3D calls. However, I can't pinpoint function which causes crash.

Problems:

1. No source code or technical support is available. There will be no assitance, and nobody else will fix the problem.
2. Memory dump produced by kernel wasn't helpful - apparently an access violation happens within nv4_disp.dll, but I can't use stacktrace to go to IDirect3DDevice9 method call, plus there's a chance that bug happens asynchronously.
3. (Main problem) Because of large number of Direct3D9Device method calls, I can't reliably log them into file or over network:
   1. Logging into file causes significant slowdown even without flushing, and because of that all last contents of the log are lost when system BSODs.
   2. Logging over network (using UDP and WINSOck's sendto)also causes significant slowdown and must not be done asynchronously (asynchronous packets are lost on BSOD), plus packets (the ones around the crash) are sometimes lost even when sent synchronously.
   3. When application is "slowed" down by logging routines, BSOD is less likely to happen, which makes tracking it down harder.

Question:
I normally don't write drivers, and don't do this level of debugging, so I have impression that I'm missing something important there's a more trivial way to track down the problem than writing IDirect3DDevice9 proxy dll with custom logging mechanism. What is it? What is the standard way of diagnosing/handling/fixing problem like this (no source code, COM interface method triggers BSOD)?

Minidump analysis(WinDBG):

Loading User Symbols
Loading unloaded module list
...........
Unable to load image nv4_disp.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for nv4_disp.dll
*** ERROR: Module load completed but symbols could not be loaded for nv4_disp.dll
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck 1000008E, {c0000005, bd0a2fd0, b0562b40, 0}

Probably caused by : nv4_disp.dll ( nv4_disp+90fd0 )

Followup: MachineOwner
---------

0: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

KERNEL_MODE_EXCEPTION_NOT_HANDLED_M (1000008e)
This is a very common bugcheck.  Usually the exception address pinpoints
the driver/function that caused the problem.  Always note this address
as well as the link date of the driver/image that contains this address.
Some common problems are exception code 0x80000003.  This means a hard
coded breakpoint or assertion was hit, but this system was booted
/NODEBUG.  This is not supposed to happen as developers should never have
hardcoded breakpoints in retail code, but ...
If this happens, make sure a debugger gets connected, and the
system is booted /DEBUG.  This will let us see why this breakpoint is
happening.
Arguments:
Arg1: c0000005, The exception code that was not handled
Arg2: bd0a2fd0, The address that the exception occurred at
Arg3: b0562b40, Trap Frame
Arg4: 00000000

Debugging Details:
------------------


EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at "0x%08lx" referenced memory at "0x%08lx". The memory could not be "%s".

FAULTING_IP: 
nv4_disp+90fd0
bd0a2fd0 39b8f8000000    cmp     dword ptr [eax+0F8h],edi

TRAP_FRAME:  b0562b40 -- (.trap 0xffffffffb0562b40)
ErrCode = 00000000
eax=00000808 ebx=e37f8200 ecx=e4ae1c68 edx=e37f8328 esi=e37f8400 edi=00000000
eip=bd0a2fd0 esp=b0562bb4 ebp=e37e09c0 iopl=0         nv up ei pl nz na po nc
cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00010202
nv4_disp+0x90fd0:
bd0a2fd0 39b8f8000000    cmp     dword ptr [eax+0F8h],edi ds:0023:00000900=????????
Resetting default scope

CUSTOMER_CRASH_COUNT:  3

DEFAULT_BUCKET_ID:  DRIVER_FAULT

BUGCHECK_STR:  0x8E

LAST_CONTROL_TRANSFER:  from bd0a2e33 to bd0a2fd0

STACK_TEXT:  
WARNING: Stack unwind information not available. Following frames may be wrong.
b0562bc4 bd0a2e33 e37f8200 e37f8200 e4ae1c68 nv4_disp+0x90fd0
b0562c3c bf8edd6b b0562cfc e2601714 e4ae1c58 nv4_disp+0x90e33
b0562c74 bd009530 b0562cfc bf8ede06 e2601714 win32k!WatchdogDdDestroySurface+0x38
b0562d30 bd00b3a4 e2601008 e4ae1c58 b0562d50 dxg!vDdDisableSurfaceObject+0x294
b0562d54 8054161c e2601008 00000001 0012c518 dxg!DxDdDestroySurface+0x42
b0562d54 7c90e4f4 e2601008 00000001 0012c518 nt!KiFastCallEntry+0xfc
0012c518 00000000 00000000 00000000 00000000 0x7c90e4f4


STACK_COMMAND:  kb

FOLLOWUP_IP: 
nv4_disp+90fd0
bd0a2fd0 39b8f8000000    cmp     dword ptr [eax+0F8h],edi

SYMBOL_STACK_INDEX:  0

SYMBOL_NAME:  nv4_disp+90fd0

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: nv4_disp

IMAGE_NAME:  nv4_disp.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  4e390d56

FAILURE_BUCKET_ID:  0x8E_nv4_disp+90fd0

BUCKET_ID:  0x8E_nv4_disp+90fd0

Followup: MachineOwner

Riarial

I don't know anything about debugging video drivers either. Can you upgrade to a version of the driver that isn't buggy? Application code isn't supposed to be able to do things that cause drivers to crash. – Appointor

@Greg Hewgill: "upgrade...?" Bug appeared at least a year ago(maybe two), affects only one application and does not go away with driver upgrade. Instead of debugging it, I'm more interesting in identifying condition that triggers it. My main problem is logging calls - it causes significant slowdown (even when I log calls over network instead of writing to HDD), and not reliable enough. "Application code isn't supposed" There's a driver bug, but nothing else triggers it. At this point I guess NVidia isn't going to fix it any time soon, so writing a proxy dll/interface to bypass it makes sense. – Riarial

Answer 1

Found a solution.

Problem:
Logging is unreliable since messages (when dumped to file) disappear during bsod, packets are sometimes lost when logging over network, and there's slowdown due to logging.

Solution:
Instead of logging to file or over network, configure system to produce full physical memory dump on BSOD and log all messages into any memory buffer. It'll be faster. Once system crashed, it'll dump entire memory into file, and it'll be possible to either view contents of log-file buffer using WinDBG's dt (if you have debug symbols) command, or you'll be able to search and locate logfile stored in memory using "memory" view.

I used circular buffer of std::strings to store messages and separate array of const char* to make things easier to read in WinDBG, but you could simply create huge array of char and store all messages within it in plaintext.

Details:
Entire process on winxp:

1. Ensure that minimum page file size is equal or larger than total amount of RAM + 1 megabytes. (Right Click "My Computer"->Properties->Advanced->Performance->Advanced->Change)
2. Configure system to produce complete memory dump on BSOD (RIght click "My Computer'->Properties->Advanced->Startup and Recovery->Settings->Write Debugging Information . Select "Complete memory dump" and specify path you want).
3. Ensure that disk (where the file will be written) has required amount of free space (total amount of RAM on your system.
4. Build app/dll (the one that does logging) with debug symbol, and Trigger BSOD.
5. Wait till memory dump is finished, reboot. Feel free to swear at driver developer while system writes memory dump and reboots.
6. Copy MEMORY.DMP system produced to a safe place, so you won't lose everything if system crashes again.
7. Launch windbg.
8. Open Memory Dump (File->Open Crash Dump).
9. If you want to see what happened, use !analyze -v command.
10. Access memory buffer that stores logged messages using one of those methods:
    1. To see contents of global variable, use dt module!variable where "module" is name of your library (without *.dll), and "variable" is name of variable. You can use wildcards. You can use address without module!variable
    2. To see contents of one field of the global variable (if global variable is a struct), use dt module!variable field where "field" is variable member.
    3. To see more details about varaible (content of arrays and substructures) use dt -b module!variable field or dt -b module!variable
    4. If you don't have symbols, you'll need to search for your "logfile" using memory window.

At this point you'll be able to see contents of log that were stored in memory, plus you'll have snapshot of the entire system at the moment when it crashed.

Also...

1. To see info about process that crashed the system, use !process.
2. To see loaded modules use lm
3. For info about thread there's !thread id where id is hexadecimal id you saw in !process output.

Riarial

Answer 2

nv4_disp+90fd0
bd0a2fd0 39b8f8000000    cmp     dword ptr [eax+0F8h],edi

This is the important part. Looking at this, it is most probable that eax is invalid, hence attempting to access an invalid memory address.

What you need to do is load nv4_disp.dll into IDA (you can get a free version), check the image base that IDA loads nv4_disp at and hit 'g' to goto address, try adding 90fd0 to the image base IDA is using, and it should take you directly to the offending instruction (depending on section structure).

From here you can analyze the control flow, and how eax is set and used. If you have a good kernel level debugger you can set a breakpoint on this address and try and get it to hit.

Analysing the function, you should attempt to figure out what the function does, what eax is meant to be pointing to at that point, what its actually pointing to, and why. This is the hard part and is a great part of the difficulty and skill of reverse engineering.

Belemnite

Technically, you're telling me to reverse-engineer entire nvidia driver, understand it, locate and fix the problem. Although this is a "proper" solution and problem can be approached from this angle, this is not what I'm looking for (fixing the problem this way requires several months of my time, having source code and debug info is recommended). I'm looking for a way to identify high-level sequence of D3D calls that triggers the bug and prevent that sequence from happening. The main problem is with logging - current function call logging is too slow and results are frequently lost on BSOD. – Riarial

Answer 3

It looks like the crash may either be caused by a bad pointer, or heap corruption. You can tell this because the crash occurs in a memory-freeing function (DxDdDestroySurface). Destroying surfaces is something that you absolutely need to do - you can't just stub this out, the surface will still get freed when the program exits, and if you disable it inside the kernel, you'll run out of on-card memory very quickly and crash that way, as well.

You can try to figure out what sequence of events leads up to this heap corruption, but there's no silver bullet here - as fileoffset suggested, you'll need to actually reverse engineer the driver to see why this happens (it may help to compare drivers before and after the offending driver version as well!)

Tessellation

You're most likely right about bad pointers, since judging from contents of log (trace of all IDirect3DDevice9 calls) I was able to read using methods I mentioned in my own answer, bsod happens after app releases directx memory and destroys IDirect3DVertexBuffer9. My main problem was with recovering logfile, so I should be able to take it from there. – Riarial