Requirements for target application for Visual Studio 11's Graphics Debugger

Question

I was super happy when the "Graphics Debugger" feature was announced for Visual Studio 11, I immediately attempted to get it working on various projects I had, unfortunately I've only managed to get it working for Windows 8 Metro applications! Specifically the C++ Metro project templates run and capture all expected information correctly.

With Developer Preview versions of Visual Studio, and on Windows 7, any attempt to launch applications under the Graphics Debugger (Alt-F5) crash on D3D11CreateDeviceAndSwapChain(), even if they work fine. On my Visual Studio 11 Beta on Windows 8 Consumer Preview machine at work, I get a bit further, the Graphics Debugger HUD displays correctly, but any attempt at getting a capture (Print Screen in the app, or the toolbar button) simply results in a message (in the Output log and a yellow bar at the top of the opened .vsglog):

The Graphics Diagnostics engine couldn't provide the results, most likely because the vsglog is making DirectX calls not supported on this machine.

This is even the case with directly porting the Direct3D calls from the Metro template applications! I don't see any messaging from the MSDN documentation that this feature is only intended for Metro, so I expect I'm simply doing something stupid, but the applications work correctly when not under the Graphics Debugger.

Other information: D3D11_CREATE_DEVICE_DEBUG is set, with the DirectX control panel enabling Direct3D debugging, and I'm getting expected Create/Destroy informational messages and no other in Output.

Beech

Have you also considered reporting this problem as feedback to Microsoft and on the relevant preview forums and mailing lists? Actual Microsoft employees may be better able to answer this question. – Professor

There's a link to here on the Microsoft VS debugger forum. I've taken a few looks around, but haven't yet found an email/blog for someone close to the VS debugger - I'm still looking though. – Beech

I would be happy to ask my professional contacts on your behalf. I know a few people close to the debugging team that might be able to offer you a better answer than the admittedly generic one I've provided below. – Professor

Answer 1

Windows 8: A-ha! Through attaching another debugger to the graphics debugger host VsGraphicsDesktopEngine.exe (found in C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\VsGraphics\x86) and forcing Direct3D debugging through the DirectX Control Panel, I got the output:

DXGI ERROR: No target window specified in DXGI_SWAP_CHAIN_DESC, and no window associated with owning factory. [ UNKNOWN ERROR #7: ]

PIX: IDXGIFactory2::CreateSwapChainForHwnd returned 887a0001

I thought that was pretty weird, since although I'm using CreateSwapChainForHwnd(), it uses DXGI_SWAP_CHAIN_DESC1, which doesn't even have the field OutputWindow. However I tried swapping my use of CreateSwapChainForHwnd() with CreateSwapChain() with the OutputWindow specified and everything works!

With further testing, I found any attempt at D3D11CreateDeviceAndSwapChain() fails, with a null adapter it runs, but captures give the bizzare "This Graphics Diagnostics engine doesn't support D3D9. Playback of your application may be incomplete.", and with a specified adapter it crashes with this stack:

04246c83()
[Frames below may be incorrect and/or missing]
dxgi.dll!CDXGIFactory::CreateSwapChainForHwndImpl(struct IUnknown *,struct DXGI_SWAP_CHAIN_DESC_INTERNAL *,bool,struct IDXGIOutput *,struct IDXGISwapChain1 * *)
dxgi.dll!CDXGIFactory::CreateSwapChain(struct IUnknown *,struct DXGI_SWAP_CHAIN_DESC *,struct IDXGISwapChain * *)
VsGraphicsHelper.dll!CHookedIDXGIFactory::CreateSwapChain(struct IUnknown *,struct DXGI_SWAP_CHAIN_DESC *,struct IDXGISwapChain * *)
VsGraphicsHelper.dll!CSpyHookedIDXGIFactory::CreateSwapChain(struct IUnknown *,struct DXGI_SWAP_CHAIN_DESC *,struct IDXGISwapChain * *)
d3d11.dll!_D3D11CreateDeviceAndSwapChain@48()
VsGraphicsHelper.dll!CHookedD3D11Top::D3D11CreateDeviceAndSwapChain(struct IDXGIAdapter *,enum D3D_DRIVER_TYPE,struct HINSTANCE__ *,unsigned int,enum D3D_FEATURE_LEVEL *,unsigned int,unsigned int,struct DXGI_SWAP_CHAIN_DESC *,struct IDXGISwapChain * *,struct ID3D11Device * *,enum D3D_FEATURE_LEVEL *,struct ID3D11DeviceContext * *)
VsGraphicsHelper.dll!CSpyHookedD3D11Top::D3D11CreateDeviceAndSwapChain(struct IDXGIAdapter *,enum D3D_DRIVER_TYPE,struct HINSTANCE__ *,unsigned int,enum D3D_FEATURE_LEVEL *,unsigned int,unsigned int,struct DXGI_SWAP_CHAIN_DESC *,struct IDXGISwapChain * *,struct ID3D11Device * *,enum D3D_FEATURE_LEVEL *,struct ID3D11DeviceContext * *)
Win32ProjectScratch.exe!Direct3DWindowBase::CreateDeviceResources() Line 363
...

It seems only D3D11CreateDevice() followed by IDXGIFactory::CreateSwapChain() works - either by specifying an adapter or querying the device for it's factory later.

Windows 7 has another gotcha: you have to use D3D11CreateDevice()/IDXGIFactory::CreateSwapChain() still, but it also raises a DirectX debug layer error on the first Present() if you are using a DXGI 1.0 factory (CreateDXGIFactory() vs. CreateDXGIFactory1()):

D3D11: ERROR: ID3D11Device::CreateTexture2D: D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX is only available for devices created off of Dxgi1.1 factories or later. [ STATE_CREATION ERROR #103: CREATETEXTURE2D_INVALIDMISCFLAGS ]

If you have the debug layer "break on severity" option turned on (which in general, you should), this will raise an exception in the debugger, making it look like another crash, however it is safely continuable - however you don't get the VS Graphics Debugger HUD, and the object table window has broken content. On the other hand, if you follow it's advice and use a DXGI 1.1 factory, you get a real crash in Present().

In summary, it seems the graphics debugger isn't quite fully baked yet!

Props to @MrGomez for the idea to debug the debugger

Beech

I looked at the other answer too, but I can't quite figure it out -- how exactly do you attach the debugger toVsGraphicsDesktopEngine.exe, and get that to again attach to your project? – Feoffee

@ReiMiyasaka: Simple, start the first instance, start graphics debugging, then start another instance, then use "Attach to Process" to debug the VsGraphicsDesktopEngine.exe that the first VS launched. (Attach only shows up in some UI configurations: I suggest reloading the "General Development" settings if it's not under the Debug menu) – Beech

Answer 2

Based on the information you've provided, the tool is crashing in D3D11CreateDeviceAndSwapChain because the abstraction layer isn't able to satisfy the necessary Direct3D extensions during rendering. This is corroborated by the error message you've cited, insofar as the Direct3D calls that are being requested are not supported by your current machine.

MSDN provides a help article on this scenario, walking you through using the standard tools to debug this process. In particular, while you have probably already considered it,dxdiag can assist your debugging the current running extensions on your system for Windows 7 or Windows 8 Consumer Preview. You should also be able to save a crash dump when Visual Studio 11 crashes for later debugging, which should illuminate which process call failed.

Please do so and, if possible, update this question with the results. That should illuminate the API call your system requires so we can resolve this issue.

Professor

Full disclosure: I'm not a Microsoft employee, so the art of psychic debugging fails me. :) – Professor

I'm not sure how I'd capture a crash dump when VS isn't crashing - presumably I could catch the first-chance exception if I debug VS though. D3D10 abolished extensions in favor of feature levels (thank god!), so I don't expect that to be the case - but I could try software/WARP, etc.... – Beech

I see. Excuse my confusion. Of course, the difficulty is you're trying to debug the Graphics Debugger, which is itself failing on a graphical fallthrough with no additional information. Oh, the joys of circular dependencies. Can I safely assume you're blocked on this entire help tree, inclusive of every branch in Related Topics down to a maximum of three layers? – Professor

Additionally: if it is not against the terms of service available with VS2011, as you are attempting to research a system dependency problem, the right solution may be to hook the process in a dynamic disassembler like OllyDbg or IDA Pro to step through the execution chain and root out the problem. Clearly, this is a last resort, due to various laws and legal restrictions. IANAL, but if Microsoft grants community testers a research exception, it can be done. – Professor

Managed to solve it - thanks for your help/poking me with a stick! Enjoy the points :) – Beech