Capture visual output of a DirectX application - even in background?

Question

I need to capture the visual output (like a screenshot) of a DirectX window. Currently, I use this approach.
But, when the window is in background, it captures whatever is in front of it.

I see that DirectX windows render even when minimized or in background, so this should be possible.
But, how? (It also needs to be fast, and it needs to work on Windows XP too, unfortunately...)

Edit: I am very busy these days... Don't worry, I'll put the bounty back if it expires.

Strapping

I don't know if this is applicable to your project but one, theoretical, solution I know of is to build a delay into the visual output capture function so that you can switch back to the DirectX window before it starts recording. – Tella

I don't understand what you mean... – Strapping

You say that you want to capture what's rendered by the DirectX window - even if said window is in the background. If the reason you want to do that is because, for example, there's a delay between starting the recording program and switching to the DirectX window then you could try building a delay into the program to allow you to switch to the DirectX window. That probably isn't your situation but, if it is, that method could help. – Tella

Nope. I just need to capture the output of a DirectX window and process the bitmap. – Strapping

Answer 1

To capture Direct3D windows that are in the background (or moved off screen), I believe you have the following options:

1. Inject and hook Direct3D within the target application via the link you have already posted or this more up-to-date example (EasyHook can be difficult to get setup but it does work really well) - you can always ask for help about getting it working. I have used that technique for capturing in a number of games without issues (most recently for an ambilight-clone project). The problem with this approach is your concern about game protection causing bans, however FRAPs also uses hooking to achieve this, so perhaps your concerns are exaggerated? I guess gamers being banned for a screen shot is an expensive way of finding out.

2. For windowed applications on Vista/Win 7 - you could inject and hook the DWM and make your capture requests through its shared surface. I have had this working on Vista, but have not finished getting it working on Windows 7, here is an example of it working for Windows 7 http://www.youtube.com/watch?v=G75WKeXqXkc. The main problem with this approach is the use of undocumented API's which could mean your application breaks without any warning upon a windows patch release - also you would have to redo the technique for each new major Windows flavour. This also does not address your need to capture in Windows XP.

3. Also within the DWM, there is a thumbnail API. This has limitations depending on what your trying to do. There is some information on this API along with other DWM API's here http://blogs.msdn.com/b/greg_schechter/archive/2006/09/14/753605.aspx

4. There are other techniques for intercepting the Direct3D calls without using EasyHook, such as substituting the various DLL's with wrappers. You will find various other game hooking/interception techniques here: http://www.gamedeception.net/

5. Simply bring the Direct3D application to the foreground (which I guess is undesirable in your situation) - this wouldn't work for off-screen windows unless you also move the window.

6. Unfortunately the only solution for Windows XP that I can think of is intercepting the Direct3D API in some form.

Just a clarification on Direct3D rendering while minimised. During my fairly limited testing on this matter I have found this to be application dependant; it is generally not recommended that rendering take place while the application is minimized (also this reference), it does continue to render while in the background however.

UPDATED: provided additional link to more up-to-date injection example for point 1.

Antarctica

I forgot to mention that EasyHook also provides kernel/driver level support for hooking - I'm guessing that this is similar to how people like PLAYXPERT have their game overlay system working. – Antarctica

I have done some research and found out that most game protectors use drivers to monitor the games... There is no way I can safely inject code into a process without a "developer license" from the game protector(s). There must be another way... How the heck does DWM get the thumbnails in the first place?? – Strapping

I believe DWM uses shared surfaces and DXGI, a good overview of this is here: msdn.microsoft.com/en-us/library/windows/desktop/… I don't think it is possible to initiate a shared surface without first having a reference to the Device so you still need something in the target application. – Antarctica

I should clarify that I don't think it is possible to get a reference to the Device without having something injected in the target application OR by using undocumented API's – Antarctica

Then I guess I will risk hooking... :| – Strapping

i once made a game in xna, i had launched visual studio from steam. when i started debugging i could access the steam overlay(shift+tab). which told me that the steam overlay hooks itself into directx – Scold

Answer 2

A quick google and i found this Code Project which relates to Windows XP. I dont know if you can apply this knowledge to Windows Vista and 7??

http://www.codeproject.com/Articles/5051/Various-methods-for-capturing-the-screen

EDIT:

I found this article as well:

http://www.codeproject.com/Articles/20651/Capturing-Minimized-Window-A-Kid-s-Trick

This links off from Justins blog post here from the comments. It seems he was working on this with someone (i see thats your link about).

http://spazzarama.com/2009/02/07/screencapture-with-direct3d/

Embarrass

Did I not mention C#? Also, it seems to look at the "front buffer", which is the whole screen. I need the buffer of the application explicitly. – Strapping

APologies i didnt see that. Mmmmmnnnn im trying to think what i would do in XNA (I know this isnt DirectX) and think of a solution. Let me think. – Embarrass

Uh, lemme expwain this. The Code Project article you linked me doesn't work with DirectX windows. Some do not show up at all, some appear gray. If I remember right, windows with no border style also don't work. Now, the screen capturing with Direct3D article involves hooking into the application. The problem is, most DirectX applications are games, and most games are protected by things such as VAC and Game Guard, which will kill the game process instantly and give me a warning or an immediate permanent ban. – Strapping

By the capturing with Direct3D article I meant the Direct3D API article for either D3D9 or 9, 10 and 11... None of them work because I cannot get EasyHook to work, and even if I could, it could trigger a game protector and mess me up really bad. – Strapping

If hooking the target application is absolutely not possible, then you can look at going via the DWM (not on XP obviously) - only works for windowed applications not fullscreen. FRAP's uses hooking also so I'm sure there is a level of hooking that is tolerated by game protection techniques but what that is I don't have any idea... Re: EasyHook, there is work under way for simplifying the process to no longer require installing into the GAC - you will find the results of this in this project once it all passes alpha (github.com/spazzarama/Direct3DHook) – Antarctica

I looked into DWM... But I cannot simply get the capture on a bitmap and I have to use WPF (if it's possible to draw a Visual on a bitmap). This approach has two huge disadvantages, and I honestly tried it... I just can't get the image on a bitmap. If I draw it on a Control and I try to convert it to a bitmap, I get the background of the control, because DWM draws directly in video memory. – Strapping

Answer 3

The code that you linked to (from spazzarama), which you said you were using in your project, captures the front buffer of your DirectX device. Have you tried capturing the back buffer instead? Going from the code on your linked site, you would change line 90 from

device.GetFrontBufferData(0, surface);

to

Surface backbuffer = device.GetBackBuffer(0, 0, BackBufferType.Mono);
SurfaceLoader.Save("Screenshot.bmp", ImageFileFormat.Bmp, backbuffer);

This would also involve removing lines 96-98 in your linked example. The backbuffer might be generated without the obstructing window.

EDIT

Nevermind all of that. I just realized that your linked sample code is using the window handle to define a region of the screen, and not actually doing anything with the DirectX window. Your sample code won't work around the obstruction because your region is already drawn with the other window in front of it by the time you access it.

Your best bet to salvage the application is probably to bring the DirectX window to the top of the screen before running the code to capture the image. You can use the Wind32API BringWindowToTop function to do that (http://msdn.microsoft.com/en-us/library/ms632673%28VS.85%29.aspx).

Looseleaf

I'm already using SetForegroundWindow for this, but it requires complex interprocess communication to make sure that two programs do not jump in front at the same time... It is highly inefficient and bottlenecks my application. – Strapping