Whole screen capture and render in DirectX [PERFORMANCE]

Question

I need some way to get screen data and pass them to DX9 surface/texture in my aplication and render it at at least 25fps at 1600*900 resolution, 30 would be better.

I tried BitBliting but even after that I am at 20fps and after loading data into texture and rendering it I am at 11fps which is far behind what I need.

GetFrontBufferData is out of question.

Here is something about using Windows Media API, but I am not familiar with it. Sample is saving data right into file, maybe it can be set up to give you individual frames, but I haven't found good enough documentation to try it on my own.

My code:

m_memDC.BitBlt(0, 0, m_Rect.Width(),m_Rect.Height(), //m_Rect is area to be captured
               &m_dc, m_Rect.left, m_Rect.top, SRCCOPY); 
      //at 20-25fps after this if I comment out the rest

//DC,HBITMAP setup and memory alloc is done once at the begining
GetDIBits( m_hDc, (HBITMAP)m_hBmp.GetSafeHandle(),
    0L,             // Start scan line
    (DWORD)m_Rect.Height(),     // # of scan lines
    m_lpData,                   // LPBYTE
    (LPBITMAPINFO)m_bi,     // address of bitmapinfo
    (DWORD)DIB_RGB_COLORS);     // Use RGB for color table
     //at 17-20fps

IDirect3DSurface9 *tmp;
m_pImageBuffer[0]->GetSurfaceLevel(0,&tmp); //m_pImageBuffer is Texture of same 
                                            //size as bitmap to prevent stretching
hr= D3DXLoadSurfaceFromMemory(tmp,NULL,NULL,
                             (LPVOID)m_lpData,
                             D3DFMT_X8R8G8B8,
                             m_Rect.Width()*4,
                             NULL,
                             &r,                 //SetRect(&r,0,0,m_Rect.Width(),m_Rect.Height();
                             D3DX_DEFAULT,0);
 //12-14fps
IDirect3DSurface9 *frameS;
hr=m_pFrameTexture->GetSurfaceLevel(0,&frameS); // Texture of that is rendered
pd3dDevice->StretchRect(tmp,NULL,frameS,NULL,D3DTEXF_NONE);
//11fps

I found out that for 512*512 square its running on 30fps (for i.e. 490*450 at 20-25) so I tried dividing screen, but it didn't seem to work well.

If there is something missing in code please write, don't vote down. Thanks

Gauge

Out of curiosity, what's the issue with GetFrontBufferData? I've had moderate success with it in the past (though doing slightly different things). – Boyce

Daniel Sloof: 3 FPS at the best, is't also in documentation that it's slow function. – Gauge

Answer 1

Starting with Windows 8, there is a new desktop duplication API that can be used to capture the screen in video memory, including mouse cursor changes and which parts of the screen actually changed or moved. This is far more performant than any of the GDI or D3D9 approaches out there and is really well-suited to doing things like encoding the desktop to a video stream, since you never have to pull the texture out of GPU memory. The new API is available by enumerating DXGI outputs and calling DuplicateOutput on the screen you want to capture. Then you can enter a loop that waits for the screen to update and acquires each frame in turn.

To encode the frames to a video, I'd recommend taking a look at Media Foundation. Take a look specifically at the Sink Writer for the simplest method of encoding the video frames. Basically, you just have to wrap the D3D textures you get for each video frame into IMFSample objects. These can be passed directly into the sink writer. See the MFCreateDXGISurfaceBuffer and MFCreateVideoSampleFromSurface functions for more information. For the best performance, typically you'll want to use a codec like H.264 that has good hardware encoding support (on most machines).

For full disclosure, I work on the team that owns the desktop duplication API at Microsoft, and I've personally written apps that capture the desktop (and video, games, etc.) to a video file at 60fps using this technique, as well as a lot of other scenarios. This is also used to do screen streaming, remote assistance, and lots more within Microsoft.

Pachton

Could you give a sample on how to connect from the dxgi surface to a file writer sink (or whatever is practical) that does H.264 compression on the GPU and only uses CPU to write stream? There seems to be no valid sample on the internet. No doubt a fast api, but without a simple sample it takes months to figure this out. – Aleris

The new StackOverflow.documentation would be a good place where to put a sample like @Aleris was asking for. – Lauro

Answer 2

If you don't like the FrontBuffer, try the BackBuffer:

LPDIRECT3DSURFACE9  surface;
surface = GetBackBufferImageSurface(&fmt);

to save it to a file use

D3DXSaveSurfaceToFile(filename, D3DXIFF_JPG, surface, NULL, NULL);

Gromwell

App isn't fullscreen and I don't have access to original render target so back buffer is just area of my app. I need to capture whole desktop/whatever is over it. – Gauge