How to create a MediaClip from RenderTargetBitmap in an efficent way?

Question

I am trying to capture a video of a XAML grid in UWP app using c#.

My approach.

1.Use RenderTargetBitmap to take a screenshot using renderTargetBitmap.RenderAsync

2.Convert the data to byte array.

3.Create an image file with the bytes and save it on disk using BitmapEncoder

4.Create a MediaClip from that image using MediaClip.CreateFromImageFileAsync

5.Add the clips to a MediaComposition composition.Clips.Add(clip)

6.Save as a video using composition.RenderToFileAsync(video);

Now this approach works.

But as you can imagine, going to disk to save the images and then read them to create the Clips, its SLOWWWW and the framerate is low.

I am looking for something that avoids going to the disk per each screenshot. Something that converts a RenderTargetBitmap (or IBuffer or byte[]) to MediaClips, without going to the disk, or some different approach to save the video.

I am developing UWP app for Hololens.

Lareine

I did not think there will be another way in UWP. – Hampson

how about this #51028400 ? – Lareine

You may take a look at Screen capture - however the note says that you must be running Pro or Enterprise version of W10. – Marchand

Just a wild guess, but IStorageFile is not a file, it's an interface. Maybe you can implement that interface (in that specific case, maybe only some properties and method will be used by the caller methods) to support a "virtual" file. – Quid

@Marchand I want to record a part of the UI, not a GraphicsCaptureItem that is the entire window of the app. Also I dont want the user to choose using a picker. – Lareine

What kind of content is in that grid? Is it possible to maybe only track the elements in the grid (changes of the positions and sizes), and then use that information to create a video? – Halsy

Why dont you convert your RenderTargetBitmap to a WriteableBitmap. When you collected every WriteableBitmap you can transform them to a MediaClip to store them – Widow

Answer 1

Try something like this:

Same as you have done.

using (var soft = SoftwareBitmap.CreateCopyFromBuffer(pixels, BitmapPixelFormat.Bgra8, renderTargetBitmap.PixelWidth, renderTargetBitmap.PixelHeight, BitmapAlphaMode.Premultiplied))
{
   CanvasBitmap canvas = CanvasBitmap.CreateFromSoftwareBitmap(CanvasDevice.GetSharedDevice(), soft); 

   MediaClip m = MediaClip.CreateFromSurface(canvas, DateTime.Now - previousFrame); 
   composition.Clips.Add(m); 
}

Remember to catch the device lost exceptions and create a new device

Caudad

This answer is the best you'll get, but fwiw, you won't get real-time support by using RenderTargetBitmap - it's too slow for that. – Transubstantiation

Answer 2

for those who are getting an exception while trying the answer from @Mediarea, try this:

CanvasRenderTarget rendertarget = null;
using (CanvasBitmap canvas = CanvasBitmap.CreateFromBytes(CanvasDevice.GetSharedDevice(), pixel_buffer, renderTargetBitmap.PixelWidth, renderTargetBitmap.PixelHeight, Windows.Graphics.DirectX.DirectXPixelFormat.B8G8R8A8UIntNormalized))
{
     rendertarget = new CanvasRenderTarget(CanvasDevice.GetSharedDevice(), canvas.SizeInPixels.Width, canvas.SizeInPixels.Height, 96);
     using (CanvasDrawingSession ds = rendertarget.CreateDrawingSession())
     {
         ds.Clear(Colors.Black);
         ds.DrawImage(canvas);
     }
}

MediaClip m = MediaClip.CreateFromSurface(rendertarget, TimeSpan.FromMilliseconds(80));
mc.Clips.Add(m);

If you use this, the error Stream is not in a state to handle the request goes away.

Puerperal