How can I capture and stream video of an iframe?

Question

On a website I have an iframe that loads my game (from another domain), I would like to capture and stream it. How can I achieve this? just the contents of this iframe as the user interact with it, not the entire screen or page.

Here's what I tried:

using getDisplayMedia() from Chromium/Mozilla browser api, adding feature-policy: to the HTTP header and allow attribute is what I thought would do the trick but what happened is that it just allows you to use the Screen Capture API which does not seem to support capturing the iframe contents as video.

second, I tried using iframe has allow='displaycapture' but when I call getDisplayUserMedia() the iframe does not show up.

My last ditched desperate attempt is a browser extension that can capture the iframe contents somehow as video stream but that seems tough since its not like the iframe screen output is being streamed as a binary somewhere deep inside the chromium engine.

Congenital

Unfortunately there is still no good API for that... As you found out, gDUM only renders tabs or full screen. In Firefox there are viewportOffsetX& co. constraints you could use to hack something around in order to capture only your iframe, if the user selects the correct tab, and the iframe is shown in screen... But if your game is made on a canvas, maybe it would be good enough to only stream this canvas content? This can be done easily. – Mangosteen

did you mean this? developer.mozilla.org/en-US/docs/Web/API/Visual_Viewport_API – Congenital

No, I was talking about some non-standard constraints that Firefox does expose for gDUM, you can see these by doing await navigator.mediaDevices.getSupportedConstraints(). – Mangosteen

@KJ This question is unrelated. And, at least for my use case, I'm looking at self-owned same-origin iframes. Basically, instead of using CanvasCaptureStream, hoping for an IframeCaptureStream. Ideally, off screen too. In my applications I do this server-side today, but client-side would be ideal without having to involve canvas. The browser can capture a tab with getDisplayMedia()... if it could be called in a way to capture just an iframe, without the user having to choose the right thing, that'd be great. – Cerumen

Answer 1

For security reasons, it's impossible to record an embedded page from a 3rd-party origin. You'll need to have it under the same domain, in which case you can use <iframe allow='displaycapture'/> or add the HTTP header Permissions-Policy: display-capture=(self). Other than that, you're out of luck.

Randarandal

That's only half of the answer though. How can we initiate getDisplayMedia() with a self-capture option? – Cerumen

When you initiate getDisplayMedia(), the user will get a prompt asking what to share, and they themselves will have to choose the current screen. You can also use the preferCurrentTab option to help guide the user. Unfortunately, there isn't a way to directly select which window to share in code, the user must do it themselves. – Randarandal

Ok, so the API that only allows selection of the current tab... that does not exist yet? Is not implemented yet? (I know i"ve read a discussion about it... cannot find it...) – Cerumen

I didn't realize that, but I guess you'll have to manage without it until it gets implemented. This is unfortunately a really hard feature to implement, and in most cases it's better to leave it to dedicated streaming platforms like Twitch. If you really want to, however, this is the best you can do. – Randarandal

Found it: docs.google.com/presentation/d/… – Cerumen