How can I get audio stream playing in a tab in chrome extension manifest v3

Question

In order to capture output audio stream from a tab in manifest v2 one could use chrome.tabCapture.capture API in the background script to get the stream. But, in manifest v3 tabCapture has been moved to foreground and it isn't available in background script. Neither it is available in content scripts.

After trying out a few things, I could finally make tabCapture work in popup script. However, in my use case popup can't remain open for a long time for the stream to get captured. Instead, I stumbled upon getMediaStreamId method of tabCapture which gives me a streamId. So, I came up with this:

chrome.tabs.query({
  active: true,
  currentWindow: true
}, (tabs) => {
  var tab = tabs[0];
  chrome.tabCapture.getMediaStreamId({consumerTabId: tab.id}, (streamId) => {
    chrome.tabs.sendMessage(tab.id, { action: 'streamId', streamId: streamId });
  });
});

In the above script which is present in popup.js file, after getting the streamId, I send that to the content script. And I'm stuck here on to how to create a stream out of this streamId so that the stream can be captured/recorded. The documentation mentions using getUserMedia() but I haven't been lucky there. This is what I could come up with via a little help from other SO questions (but it doesn't work, I get this error DOMException: Error starting tab capture):

navigator.mediaDevices.getUserMedia({
  audio: {
    mandatory: {
      chromeMediaSource: 'tab',
      chromeMediaSourceId: streamId
    }
  }
})

How can I make tabCapture work in manifest v3, is what I'm trying to find out.

Doby

Add an extension iframe to the web page (see web_accessible_resources) and capture the audio inside. – Florous

If the site is displaying a stream and you want to record it, there is an API to do so; you can clone the stream and render it through canvas as well and modify it how you'd like. – Gustation

@Gustation No, site isn't displaying a stream. – Doby

Hi @wOxxOm, I tried opening the popup.html inside an iframe. But the moment tabCapture.capture API gets called, I get an error like this: Extension has not been invoked for the current page which is obvious since the src of the iframe is chrome-extension://<extension_runtime_id>/popup.html. Could you provide more details around the way you suggested? – Doby

Show your UI inside the iframe from the beginning so the click is handled in the same context. The initial UI can be small i.e. just a button, auto-resized on a click inside or when hovered. – Florous

@wOxxOm Tried a few things, but I'm not able to achieve this. Can you please point me to any example/code snippet? – Doby

I'm still trying to work out the best method for capturing audio, but I've got a working solution which I've posted below. The primary issue is that it stops capturing if the extension pop up is closed. – Overspread

Answer 1

here's an solution for capturing audio within the extension pop up, as opposed to the service worker.

manifest.json

{
    "name": "Audio Share",
    "version": "0.1",
    "manifest_version": 3,
    "permissions": [
        "tabs",
        "activeTab",
        "tabCapture"
    ],
    "host_permissions": [
        "https://*/"
    ],
    "action": {
        "default_popup": "home.html",
        "default_action": "home.js"
    }
}

home.js

function saveToFile(blob, name) {
    const url = window.URL.createObjectURL(blob);
    const a = document.createElement("a");
    document.body.appendChild(a);
    a.style = "display: none";
    a.href = url;
    a.download = name;
    a.click();
    URL.revokeObjectURL(url);
    a.remove();
}

function captureTabAudio() {
    chrome.tabCapture.capture({audio: true, video: false}, (stream) => {

        // these lines enable the audio to continue playing while capturing
        context = new AudioContext();
        var newStream = context.createMediaStreamSource(stream);
        newStream.connect(context.destination);

        const recorder = new MediaRecorder(stream);
        const chunks = [];
        recorder.ondataavailable = (e) => {
            chunks.push(e.data);
        };
        recorder.onstop = (e) => saveToFile(new Blob(chunks), "test.wav");
        recorder.start();
        setTimeout(() => recorder.stop(), 5000);
    })
}

document.addEventListener('DOMContentLoaded', function () {
    document.getElementById("share-audio-button").addEventListener("click", function ()
        captureTabAudio();
    })
});

home.html

<html>
    <head>
        <script src="home.js"></script>
    </head>

    <body>
        <h1>Audio Share</h1>
        <button id="share-audio-button">Share Audio</button>
    </body>
</html>

Overspread

Your code does work, however you failed to mention that it only works for as long as the popup remains open. This means no clicking outside the extension popup. – Spheroidal

Yea good call out. I didn't get far enough to determine how else to do this. I'm not clear if MV3 is attempting to crack down on this use case for some reason? Or I just haven't worked out how it's suppose to be handled. I think I went back to MV2 for my purposes. – Overspread

Answer 2

The sidePanel API is the most exciting feature released this year. Unlike the popup, it remains open and it has access to all chrome APIs.

Cupbearer

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. – Ostiole