How to record microphone audio in JavaScript and submit to DialogFlow?

Question

How can I record audio from the microphone in JavaScript and submit it to DialogFlow, without going through a server?

Narrows

https://mcmap.net/q/156310/-html5-record-audio-to-file – Consignor

Why the downvote? I asked the question so I could answer it right after with the full explanation. (You don't just need to record the audio, you also have to convert it to one of the specific formats that DialogFlow understands.) – Narrows

Answer 1

There are two parts to this question:

1. How to record microphone audio in a format DialogFlow will understand.
2. How to actually submit that audio to DialogFlow, with proper authentication.

Part 1

For recording microphone audio in a format DialogFlow will understand, I use opus-recorder, then convert the blob it returns using the code below:

function BlobToDataURL(blob: Blob) {
    return new Promise((resolve, reject)=>{
        const reader = new FileReader();
        reader.addEventListener("loadend", e=>resolve(reader.result as string));
        reader.readAsDataURL(blob);
    }) as Promise<string>;
}

const micRecorder = new Recorder({
    encoderSampleRate: 16000,
    originalSampleRateOverride: 16000, // necessary due to Google bug? (https://github.com/chris-rudmin/opus-recorder/issues/191#issuecomment-509426093)
    encoderPath: PATH_TO_ENCODER_WORKER_JS,
});
micRecorder.ondataavailable = async typedArray=>{
    const audioData = new Blob([typedArray], {type: "audio/ogg"});
    const audioData_dataURL = await BlobToDataURL(audioData);
    const audioData_str = audioData_dataURL.replace(/^data:.+?base64,/, "");

    // here is where you need part 2, to actually submit the audio to DialogFlow
};
micRecorder.start();

Part 2

To submit the audio-data to DialogFlow, see my answer here: https://mcmap.net/q/1813316/-how-to-make-http-call-on-dialogflow-v2-using-javascript-ajax-call

Narrows