WebRTC - disable all audio processing

Question

I'm currently trying to get a clean as possible audio channel via webrtc. Via the getUserMedia mediaconstraints object, I've set the following options:

constraints: {
        audio: {
            mandatory: {
                echoCancellation: false,
                googEchoCancellation: false,
                googAutoGainControl: false,
                googAutoGainControl2: false,
                googNoiseSuppression: false,
                googHighpassFilter: false,
                googTypingNoiseDetection: false,
                //googAudioMirroring: false // For some reason setting googAudioMirroring causes a navigator.getUserMedia error:  NavigatorUserMediaError
            }
        },
        video: false
    },

This greatly improves the audio quality, but there still seems to be audio processing present which causes the mutilation of the audio in the form of high frequency noise with some of the test samples.

There is a Chrome flag --use-file-for-fake-audio-capture as described at http://peter.sh/experiments/chromium-command-line-switches/#use-file-for-fake-audio-capture which allows input via file for testing. As mentioned in the description of the flag, all audio processing has to be disabled or the audio will be distorted - so there seems to be additional options for this purpose.

I also tried the --disable-audio-track-processing --audio-buffer-size=16 --enable-exclusive-audio Chrome flags, but still there seems to be some audio processing.

Is there any way to disable the still present audio processing (preferably via JS API)?

Fetch

Does this happen locally or in a peer connection? It could that Chrome is changing the bitrate dynamically for Opus(this will only be the case if you are noticing this in a connection). – Pinworm

Over a peer connection. I think it is some kind of voice audio optimization. Always kicks in with after x amount of time a sample has been playing. – Fetch

I read it so I can say for sure that the bitrate is adjusted according to available bandwith. – Vociferant

Beware of this open issue bugs.chromium.org/p/webrtc/issues/detail?id=5758 – Gestapo

Make sure there are no system audio enhancements turned on. I am afraid we cannot control those from the browser level. Asked about it here: how-to-disable-system-audio-enhancements-using-webrtc – Bronder

I think this link involves every answer related to Audio_Processing whether it is Pre-Processing or Post-Processing: Android_Audio_Processing_Using_WebRTC, You can also visit this reference: https://mcmap.net/q/245096/-how-to-record-microphone-to-more-compressed-format-during-webrtc-call-on-android – Viridity

Answer 1

This is the updated way to disable audio processing and get a clean signal:

navigator.mediaDevices.getUserMedia({
  audio: {
    autoGainControl: false,
    channelCount: 2,
    echoCancellation: false,
    latency: 0,
    noiseSuppression: false,
    sampleRate: 48000,
    sampleSize: 16,
    volume: 1.0
  }
});

If you are streaming audio via WebRTC, it defaults to radio or phone quality audio optimized for voice. So make sure your SDP has stereo and maxaveragebitrate params:

a=fmtp:111 minptime=10;useinbandfec=1; stereo=1; maxaveragebitrate=510000

Assertive

echoCancellation, noiseSuppression, autoGainControl must be true not false – Balladry

Those options are designed to make the spoken voice clearer for meetings, but they can mute/hide music or other types of audio that you might want heard. Disabling those options gives the clean raw signal without any processing. – Assertive

Answer 2

I'd wager that the variable bitrate (default) behavior of the opus codec is causing some compression or adjustment. You could manually mangle the SDP offer to use CBR (constant bitrate) instead of VBR (variable bit rate). When you get the SDP offer from the browser, change the line:

a=fmtp:111 minptime=10; useinbandfec=1

to:

a=fmtp:111 minptime=10; cbr=1

Note that I'm both adding cbr=1 and removing useinbandfec=1. I'm not positive that dropping useinbandfec is necessary, but it seems that in-band FEC (forwarding error correction) causes compression adjustment which you'd want to avoid as well.

Moneybag

Answer 3

Thanks to https://mcmap.net/q/566784/-webrtc-disable-all-audio-processing answer. I have successfully mungled remote sdp offer by removing CN codecs and fixing fmtp string. That has removed comfort noise from audio and disabled audio processing

const removeComfortNoiseFromSdp = (sdp: sdpTransform.SessionDescription): sdpTransform.SessionDescription => {
  const audioMedias = sdp.media.filter(m => m.type === 'audio')
  const audioMedia = audioMedias[0]
  if (!audioMedia) {
    return sdp // no audio media
  }
  const whitelistedRtps = audioMedia.rtp.filter(rtp => {
    const codec = `${rtp.codec}`.toLowerCase()
    return codec !== 'cn'
  })
  const whitelistedPayloads = whitelistedRtps.map(rtp => rtp.payload)
  audioMedia.rtp = whitelistedRtps
  audioMedia.payloads = whitelistedPayloads.join(' ')
  audioMedia.fmtp = audioMedia.fmtp.map(fmtp => {
    return {
      ...fmtp,
      config: `${fmtp.config}`.replace('useinbandfec=1;', 'useinbandfec=1; stereo=1; maxaveragebitrate=510000'),
    }
  })
  return sdp
}

Motteo