AVAudioEngine inputNode's format changes when playing an AVAudioPlayerNode

Question

I'll start with a simple "playground" view controller class I've made that demonstrates my problem:

class AudioEnginePlaygroundViewController: UIViewController {
    private var audioEngine: AVAudioEngine!
    private var micTapped = false
    override func viewDidLoad() {
        super.viewDidLoad()
        configureAudioSession()
        audioEngine = AVAudioEngine()
    }

    @IBAction func toggleMicTap(_ sender: Any) {
        guard let mic = audioEngine.inputNode else {
            return
        }
        if micTapped {
            mic.removeTap(onBus: 0)
            micTapped = false
            return
        }
        stopAudioPlayback()

        let micFormat = mic.inputFormat(forBus: 0)
        print("installing tap: \(micFormat.sampleRate) -- \(micFormat.channelCount)")
        mic.installTap(onBus: 0, bufferSize: 2048, format: micFormat) { (buffer, when) in
            print("in tap completion")
            let sampleData = UnsafeBufferPointer(start: buffer.floatChannelData![0], count: Int(buffer.frameLength))
        }
        micTapped = true
        startEngine()
    }

    @IBAction func playAudioFile(_ sender: Any) {
        stopAudioPlayback()
        let playerNode = AVAudioPlayerNode()

        let audioUrl = Bundle.main.url(forResource: "test_audio", withExtension: "wav")!
        let audioFile = readableAudioFileFrom(url: audioUrl)
        audioEngine.attach(playerNode)
        audioEngine.connect(playerNode, to: audioEngine.outputNode, format: audioFile.processingFormat)
        startEngine()
        playerNode.scheduleFile(audioFile, at: nil, completionHandler: nil)
        playerNode.play()
    }

    // MARK: Internal Methods

    private func configureAudioSession() {
        do {
            try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayAndRecord, with: [.mixWithOthers, .defaultToSpeaker])
            try AVAudioSession.sharedInstance().setActive(true)
        } catch { }
    }

    private func readableAudioFileFrom(url: URL) -> AVAudioFile {
        var audioFile: AVAudioFile!
        do {
            try audioFile = AVAudioFile(forReading: url)
        } catch { }
        return audioFile
    }

    private func startEngine() {
        guard !audioEngine.isRunning else {
            return
        }

        do {
            try audioEngine.start()
        } catch { }
    }

    private func stopAudioPlayback() {
        audioEngine.stop()
        audioEngine.reset()
    }
}

The above VC has a single AVAudioEngine instance and two UIButton actions: one that plays an audio file found at a hard-coded url and another that toggles the installation/removal of a tap on the engine's inputNode.

My goal here is to get both live microphone tapping and audio file playback working at the same time but totally exclusive of each other. That is, I want to be able to trigger playback no matter the current state of my mic tap and vice versa. Everything works completely as expected if I install the tap before triggering the audio file playback. However, if I play the audio file first, then try to install the tap, I get the following crash:

[avae] AVAEInternal.h:70:_AVAE_Check: required condition is false: [AVAEGraphNode.mm:810:CreateRecordingTap: (IsFormatSampleRateAndChannelCountValid(format))]

which led me to check the microphone format's data via the log statement above the installTap call. Sure enough, when I install the tap before playback I get the expected sample rate of 44100.0 and channel count of 1. But when I play the audio file first and then install the mic tap, my log shows a sample rate of 0 and a channel count of 2 which gives me the error shown above.

I've tried tinkering with AVAudioEngine's start/reset flow, tried different category/mode combinations of my AVAudioSession (see my configureAudioSession method), and tried manually creating the tap format like so:

let micFormat = mic.inputFormat(forBus: 0)
var trueFormat: AVAudioFormat!
if micFormat.sampleRate == 0 {
    trueFormat = AVAudioFormat(standardFormatWithSampleRate: 44100, channels: 1)
} else {
    trueFormat = micFormat
}
print("installing tap: \(micFormat.sampleRate) -- \(micFormat.channelCount)")
mic.installTap(onBus: 0, bufferSize: 2048, format: trueFormat) { (buffer, when) in
    print("in tap completion")
    let sampleData = UnsafeBufferPointer(start: buffer.floatChannelData![0], count: Int(buffer.frameLength))
}

which gives me a similar, but different, error:

[avae] AVAEInternal.h:70:_AVAE_Check: required condition is false: [AVAudioIONodeImpl.mm:896:SetOutputFormat: (IsFormatSampleRateAndChannelCountValid(hwFormat))]

I can't see any reason why the microphone's format data would vary depending on whether or not an AVAudioPlayerNode has been played.

Stephine

Answer 1

After some searching I've found the problem. The issue lies in the audio engine's inputNode singleton. From the docs:

The audio engine creates a singleton on demand when inputNode is first accessed. To receive input, connect another audio node from the output of the input audio node, or create a recording tap on it.

Plus a reference to the format issue I was experiencing:

Check the input format of input node (specifically, the hardware format) for a non-zero sample rate and channel count to see if input is enabled.

In my playground class, the flow for triggering audio file playback never accesses the engine's inputNode before it creates an "active chain" with:

audioEngine.connect(playerNode, to: audioEngine.outputNode, format: audioFile.processingFormat)

It seems that you must access AVAudioEngine's inputNode before start()ing it if you want the engine to internally configure itself for input. Even stop()ing and reset()ing the engine does not cause an access of inputNode to reconfigure the engine. (I suspect the manually breaking the active chain via disconnectNode calls would allow for the internal reconfiguration but I don't yet know that for sure).

So code-wise the fix was simple: simply access the engine's input node immediately after instantiation so that the engine is configured for audio input. Here's the entire class with both file playback and mic tapping working together:

import UIKit

class AudioEnginePlaygroundViewController: UIViewController {
    private var audioEngine: AVAudioEngine!
    private var mic: AVAudioInputNode!
    private var micTapped = false

    override func viewDidLoad() {
        super.viewDidLoad()
        configureAudioSession()
        audioEngine = AVAudioEngine()
        mic = audioEngine.inputNode!
    }

    @IBAction func toggleMicTap(_ sender: Any) {
        if micTapped {
            mic.removeTap(onBus: 0)
            micTapped = false
            return
        }

        let micFormat = mic.inputFormat(forBus: 0)
        mic.installTap(onBus: 0, bufferSize: 2048, format: micFormat) { (buffer, when) in
            let sampleData = UnsafeBufferPointer(start: buffer.floatChannelData![0], count: Int(buffer.frameLength))
        }
        micTapped = true
        startEngine()
    }

    @IBAction func playAudioFile(_ sender: Any) {
        stopAudioPlayback()
        let playerNode = AVAudioPlayerNode()

        let audioUrl = Bundle.main.url(forResource: "test_audio", withExtension: "wav")!
        let audioFile = readableAudioFileFrom(url: audioUrl)
        audioEngine.attach(playerNode)
        audioEngine.connect(playerNode, to: audioEngine.outputNode, format: audioFile.processingFormat)
        startEngine()
        playerNode.scheduleFile(audioFile, at: nil, completionHandler: nil)
        playerNode.play()
    }

    // MARK: Internal Methods

    private func configureAudioSession() {
        do {
            try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayAndRecord, with: [.mixWithOthers, .defaultToSpeaker])
            try AVAudioSession.sharedInstance().setActive(true)
        } catch { }
    }

    private func readableAudioFileFrom(url: URL) -> AVAudioFile {
        var audioFile: AVAudioFile!
        do {
            try audioFile = AVAudioFile(forReading: url)
        } catch { }
        return audioFile
    }

    private func startEngine() {
        guard !audioEngine.isRunning else {
            return
        }

        do {
            try audioEngine.start()
        } catch { }
    }

    private func stopAudioPlayback() {
        audioEngine.stop()
        audioEngine.reset()
    }
}

Stephine

Thanks for posting this follow-up! – Aylmer

I need to change sample rate to 16khz (mic : 44100, I want: 16000). How to do that? Any hints. Already tried main mixture node but app crashes if I set different format – Nickinickie