How do you attach a reverb effect to AudioRecord/PCM data and save it to a file?

Question

I want to record input from the microphone, attach a reverb effect, and persist the result to a file. My use-case is an app that lets you sing a song and select different preset reverb options after recording, and then save your performance and store it on a backend server. The file that I send to the server needs to have a reverb effect applied to it.

So far I've been able to record input using AudioRecord, and I can add a reverb effect to AudioTrack to hear the reverb effect, but I'm stuck on figuring out how to save the audio with the reverb effect embedded. Here's what I have so far:

private void startRecording() {
    final int bufferSize = AudioRecord.getMinBufferSize(SAMPLE_RATE, CHANNEL_CONFIG, AUDIO_FORMAT);
    mRecorder = new AudioRecord(MediaRecorder.AudioSource.VOICE_COMMUNICATION, SAMPLE_RATE, CHANNEL_CONFIG, AUDIO_FORMAT, bufferSize);
    if(NoiseSuppressor.isAvailable()) {
        NoiseSuppressor ns = NoiseSuppressor.create(mRecorder.getAudioSessionId());
        ns.setEnabled(true);
    }

    if(AcousticEchoCanceler.isAvailable()) {
        AcousticEchoCanceler aec = AcousticEchoCanceler.create(mRecorder.getAudioSessionId());
        aec.setEnabled(true);
    }

    mPlayer = new AudioTrack(AudioManager.STREAM_MUSIC, SAMPLE_RATE, AudioFormat.CHANNEL_OUT_MONO, AUDIO_FORMAT, bufferSize, AudioTrack.MODE_STREAM);
    PresetReverb reverb = new PresetReverb(1, mPlayer.getAudioSessionId());
    reverb.setPreset(PresetReverb.PRESET_LARGEHALL);
    reverb.setEnabled(true);

    mPlayer.setAuxEffectSendLevel(1.f);

    mRecorder.startRecording();
    mPlayer.play();

    new RecordingThread(mRecorder, mPlayer, bufferSize) {

        @Override
        public void onReleased() {
            mRecorder = null;
            mPlayer = null;
        }
    }.start();
}

public class RecordingThread extends Thread {

    private static final String TAG = RecordingThread.class.getSimpleName();

    private final AudioRecord mRecorder;
    private final AudioTrack mPlayer;
    private final int mBufferSize;

    public RecordingThread(AudioRecord recorder, AudioTrack player, int bufferSize) {
        mRecorder = recorder;
        mPlayer = player;
        mBufferSize = bufferSize;
    }

    @Override
    public void run() {
        byte[] buffer = new byte[mBufferSize];

        int read;
        while ((read = mRecorder.read(buffer, 0, buffer.length)) > 0) {
            mPlayer.write(buffer, 0, buffer.length);
        }

        mRecorder.release();
        mPlayer.release();

        onReleased();

        Timber.tag(TAG);
        Timber.i("Exited with code %s", read);
    }

    public void onReleased() {
    }
}

So ideally, it'd be nice to be able to attach the PresetReverb on the AudioRecord instead of the AudioTrack, but when I do I get:

java.lang.RuntimeException: Cannot initialize effect engine for type: 47382d60-ddd8-11db-bf3a-0002a5d5c51b Error: -3

So now I'm thinking I need to pipe the PCM data from AudioRecord to some external service/library, or to modify the buffer with whatever a reverb algorithm would look like.

Meyers

Answer 1

As per https://developer.android.com/reference/android/media/audiofx/PresetReverb.html

The PresetReverb is an output mix auxiliary effect and should be created on Audio session 0. In order for a MediaPlayer or AudioTrack to be fed into this effect, they must be explicitely attached to it and a send level must be specified. Use the effect ID returned by getId() method to designate this particular effect when attaching it to the MediaPlayer or AudioTrack.

Since this is designed to be used with audiotrack, this cannot be attached to AudioRecord. I think you have a similar conclusion.

I would suggest you to use this SDK https://github.com/superpoweredSDK/Low-Latency-Android-Audio-iOS-Audio-Engine. It supports android and following effects:

Effects: echo, flanger, gate, reverb, rool, whoosh, 3 band equalizer, biquad IIR filters (low-pass, high-pass, bandpass, high-shelf, low-shelf, parametric, notch)

You can also consider https://github.com/ClearVolume/ClearAudio. You can use this file to apply reverb effect https://github.com/ClearVolume/ClearAudio/blob/master/src/java/clearaudio/synthesizer/filters/ReverbFilter.java

All the best !

---

Adding some more details on how this can be done using superpoweredSDK. The superpowered SDK has a filter called SuperpoweredReverb. The way this works is the following :

Let's see this through some code:

reverb = new SuperpoweredReverb(samplerate);
reverb->enable(true);

reverb->setMix(floatValue); //  Limited to >= 0.0f and <= 1.0f.
reverb->setRoomSize(floatValue); // Limited to >= 0.0f and <= 1.0f.

SuperpoweredShortIntToFloat(intBuffer, floatBuffer, numberOfSamples);

bool res = reverb->process(floatBuffer, floatBuffer, numberOfSamples);

SuperpoweredFloatToShortInt(floatBuffer, intBuffer, numberOfSamples);

Let me try my best to explain what's happening here:

Create filter

reverb = new SuperpoweredReverb(samplerate);
reverb->enable(true);

Optional: Apply additional options to this filter such as

reverb->setMix(floatValue); //  Limited to >= 0.0f and <= 1.0f.
reverb->setRoomSize(floatValue); // Limited to >= 0.0f and <= 1.0f.

Get the source samples ; If the sample are in Integer use SuperpoweredShortIntToFloat to convert. Here intBuffer has the input audio samples and numberOfSamples is the number of samples. floatBuffer will contain float converted buffer

SuperpoweredShortIntToFloat(intBuffer, floatBuffer, numberOfSamples);

Call process to apply reverb effect. This method will apply reverb effect and copy the output to floatBuffer again

bool res = reverb->process(floatBuffer, floatBuffer, numberOfSamples);

To convert it back to Int use the following method

SuperpoweredFloatToShortInt(floatBuffer, intBuffer, numberOfSamples);

Then you can store this reverb applied audio sample anywhere you want, either in file or play it

---

Some more findings from superpoweredSDK, it also has recording functionality. See http://superpowered.com/docs/class_superpowered_recorder.html

An example:

recorder = new SuperpoweredRecorder(temporaryPath, samplerate);
recorder->start(destinationPath);

// With input samples
SuperpoweredShortIntToFloat(intBuffer, floatBuffer, numberOfSamples);

bool res = recorder->process(floatBuffer, NULL, numberOfSamples);

SuperpoweredFloatToShortInt(floatBuffer, intBuffer, numberOfSamples);

Quintessence

Thanks for the response. I've already tried Superpowered, but there is an extreme lack of tutorials and usage guides. So I got complete lost trying to implement anything. I have very limited JNI experience. Can you point me to a Superpowered implementation that is using reverb? – Meyers

JNI code is github.com/CayaQi/SuperpoweredDemo/blob/master/app/src/main/jni/… and android code is also available in the repo. – Quintessence

That example would be useful for just hearing the reverb effect, but it doesn't show how to save a file with an effect, just how to apply the effect while playing the audio file – Meyers

Edited answer, check if it helps you – Quintessence

Yes, that does help. So floatBuffer is the PCM buffer? I could use AudioRecorder to record and pass that into the reverb method through JNI, then return the processed buffer back to my Java code to save it? Or would it make more sense to do it all in the C++ code. Also, what's a reasonable value for numberOfSamples? I see it's minimum 16, and it look like a current bug limits it to 1024. Should I use 1024, or would there be a reason to use something different? – Meyers

If you see superpowered.com/docs/class_superpowered_reverb.html, the input is "32-bit interleaved stereo input buffer" and yes it is PCM. Since write operation can take time so you need to put in another thread. The number of samples as per documentation should be "Should be 16 minimum, and a multiply of 8." Also if you look at the example I shared earlier (github.com/CayaQi/SuperpoweredDemo/blob/master/app/src/main/jni/…). It works on android so you can just use it and modify it for recording. – Quintessence

Also that example also has recording looks like. See SuperpoweredRecorder *recorder = new SuperpoweredRecorder(pathTemp,44100,1); More details at superpowered.com/docs/class_superpowered_recorder.html – Quintessence

Hey Manish -- that was a great and thorough answer. I'm CEO of Superpowered. Had a question for you. What's best way to email you? – Sniff

When I apply reverb effect to audio file, the voice is getting deeper (its pitch is changing I guess) and its length is getting shorter. I tried to solve problem with fixing sample rate, buffer size, number of samples but no success. Can you help me ? I am about to cancel the whole project due to this issue and working on this issue for 4 days. – Ironic

Have you created a SO issue somewhere ? – Quintessence