Android: AudioRecord Class Problem: Callback is never called

Question

My Android Java Application needs to record audio data into the RAM and process it. This is why I use the class "AudioRecord" and not the "MediaRecorder" (records only to file).

Till now, I used a busy loop polling with "read()" for the audio data. this has been working so far, but it peggs the CPU too much. Between two polls, I put the thread to sleep to avoid 100% CPU usage. However, this is not really a clean solution, since the time of the sleep is not guaranteed and you must subtract a security time in order not to loose audio snippets. This is not CPU optimal. I need as many free CPU cycles as possible for a parallel running thread.

Now I implemented the recording using the "OnRecordPositionUpdateListener". This looks very promising and the right way to do it according the SDK Docs. Everything seems to work (opening the audio device, read()ing the data etc.) but the Listner is never called.

Does anybody know why?

Info: I am working with a real Device, not under the Emulator. The Recording using a Busy Loop basically works (however not satifiying). Only the Callback Listener is never called.

Here is a snippet from my Sourcecode:

public class myApplication extends Activity {

  /* audio recording */
  private static final int AUDIO_SAMPLE_FREQ = 16000;
  private static final int AUDIO_BUFFER_BYTESIZE = AUDIO_SAMPLE_FREQ * 2 * 3; // = 3000ms
  private static final int AUDIO_BUFFER_SAMPLEREAD_SIZE = AUDIO_SAMPLE_FREQ  / 10 * 2; // = 200ms

  private short[] mAudioBuffer = null; // audio buffer
  private int mSamplesRead; // how many samples are recently read
  private AudioRecord mAudioRecorder; // Audio Recorder

  ...

  private OnRecordPositionUpdateListener mRecordListener = new OnRecordPositionUpdateListener() {

    public void onPeriodicNotification(AudioRecord recorder) {
      mSamplesRead = recorder.read(mAudioBuffer, 0, AUDIO_BUFFER_SAMPLEREAD_SIZE);
      if (mSamplesRead > 0) {

        // do something here...

      }
    }

    public void onMarkerReached(AudioRecord recorder) {
      Error("What? Hu!? Where am I?");
    }
  };

  ...

  public void onCreate(Bundle savedInstanceState) {

  try {
      mAudioRecorder = new AudioRecord(
          android.media.MediaRecorder.AudioSource.MIC, 
          AUDIO_SAMPLE_FREQ,
          AudioFormat.CHANNEL_CONFIGURATION_MONO,
          AudioFormat.ENCODING_PCM_16BIT,  
          AUDIO_BUFFER_BYTESIZE);
    } catch (Exception e) {
      Error("Unable to init audio recording!");
    }

    mAudioBuffer = new short[AUDIO_BUFFER_SAMPLEREAD_SIZE];
    mAudioRecorder.setPositionNotificationPeriod(AUDIO_BUFFER_SAMPLEREAD_SIZE);
    mAudioRecorder.setRecordPositionUpdateListener(mRecordListener);
    mAudioRecorder.startRecording();

    /* test if I can read anything at all... (and yes, this here works!) */
    mSamplesRead = mAudioRecorder.read(mAudioBuffer, 0, AUDIO_BUFFER_SAMPLEREAD_SIZE);

  }
}

Ofilia

Answer 1

I believe the problem is that you still need to do the read loop. If you setup callbacks, they will fire when you've read the number of frames that you specify for the callbacks. But you still need to do the reads. I've tried this and the callbacks get called just fine. Setting up a marker causes a callback when that number of frames has been read since the start of recording. In other words, you could set the marker far into the future, after many of your reads, and it will fire then. You can set the period to some bigger number of frames and that callback will fire every time that number of frames has been read. I think they do this so you can do low-level processing of the raw data in a tight loop, then every so often your callback can do summary-level processing. You could use the marker to make it easier to decide when to stop recording (instead of counting in the read loop).

Row

your answer is very helpful. I would highly appreciate if you could enlighten me on how to properly read the live input stream from a microphone. A constant while loop seems inapropriate as it would block the entire process, is that right? I thought the callbacks were there as a solution, but judging by your answer, they are not. Thank you. – Sandblast

The constant while loop to read audio data must be in a separate thread. You certainly can't use the main thread to read audio. The callbacks would fire when an appropriate amount of audio data has been collected. You could put processing logic into the read loop, or you could use the callbacks to trigger processing on the buffer that you just read in. The reads block but only within that thread, so you should be okay. – Row

It looks that setRecordPositionUpdateListener isn't needed at all. It adds much more confusion rather than benefits. – Thrice

Answer 2

Here is my code used to find average noise. Notice that it is based on listener notifications so it will save device battery. It is definitely based on examples above. Those example saved much time for me, thanks.

private AudioRecord recorder;
private boolean recorderStarted;
private Thread recordingThread;
private int bufferSize = 800;
private short[][] buffers = new short[256][bufferSize];
private int[] averages = new int[256];
private int lastBuffer = 0;

protected void startListenToMicrophone() {
    if (!recorderStarted) {

        recordingThread = new Thread() {
            @Override
            public void run() {
                int minBufferSize = AudioRecord.getMinBufferSize(8000, AudioFormat.CHANNEL_CONFIGURATION_MONO,
                        AudioFormat.ENCODING_PCM_16BIT);
                recorder = new AudioRecord(AudioSource.MIC, 8000, AudioFormat.CHANNEL_CONFIGURATION_MONO,
                        AudioFormat.ENCODING_PCM_16BIT, minBufferSize * 10);
                recorder.setPositionNotificationPeriod(bufferSize);
                recorder.setRecordPositionUpdateListener(new OnRecordPositionUpdateListener() {
                    @Override
                    public void onPeriodicNotification(AudioRecord recorder) {
                        short[] buffer = buffers[++lastBuffer % buffers.length];
                        recorder.read(buffer, 0, bufferSize);
                        long sum = 0;
                        for (int i = 0; i < bufferSize; ++i) {
                            sum += Math.abs(buffer[i]);
                        }
                        averages[lastBuffer % buffers.length] = (int) (sum / bufferSize);
                        lastBuffer = lastBuffer % buffers.length;
                    }

                    @Override
                    public void onMarkerReached(AudioRecord recorder) {
                    }
                });
                recorder.startRecording();
                short[] buffer = buffers[lastBuffer % buffers.length];
                recorder.read(buffer, 0, bufferSize);
                while (true) {
                    if (isInterrupted()) {
                        recorder.stop();
                        recorder.release();
                        break;
                    }
                }
            }
        };
        recordingThread.start();

        recorderStarted = true;
    }
}

private void stopListenToMicrophone() {
    if (recorderStarted) {
        if (recordingThread != null && recordingThread.isAlive() && !recordingThread.isInterrupted()) {
            recordingThread.interrupt();
        }
        recorderStarted = false;
    }
}

Layard

This example seems to spin CPU during the recording. The while(true) loop will be executed repeatedly... – Pulchi

Hi @Layard , why have you set the periodInFrames of the setPositionNotificationPeriod as same as the buffers size. – Colum

Answer 3

Now I implemented the recording using the "OnRecordPositionUpdateListener". This looks very promising and the right way to do it according the SDK Docs. Everything seems to work (opening the audio device, read()ing the data etc.) but the Listner is never called.

Does anybody know why?

I found that the OnRecordPositionUpdateListener is ignored until you do your first .read().

In other words, I found that if I set up everything per the docs, my the Listener never got called. However, if I first called a .read() just after doing my initial .start() then the Listener would get called -- provided I did a .read() every time the Listener was called.

In other words, it almost seems like the Listener event is only good for once per .read() or some such.

I also found that if I requested any less than buffSize/2 samples to be read, that the Listener would not be called. So it seems that the listener is only called AFTER a .read() of at least half of the buffer size. To keep using the Listener callback, one must call read each time the listener is run. (In other words, put the call to read in the Listener code.)

However, the listener seems to be called at a time when the data isn't ready yet, causing blocking.

Also, if your notification period or time are greater than half of your bufferSize they will never get called, it seems.

UPDATE:

As I continue to dig deeper, I have found that the callback seems to ONLY be called when a .read() finishes......!

I don't know if that's a bug or a feature. My initial thought would be that I want a callback when it's time to read. But maybe the android developers had the idea the other way around, where you'd just put a while(1){xxxx.read(...)} in a thread by itself, and to save you from having to keep track of each time read() finished, the callback can essentially tell you when a read has finished.

Oh. Maybe it just dawned on me. And I think others have pointed this out before me here but it didn't sink in: The position or period callback must operate on bytes that have been read already...

I guess maybe I'm stuck with using threads.

In this style, one would have a thread continuously calling read() as soon as it returned, since read is blocking and patiently waits for enough data to return.

Then, independently, the callback would call your specified function every x number of samples.

Pinero

Thank you for your research. It helps! A bit more on this problem: onPeriodicNotification callback is never called on API 16, 17, 18, unless one makes a read() call just after the start. However, it seems like the problem is fixed on API >=19. – Matusow

DmitryO, Thank you very much for the additional information! I can't wait to try it out. – Pinero

Answer 4

For the ones who are recording Audio over an Intent Service, they might also experience this callback problem. I have been working on this issue lately and I came up with a simple solution where you call your recording method in a seperate thread. This should call onPeriodicNotification method while recording without any problems.

Something like this:

public class TalkService extends IntentService {
...
@Override
protected void onHandleIntent(Intent intent) {
Context context = getApplicationContext();
tRecord = new Thread(new recordAudio());
tRecord.start();
...
while (tRecord.isAlive()) {
    if (getIsDone()) {
        if (aRecorder.getRecordingState() == AudioRecord.RECORDSTATE_STOPPED) {
            socketConnection(context, host, port);
        }
    }
} }
...
class recordAudio implements Runnable {

        public void run() {
            try {
                OutputStream osFile = new FileOutputStream(file);
                BufferedOutputStream bosFile = new BufferedOutputStream(osFile);
                DataOutputStream dosFile = new DataOutputStream(bosFile);

                aRecorder = new AudioRecord(MediaRecorder.AudioSource.MIC,
                        sampleRate, channelInMode, encodingMode, bufferSize);

                data = new short[bufferSize];

                aRecorder.setPositionNotificationPeriod(sampleRate);
                aRecorder
                        .setRecordPositionUpdateListener(new AudioRecord.OnRecordPositionUpdateListener() {
                            int count = 1;

                            @Override
                            public void onPeriodicNotification(
                                    AudioRecord recorder) {
                                Log.e(WiFiDirect.TAG, "Period notf: " + count++);

                                if (getRecording() == false) {
                                    aRecorder.stop();
                                    aRecorder.release();
                                    setIsDone(true);
                                    Log.d(WiFiDirect.TAG,
                                            "Recorder stopped and released prematurely");
                                }
                            }

                            @Override
                            public void onMarkerReached(AudioRecord recorder) {
                                // TODO Auto-generated method stub

                            }
                        });

                aRecorder.startRecording();
                Log.d(WiFiDirect.TAG, "start Recording");
                aRecorder.read(data, 0, bufferSize);
                for (int i = 0; i < data.length; i++) {
                    dosFile.writeShort(data[i]);
                }

                if (aRecorder.getRecordingState() == AudioRecord.RECORDSTATE_RECORDING) {
                    aRecorder.stop();
                    aRecorder.release();
                    setIsDone(true);
                    Log.d(WiFiDirect.TAG, "Recorder stopped and released");
                }

            } catch (Exception e) {
                // TODO: handle exception
            }
        }
    }

Uncrown