Capturing audio data and save to raw PCM file in AudioFlinger

Question

After some research, I find out this is possible to capture the audio data in libaudioflinger of Android.

I think the audio data is being written to HAL in here:

ssize_t framesWritten = mNormalSink->write((char *)mSinkBuffer + offset, count);

Full code: https://android.googlesource.com/platform/frameworks/av/+/lollipop-release/services/audioflinger/Threads.cpp#2118

So, I would like to save the mSinkBuffer + offset to a file (which I expected it will be raw PCM audio file). I using those stream to write it to file:

std::ofstream audioData ("/data/audiodata.raw", std::fstream::app);
audioData.write((char *)mSinkBuffer + offset, count);
audioData.close();

The file is successfully written and it has data in it. But, when I play the PCM file (audiodata.raw) with aplay or ffplay, the only sound I got is noise.

aplay -t raw -c 2 -f S16_LE -r 48000 audiodata.raw

I was worry about the config of aplay. So I print some log of the libaudioflinger:

10-07 10:14:54.575  1300  1366 I AudioFlinger: I/O handle: 13
10-07 10:14:54.575  1300  1366 I AudioFlinger: Standby: no
10-07 10:14:54.575  1300  1366 I AudioFlinger: Sample rate: 48000 Hz
10-07 10:14:54.575  1300  1366 I AudioFlinger: HAL frame count: 512
10-07 10:14:54.575  1300  1366 I AudioFlinger: HAL format: 0x1 (AUDIO_FORMAT_PCM_16_BIT)
10-07 10:14:54.575  1300  1366 I AudioFlinger: HAL buffer size: 2048 bytes
10-07 10:14:54.575  1300  1366 I AudioFlinger: Channel count: 2
10-07 10:14:54.575  1300  1366 I AudioFlinger: Channel mask: 0x00000003 (front-left, front-right)
10-07 10:14:54.575  1300  1366 I AudioFlinger: Processing format: 0x5 (AUDIO_FORMAT_PCM_FLOAT)
10-07 10:14:54.576  1300  1366 I AudioFlinger: Processing frame size: 8 bytes
10-07 10:14:54.576  1300  1366 I AudioFlinger: Pending config events:
10-07 10:14:54.576  1300  1366 I AudioFlinger:  none
10-07 10:14:54.576  1300  1366 I AudioFlinger: Output device: 0x2 (AUDIO_DEVICE_OUT_SPEAKER)
10-07 10:14:54.576  1300  1366 I AudioFlinger: Input device: 0 (AUDIO_DEVICE_NONE)
10-07 10:14:54.576  1300  1366 I AudioFlinger: Audio source: 0 (default)

I don't know what I did wrong. Please help me!

Thank you in advanced!

Rockefeller

Answer 1

Open The file in append |binary mode

std::ofstream audioData ("/data/audiodata.raw", std::fstream::app | std::fstream::binary);

binary - binary - Operations are performed in binary mode rather than text.

Raw PCM buffer should be written in binary mode.

Please check the code for the difference between the bytes and count ( added some comments for the reference , I hope this will solve your problem)

if (mNormalSink != 0) {

    /* Count is the number of Frames or sample written != bytes  */
    const size_t count = mBytesRemaining / mFrameSize;

    ATRACE_BEGIN("write");
    // update the setpoint when AudioFlinger::mScreenState changes
    uint32_t screenState = AudioFlinger::mScreenState;
    if (screenState != mScreenState) {
        mScreenState = screenState;
        MonoPipe *pipe = (MonoPipe *)mPipeSink.get();
        if (pipe != NULL) {
            pipe->setAvgFrames((mScreenState & 1) ?
                    (pipe->maxFrames() * 7) / 8 : mNormalFrameCount * 2);
        }
    }
    ssize_t framesWritten = mNormalSink->write((char *)mSinkBuffer + offset, count);

    ATRACE_END();
    if (framesWritten > 0) {
        bytesWritten = framesWritten * mFrameSize;
        // std::ofstream audioData ("/data/audiodata.raw", std::fstream::binary);
        /* File write or stream write is the number of bytes written to the file */
        audioData.write((char *)mSinkBuffer + offset, bytesWritten);
        // audioData.close();
    } else {
        bytesWritten = framesWritten;
    }
// otherwise use the HAL / AudioStreamOut directly
}

Audacity - Open a raw file

File -> Import -> raw data

Select the raw file path

Based on your raw file attached.

Use these Setting

Encoding : 32 -bit float

Byte order: Little Endian

Channels : 2 channels ( stereo)

Start offset : 0

Amount to import : 100

Sample Rate : 48000

File plays properly without any glitches/noise.

Ehman

Thanks for your comment! Unfortunately, it's still contains only noise when I tried to play it. Do you have any other idea? – Rockefeller

Can you open the raw file in audacity (import option) and check if the wave form or audio data is present. What is the count size you are getting for each execution? – Ehman

I can only Import > Raw Data. And if I use std::fstream::binary and then I can't hear anything but noise. If I use only std::fstream::app, I can hear some of the music with noise. – Rockefeller

and, the count is 1024 – Rockefeller

Can you try only binary mode , open the file 1 time at beginning and then close it at the end of your program. you don't need to use append mode . – Ehman

audioflinger is about to write the data to HAL again and again with different data, like a stream. So I think I can't open it just one time and close it. If I do that, it will contain very less data (maybe only a pulse, I think) – Rockefeller

No audio don't work that way, there will be a stream of buffer that will be submitted to the Audio driver continuously. ( 1024 is very few millisecond data ) So try and open the file once and then close it once done . – Ehman

you are right. It's a stream of buffer. Please take a look at this function: android.googlesource.com/platform/frameworks/av/+/…. This function is continuously called to submit that stream of buffer to the audio driver. So I really don't know when to open and when to close the file.. – Rockefeller

You can check these option : open the raw pcm file in AudioFlinger::PlaybackThread::createTrack_l, close the raw pcm file in AudioFlinger::PlaybackThread::destroyTrack_l, Write the audio buffer in AudioFlinger::PlaybackThread::threadLoop_write. – Ehman

Thanks for your dedicated support! I'm really really appreciate it!. Unfortunately, I modified as your suggestion but the output still contains only noise. Code: gist.github.com/namchuai/e782ec8006fc0bc79b774a1d2e34cab5 I think I should upload the raw PCM I recorded. Here's the link: github.com/namchuai/stock-scraping/blob/master/audiodata.raw – Rockefeller

Thx for sharing the code, this helps - make the changes as update above. – Ehman

I tried your code: audioData.write((char )mSinkBuffer + offset, (countmFrameSize)); But the output is quite same, I think. Only noise. – Rockefeller

I hope the mSinkbuffer is having the proper data pointer, please check again, made some more changes. – Ehman

Did you check with these changes once .. let me know. – Ehman

Sorry for took you so long. I tried with your but still contains noise. I uploaded it to github.com/namchuai/stock-scraping/blob/master/audiodata.raw. I tried with audacity and change some configuration like channel, sample rate but no luck. – Rockefeller

FYI, framesWritten = 1024 and mFrameSize = 8 – Rockefeller

do you have any other idea? If yes, please let me know :( – Rockefeller

I placed some logs and see that threadloop_write() is called by multiple thread.. I think it might be the cause. – Rockefeller

Can you share the log path and the modified file. – Ehman

I modified as your change on Oct 15 at 20:02. I can hear the sound with Audacity now. But it still contains so much noise and only right channel is working. Left channel is only noise. I played the raw file in audacity with Signed 16-bit PCM, No endianess, 2 channesl, 96000Hz. The file is located at: drive.google.com/file/d/1PIT_HNxjN9Y739Jmygm_UTiDbNFLXPfG/view – Rockefeller

Play with the setting I have added in my answer edited . – Ehman

I have no word to say! you are my hero now. I'm really appreciated your support! Thank you very much! – Rockefeller

Thanks for the kind words.. Just out of curiosity, want to know what are you developing and what is your final product. – Ehman

I'm capturing the android's audio output and streaming it to another android device :). just a small proof of concept project – Rockefeller

Answer 2

When you have to play your raw file you must use a codec format for the sound frames,such as wav or mp3 etc. Try to pass the raw data to a wav WAVEFORMAT defined in winmm and get audio from the devices that can be opened and closed via the WAVEFORMAT types in C++.

Broil

actually, the raw file is contains only audio data. That's why I have to specify the sampling rate, channel, etc to the aplay for it to play the file. Unlike WAV, which has a header define those needed information. – Rockefeller