Android > 4.0 : Ideas how to record/capture internal audio (e.g. STREAM_MUSIC)?

Question

Some months ago, with Android ICS (4.0), I developed an android kernel module which intercepted the "pcmC0D0p"-module to fetch all system audio.

My target is to stream ALL audio (or at least the played music) to a remote speaker via AirPlay.

The kernel module worked, but there where several problems (kernel-versions, root-privileges etc.) so I stopped working on this.

Now, we have Android 4.1 and 4.2 and I have new hope!

Who has an idea how to capture the audio in Android?

I had following ideas:

1. Connect via bluetooth to the same phone, set routing to BT and grab the audio on the "other end": this shouldn't work

2. Intercept the audio with a kernel module like done before: hardcore, get it worked but not applicable

3. JACK Audio Connection Kit: sadly Android uses "tinyALSA" and not "ALSA". TinyALSA does NOT support any filters like JACK (but this brought the idea with the kernel module)

4. Use PulseAudio as a replacement for AudioFlinger, but this is also not applicable

---

EDIT (forgot them):

5. I compiled "tinymix" (baby-version of ALSA-mixer) from tinyALSA (the ALSA on Android) and tried to route the audio-out to mic-in - but with no success (not understandable for me). And this also needs rooting: not applicable

6. I tested OpenSL ES, but I'm not a C-crack and it ended in "I can record microphone, but no more" (maybe I was wrong?)

---

I just found ROUTE_TYPE_LIVE_AUDIO:

A device that supports live audio routing will allow the media audio stream to be routed to supported destinations. This can include internal speakers or audio jacks on the device itself, A2DP devices, and more.

Once initiated this routing is transparent to the application. All audio played on the media stream will be routed to the selected destination.

Maybe this helps in any way?

I'm running out of ideas but want to "crack this nut", maybe someone can help me?

EDIT:

I'm really new in C & kernel-coding (but I successfully created a cross-compiled audio-interception-module) - but isn't it in any way possible to listen at the point the PCM-data goes from userspace (JAVA, C-layer?) to the kernel-space (tinyALSA, kernel-module), without hacking & rooting?

Schmeltzer

There's no support in Android for doing this, so it would only work on a custom ROM where you've added this functionality (e.g. by modifying some kernel module as you suggested). Newer Qualcomm platforms support WiFi Display though, which is a different technology from AirPlay that serves as a kind of wireless HDMI connection to compatible devices (e.g. some newer TVs). IIRC, the MediaRouter is a widget that you can add to your app and allows the user to select where audio should be routed. It only allows the user to select devices supported/detected by the phone/tablet though. – Ceramal

Thank you for your response. That's really bad :*( But I'll keep on searching for a solution. I will extend my "test"-documentation above, forgot something. – Schmeltzer

"isn't it in any way possible to listen at the point the PCM-data goes from userspace (JAVA, C-layer?) to the kernel-space (tinyALSA, kernel-module), without hacking & rooting?" Unfortunately for you there isn't. Another thing you might want to keep in mind is all playback doesn't necessarily go to the same ALSA playback device (pcmCxDyp). Normal playback might go to one device, low power playback to another, and low latency playback to yet another. amixer/tinymix won't do any good unless the platform provides some sort of readback of played data, which typically isn't the case. – Ceramal

Hi Michael, yes your're right. But tinyALSA has one good: it ALWAYS uses pcmC0D0p for playback, and pcmC0D0c for capturing. I've analysed the source of it. At the moment, I'm re-installing the VirtualBox for android kernel-development and give my kernel-module another try (after some months cooling down) - I won't give up :) – Schmeltzer

"it ALWAYS uses pcmC0D0p for playback, and pcmC0D0c for capturing." Not necessarily. I've worked with phones that uses libtinyalsa for USB audio playback, and in those cases pcmC1D0p was used. – Ceramal

Oh okay, good to know. I think in my cases, that's no problem. – Schmeltzer

And now for the completely ridiculous .... send the sound out BT and pick it up with the radio modems or using the headphones as an aerial? Pump the volume to the speaker right up and sense it in the power voltage ??? There is a fair bit of feedback from the headphones to the external microphone so that might be a shot with some signal cleanup! – Eton

Hi Martin, I'm working on a similar problem, my phone is rooted so using root is acceptable for me. Do you have any code to share in order to be able to get the internal audio at kernel or user level (C/C++)? Thank you. – Blew

Hello Zen, sadly not! It's not simply to achieve - I think we have to wait until Android is implementing it! – Schmeltzer

Hello Martin, any news on this? Did newer Android versions help getting us there? I really want to add this to my app as well! – Larentia

Hi Daniel, sadly not. I have observed this topic all the time and tried all new ways when they popped up. It's still not possible to grab the audio-output on an unrooted device. Even with root, it's more than incredible complex. – Schmeltzer

Answer 1

You must pass audio traffic through your local socket server:

1. Create local socket server

2. Modify audio http stream address to path is thru your local socket server, for example for address

   http://example.com/stream.mp3  ->
   
   http://127.0.0.1:8888/http://example.com/stream.mp3
   

   where 8888 is your socket port.

3. Play

   http://127.0.0.1:8888/http://example.com/stream.mp3 
   

   with default media player

4. Read all incoming requests to port 8888 in your local socket server, analyze it and pass to example.com server.

5. Read response from example.com and pass it to local client (media player). HERE you can capture audio stream!

Ecumenicism

If I understand this correctly, you are pretty much recommending a capturing proxy. While it may sort of work for the initial load of remote audio, this does not solve the OP's problem in any capacity. Just putting this comment here because the bounty confused me into thinking this was an accepted answer, hopefully mentioning it saves other people some time. – Jube