Modifying in-call voice playback in Android custom ROM

Question

I would like to modify Android OS (official image from AOSP) to add preprocessing to a normal phone call playback sound.

I've already achieved this filtering for app audio playback (by modifying HAL and audioflinger).

I'm OK with targeting only a specific device (Nexus 5X). Also, I only need to filter playback - I don't care about recording (uplink).

UPDATE #1:

To make it clear - I'm OK with modifying Qualcomm-specific drivers, or whatever part that it is that runs on Nexus 5X and can help me modify in-call playback.

UPDATE #2:

I'm attempting to create a Java layer app that routes the phone playback to the music stream in real time.

I've already succeeded in installing it as a system app, getting permissions for initializing AudioRecord with AudioSource.VOICE_DOWNLINK. However, the recording gives blank samples; it doesn't record the voice call.

This is the code inside my worker thread:

// Start recording
int recBufferSize = AudioRecord.getMinBufferSize(44100, AudioFormat.CHANNEL_IN_STEREO, AudioFormat.ENCODING_PCM_16BIT);
mRecord = new AudioRecord(MediaRecorder.AudioSource.VOICE_DOWNLINK, 44100, AudioFormat.CHANNEL_IN_STEREO, AudioFormat.ENCODING_PCM_16BIT, recBufferSize);

// Start playback
int playBufferSize = AudioTrack.getMinBufferSize(44100, AudioFormat.CHANNEL_OUT_STEREO, AudioFormat.ENCODING_PCM_16BIT);
mTrack = new AudioTrack(AudioManager.STREAM_MUSIC, 44100, AudioFormat.CHANNEL_OUT_STEREO, AudioFormat.ENCODING_PCM_16BIT, playBufferSize, AudioTrack.MODE_STREAM);

mRecord.startRecording();;
mTrack.play();

int bufSize = 1024;
short[] buffer = new short[bufSize];
int res;
while (!interrupted())
{
    // Pull recording buffers and play back
    res = mRecord.read(buffer, 0, bufSize, AudioRecord.READ_NON_BLOCKING);
    mTrack.write(buffer, 0, res, AudioTrack.WRITE_BLOCKING);
}

// Stop recording
mRecord.stop();
mRecord.release();
mRecord = null;

// Stop playback
mTrack.stop();
mTrack.release();;
mTrack = null;

I'm running on a Nexus 5X, my own AOSP custom ROM, Android 7.1.1. I need to find the place which will allow call recording to work - probably somewhere in hardware/qcom/audio/hal in platform code.

Also I've been looking at the function voice_check_and_set_incall_rec_usecase at hardware/qcom/audio/hal/voice.c However, I wasn't able to make sense of it (how to make it work the way I want it to).

UPDATE #3:

I've opened a more-specific question about using AudioSource.VOICE_DOWNLINK, which might draw the right attention and will eventually help me solve this question's problem as well.

Anaclitic

I wanted to do that. Just to have it monitor the call for keywords then inject sound into the call. Like I say "golf" and then there is a golf clap sound injected into the call. – Mccluskey

i doubt if modifying standard i/o is possible without playing with ndk-jni – Longspur

I'm talking about creating a custom ROM, not a normal app. – Anaclitic

IIRC, a phone call is considered sensitive information and open software has no access to it. You would have to hack the closed-source firmware known as the "radio." Probably unfeasible. – Arsenate

That's not true. There is a call recording API, and tons of apps who found their way to use it using various hacks on a lot of devices. – Anaclitic

@Anaclitic can you provide some examples of this kind of apps ? – Isosceles

play.google.com/store/apps/… – Anaclitic

I authored an app like that. Released it 2 years ago. It is possible to record the stream so I assume easy to filter it. However some devices just does not have a hardware connection between radio and encoding chip. This is to ensure nobody hacks your phone and records your calls. The chinese cheap as chips models always work for some reason. – Garganey

Thanks @Namphibian, are you able to share your solution? I am willing to pay for something that works. – Anaclitic

@Anaclitic we can discuss some details in chat. Let me know when you are free. I am more than willing to help and I am probably going to open source the code. I see you are in HCMC( I love Vietnam) and I am in Sydney so we have a 3 hour difference. Let me knwo. – Garganey

@Garganey I've attempted to add you on LinkedIn. I don't know a chat feature here in StackOverflow... – Anaclitic

@Garganey I have created a room in StackOverflow chat. Hope you get notified of this. Otherwise let me know how to contact you... – Anaclitic

@Garganey Currently i am working on the similar functionality on AOSP build and i had successfully capture voice call recording. But stuck into the playing audio file on uplink phone call stream. Can you please explain how you guys modify HAL & AudioFlinger to stream music audio during phone calls. I really appreciate your help. Thanks in advance! – Happily

Sorry this project was abandoned 4 years ago. The code is gone and I dont work on mobile anymore. – Garganey

Answer 1

There are several possible issues that come to my mind. The blank buffer might indicate that you have the wrong source selected. Also since according to https://developer.android.com/reference/android/media/AudioRecord.html#AudioRecord(int,%20int,%20int,%20int,%20int) you might not always get an exception even if something's wrong with the configuration, you might want to confirm whether your object has been initialized properly. If all else fails, you could also do an "mRecord.setPreferredDevice(AudioDeviceInfo.TYPE_BUILTIN_EARPIECE);" to route the phone's built-in earpiece directly to the input of your recorder. Yeah, it's kinda dirty and hacky, but perhaps suits the purpose.

The other thing what was puzzling me that instead of using the builder class you've tried to configure the object directly via its constructor. Is there a specific reason why you don't want to use AudioRecord.Builder (there's even a nice example at https://developer.android.com/reference/android/media/AudioRecord.Builder.html ) instead?

Paschasia

All the points you've raised have been checked, this is not related. The Builder class is indeed more up to date, but internally it goes to exactly the same place (check Android source code). – Anaclitic

So plugging in to the function the built-in earpiece's source didn't work as a workaround either? – Paschasia

No. It's not related to the underlying issue. – Anaclitic