Android Play PCM byte array from Converted from Base64 String Slow Sounds
Asked Answered
T

1

2

As the very long title suggests, I'm having trouble playing the audio from a audio that I send over the network through PubNunb. What I do is I send the audio while recording from AudioRecord using this code:

 AudioConfig audioConfig = getValidSampleRates(AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT);
    buffer = new byte[audioConfig.getBufferSize()];

    recorder = new AudioRecord(MediaRecorder.AudioSource.MIC, audioConfig.getSampleSize(), AudioFormat.CHANNEL_IN_MONO, AUDIO_FORMAT, audioConfig.getBufferSize());

Recorded data is sent when the user holds the button down:

   private class RecorderRunnable implements Runnable {
    @Override
    public void run() {
        while(mRecording) {
            Log.d("RECORDER_STATE", "Recording LOOP");
            recorder.read(buffer, 0, buffer.length);

            String base64EncodedBuffer = Base64.encodeToString(buffer, Base64.NO_WRAP);

            pubnub.publish(MainActivity.CHANNEL_ID, base64EncodedBuffer, new Callback() {
                @Override
                public void successCallback(String channel, Object message) {
                    super.successCallback(channel, message);
                }
            });
        }
    }
}

Receive code:

       @Override
                    public void successCallback(String channel, final Object message) {


                        byte[] decodedBase64 = Base64.decode(message.toString(), Base64.NO_WRAP);

                        speaker.write(decodedBase64, 0, decodedBase64.length);
                    }

Issue: I get the audio, but I get sounds that are really slow. "Hello" would sound like: "Hee-*static*-ll-*static*-oo"

To rule out possible causes, I tried immediately playing the audio like this (without the network):

 while(mRecording) {
            Log.d("RECORDER_STATE", "Recording LOOP");
            recorder.read(buffer, 0, buffer.length);

            String base64EncodedBuffer = Base64.encodeToString(buffer, Base64.NO_WRAP);

            byte[] decodedBase64 = Base64.decode(base64EncodedBuffer, Base64.NO_WRAP);

            speaker.write(decodedBase64, 0, decodedBase64.length);
        }

(Note: I did the convert to base64 and back to byte array on purpose.)

The result for the code above (directly playing it after recording) is pretty good. So I'm wondering what I'm doing wrong when handling it over the network.

Any suggestion is appreciated. Thank you.

Edit: 08/28/2015 Found a good explanation for this here. But now the question is, what's the best way of handling network jitter/buffering and packetloss using my current implementation.

Thaothapa answered 28/8, 2015 at 10:49 Comment(0)
P
4

pubnub delivers data through tcp (not udp). http://www.pubnub.com/knowledge-base/discussion/263/does-pubnub-support-the-udp-protocol

There should be no need to handle packet loss in your application.

You may need to handle jitter by creating a buffer of some sort. Since there would be no rigid realtime constraint, I will discuss an approach rather than pasting code.

You can make a buffer using a queue. I suggest having two threads. One for your reader (your player) and one for writer (the network stream). Let the writer queue up some data (maybe a few seconds of data) before letting the reader read. On paper, with a very simply proof of concept, you should not have issues with simultaneous reads and writes since the writer is writing to the end of the queue and the reader is reading at the beginning of the queue.

Think of it as a bucket that is halfway full. You pour water in and let water leak out at the same rate.

Paranoia answered 28/8, 2015 at 18:12 Comment(4)
Thanks for the detailed response. I read some articles about queuing buffers which state that this needs to be done in native (ndk). Does this have to be the case? If it's in Java, would an ArrayList of byte array be suffice?Thaothapa
I see no reason to use the NDK. The NDK would give you the ability to write highly optimized C code and trim and possibly use some of the low latency features that Android occasionally offers. Just start out with something simple and improve and optimize it later.Paranoia
Was able to fix the slow sound. Now dealing with the issue where it's not playing seemlessly. Thanks again!Thaothapa
Hello again. It's been over a week now, and I'm still having issues with the slow, jittery audio. How would you determine the correct amount of seconds for the writer to "queue up" -- by the way, I'm not really sure how to implement this other than creating a ConcurrentLinkedQueue which I did for reading and not writing. Can you elaborate a bit more, please? (constantly pulling my hair out)Thaothapa

© 2022 - 2024 — McMap. All rights reserved.