Login/Register
Android MediaCodec not decoding all input buffers
Asked Answered
androidandroid-mediacodecaudiotrack
F

1

11

In Android 4.4.2, I am using MediaCodec to decode mp3 files. I'm using queueInputBuffer() to queue an input mp3 encoded frames and dequeueOutputBuffer() to get the decoded frames. But decoder gives decoded output from 8th frame onwards ( based on the bufferInfo.presentationTimeUs) and skips initial 7 frames. This scenario occurs only for few streams but not for all the streams. Also, this sort of behavior is consistent over many runs.

I want the decoded output of all the frames and I don't want any frames to skipped. Can anyone help me to understand why the frames are skipped? I assure that stream is not corrupted. Because I am getting INFO_TRY_AGAIN till 7th frame, when valid buffer index is returned by `dequeueOutputBuffer', its presentation time is always 8th frame's.

Following is the queuing code :

Log.e(TAG, "audptOffset = "+audptOffset+"input PT = "+audpt);
                audcodec.queueInputBuffer(audInbufIndex, 0, audchunkSize, audpt, 0);

Following is how i call dequeue and write to AudioTrack:

if(!audoutputDone ){
                if(!waitForAudioRelease){
                    auoutBufIndex  = audcodec.dequeueOutputBuffer(auinfo, 100);
                    Log.e(TAG, "Output PT = " + auinfo.presentationTimeUs+"auoutBufIndex = "+auoutBufIndex);
                }
                if (auoutBufIndex >= 0) {
                    waitForAudioRelease = true;
                } else if (auoutBufIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
                    auddecOutBufArray = audcodec.getOutputBuffers();
                } else if (auoutBufIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
                    MediaFormat newFormat = audcodec.getOutputFormat();
                    int sampleRate1 = newFormat.getInteger(MediaFormat.KEY_SAMPLE_RATE);
                    int channelCount1 =newFormat.getInteger(MediaFormat.KEY_CHANNEL_COUNT);
                    Log.e(TAG, "INFO_OUTPUT_FORMAT_CHANGED");
                    int minBufSize1 = AudioTrack.getMinBufferSize(sampleRate1, (channelCount1==2)?
                            AudioFormat.CHANNEL_OUT_STEREO:AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT);
                    audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC,
                            sampleRate1, (channelCount1==2)?AudioFormat.CHANNEL_OUT_STEREO:AudioFormat.CHANNEL_OUT_MONO,
                            AudioFormat.ENCODING_PCM_16BIT, minBufSize1,
                            AudioTrack.MODE_STREAM);
                    audioTrack.play();
                    waitForAudioRelease = false;
                }
                audionowUs = System.currentTimeMillis();
                if (auoutBufIndex >= 0) {               
                    auwhenRealUs = (auinfo.presentationTimeUs/1000) + mStartTimeRealMs - (audptOffset/1000);
                    aulateByUs = audionowUs - auwhenRealUs;


                    if(!audioWaitTillStartTime){
                        while((mStartTimeRealMs+((auinfo.presentationTimeUs/1000) - (audptOffset/1000))) >= audionowUs){
                            try {
                                Thread.sleep(10);
                            } catch (InterruptedException e) {
                                // TODO Auto-generated catch block
                                e.printStackTrace();
                            }
                            audionowUs = System.currentTimeMillis();
                        }
                        Log.e(TAG,"Play is going to start PT Difference = "+((auinfo.presentationTimeUs/1000) - (audptOffset/1000)));
                    }

Adding More Logs : 

02-22 17:46:03.164: E/CL(28650): received play command from server
02-22 17:46:03.209: E/RealTimeClient(28650): created decoder for audio/mpeg
02-22 17:46:03.234: E/Music(28650): Output PT = 0
02-22 17:46:03.234: E/Music(28650): pt of first frame received 1215000
02-22 17:46:03.234: E/Music(28650): Input PT = 1215000
02-22 17:46:03.234: E/Music(28650): Output PT = 0
02-22 17:46:03.234: E/Music(28650): Input PT = 1241122
02-22 17:46:03.239: E/Music(28650): Output PT = 0
02-22 17:46:03.239: E/Music(28650): Input PT = 1267244
02-22 17:46:03.239: E/Music(28650): Output PT = 0
02-22 17:46:03.239: E/Music(28650): Input PT = 1293367
02-22 17:46:03.239: E/Music(28650): Output PT = 0
02-22 17:46:03.239: E/Music(28650): Input PT = 1319489
02-22 17:46:03.239: E/Music(28650): Output PT = 0
02-22 17:46:03.244: E/Music(28650): INFO_OUTPUT_FORMAT_CHANGED
02-22 17:46:03.249: I/Reverb(28650):  getpid() 28650, IPCThreadState::self()->getCallingPid() 28650
02-22 17:46:03.249: E/Reverb(28650): Reverb::StartElementHandler, wrong element or attributes: boolean
02-22 17:46:03.249: E/Music(28650): Input PT = 1345612
02-22 17:46:03.254: E/Music(28650): Output PT = 1293367
02-22 17:46:03.259: E/Music(28650): Input PT = 1371734
02-22 17:46:03.259: E/Music(28650): Input PT = 1397857
02-22 17:46:03.259: E/Music(28650): Input PT = 1423979
02-22 17:46:03.259: E/Music(28650): Input PT = 1450102
02-22 17:46:03.264: E/Music(28650): Input PT = 1476224
02-22 17:46:03.269: E/Music(28650): Input PT = 1502346
02-22 17:46:03.269: E/Music(28650): Input PT = 1528469
02-22 17:46:03.269: E/Music(28650): Input PT = 1554591
02-22 17:46:03.269: E/Music(28650): Input PT = 1580714
02-22 17:46:03.269: E/Music(28650): Input PT = 1606836
02-22 17:46:03.269: E/Music(28650): Input PT = 1632959
02-22 17:46:03.269: E/Music(28650): Input PT = 1659081
02-22 17:46:04.124: W/AudioTrack(28650): releaseBuffer() track 0x5e2faf28 name=0x3 disabled, restarting
02-22 17:46:04.129: E/Music(28650): Output PT = 1319489
02-22 17:46:04.129: E/Music(28650): Input PT = 1685204
02-22 17:46:04.159: E/Music(28650): Output PT = 1345612
Fechner answered 25/1, 2016 at 10:21 Comment(3)
do you have the log output? sometimes the decoder returns messages in the log output. – Greige
@GabrielBursztyn Decoder didnt specifically throw any messge in this regard, anyhow I'll simulate and post logs asap – Fechner
@GabrielBursztyn, Added more logs, this may help – Fechner
S
2

MPEG-1 Layer III (MP3) has dependent frames, you can't just start from any frame like Layer I or Layer II. Quoting from the provided link, "In the worst case, 9 input frames may be needed before being able to decode one single frame." This is most likely what you are seeing. Although each of the first 7 frames has a PTS associated with it, it isn't until you reach the 8th frame that the decoder is actually able to fully decode a frame and begin playback. Playback begins with the 8th frames PTS. You would need to painfully parse through the bytes of the stream in question by hand to fully verify this is what is happening, but I suspect you are in fact playing all of the frames.

Saar answered 23/2, 2016 at 15:51 Comment(2)
Playback begins with the 8th frame's PTS - does it mean, PTS of first frame output is updated with PTS of 8th frame ? – Fechner
It the worst case, you can't play the first 8 frames because there isn't enough information to create a fully decoded frame. Those PTS will be ignored and the first PTS to be output will be associated with the first fully decoded frame (it could be any of the first 9 frames) that can be passed into an audio player. If you input a partially decoded frame with its PTS into the player, you will hear garbage. After you get the first fully decoded frame, you will see every PTS you input as an output. This appears to be exactly what is happening in the log. – Saar

© 2022 - 2024 — McMap. All rights reserved.