how to reduce MediaCodec H264 encoder latency

Question

I'm trying to encode h264 into stream in real-time low latency with Android6.0's MediaCodec. There are about 6 frames latency from encoder which I wanna know how to reduce

Codes are from screenrecord.cpp

part codes are:

while (true) {
     encoder->dequeueOutputBuffer(&bufIndex);
}

the encoder is feed by Surface of screen in 320x480 60 fps, and it outputs stream data by dequeueOutputBuffer(). There are about 6 frames feed to encoder whose data are not return immedately by dequeueOutputBuffer(). In other words, when the N-th frame are feeded, the encoder output data of the (N-6)th frame.

Question 1 - How to reduce frames buffered in encoder? Is it possible to output the N-th frame just after N-th frame is feed to encoder?

PS: Question 1 is the same to this question.(no good solution yet).

When surface graphic content suddenly become static, no more frames are feed to encoder. It's strange that the latest 6 frames data cannot output on it's "usual speed", they are slowly outputted in about 10 seconds.

Question 2 - How to reduce the latency of latest frames output when suddenly no data feed to encoder?

Drava

Answer 1

The buffering is determined by the video encoder. I don't think there's much you can do about it, other than select a video encoder that doesn't behave this way.

If you don't want the output to pause, the easiest fix is to not pause the input. Keep feeding the same frame in. The encoder should recognize that nothing has changed and generate very little output. screenrecord doesn't work this way -- it just submits frames as it gets them -- so you'd have to add a timeout that re-submits frames.

Takashi

I'm trying the same by feeding the previous sample in a loop until I get another sample from my source to my MediaFoundation h264 hardware encoder. It generates output faster(Almost realtime), but the generated samples are more than I have expected in size (Almost 300KB in size having configured bitrate 4000000 (500KB)) even when there is no change. #59051943 – Tarpon