I have a MediaPlayer in a Fragment which retains its instance on configuration changes. The player is playing a video loaded from my assets directory. I have the scenario set up with the goal of reproducing the YouTube app playback where the audio keeps playing during the configuration changes and the display is detached and reattached to the media player.
When I start the playback and rotate the device, the position jumps forward about 6 seconds and (necessarily) the audio cuts out when this happens. Afterwards, the playback continues normally. I have no idea what could be causing this to happen.
As requested, here is the code:
public class MainFragment extends Fragment implements SurfaceHolder.Callback, MediaController.MediaPlayerControl {
private static final String TAG = MainFragment.class.getSimpleName();
AssetFileDescriptor mVideoFd;
SurfaceView mSurfaceView;
MediaPlayer mMediaPlayer;
MediaController mMediaController;
boolean mPrepared;
boolean mShouldResumePlayback;
int mBufferingPercent;
SurfaceHolder mSurfaceHolder;
@Override
public void onInflate(Activity activity, AttributeSet attrs, Bundle savedInstanceState) {
super.onInflate(activity, attrs, savedInstanceState);
final String assetFileName = "test-video.mp4";
try {
mVideoFd = activity.getAssets().openFd(assetFileName);
} catch (IOException ioe) {
Log.e(TAG, "Can't open file " + assetFileName + "!");
}
}
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setRetainInstance(true);
// initialize the media player
mMediaPlayer = new MediaPlayer();
try {
mMediaPlayer.setDataSource(mVideoFd.getFileDescriptor(), mVideoFd.getStartOffset(), mVideoFd.getLength());
} catch (IOException ioe) {
Log.e(TAG, "Unable to read video file when setting data source.");
throw new RuntimeException("Can't read assets file!");
}
mMediaPlayer.setOnPreparedListener(new MediaPlayer.OnPreparedListener() {
@Override
public void onPrepared(MediaPlayer mp) {
mPrepared = true;
}
});
mMediaPlayer.setOnBufferingUpdateListener(new MediaPlayer.OnBufferingUpdateListener() {
@Override
public void onBufferingUpdate(MediaPlayer mp, int percent) {
mBufferingPercent = percent;
}
});
mMediaPlayer.prepareAsync();
}
@Override
public View onCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState) {
super.onCreateView(inflater, container, savedInstanceState);
View view = inflater.inflate(R.layout.fragment_main, container, false);
mSurfaceView = (SurfaceView) view.findViewById(R.id.surface);
mSurfaceView.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
mMediaController.show();
}
});
mSurfaceHolder = mSurfaceView.getHolder();
if (mSurfaceHolder == null) {
throw new RuntimeException("SufraceView's holder is null");
}
mSurfaceHolder.addCallback(this);
return view;
}
@Override
public void onViewCreated(View view, Bundle savedInstanceState) {
super.onViewCreated(view, savedInstanceState);
mMediaController = new MediaController(getActivity());
mMediaController.setEnabled(false);
mMediaController.setMediaPlayer(this);
mMediaController.setAnchorView(view);
}
@Override
public void onResume() {
super.onResume();
if (mShouldResumePlayback) {
start();
} else {
mSurfaceView.post(new Runnable() {
@Override
public void run() {
mMediaController.show();
}
});
}
}
@Override
public void surfaceCreated(SurfaceHolder holder) {
mMediaPlayer.setDisplay(mSurfaceHolder);
mMediaController.setEnabled(true);
}
@Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
// nothing
}
@Override
public void surfaceDestroyed(SurfaceHolder holder) {
mMediaPlayer.setDisplay(null);
}
@Override
public void onPause() {
if (mMediaPlayer.isPlaying() && !getActivity().isChangingConfigurations()) {
pause();
mShouldResumePlayback = true;
}
super.onPause();
}
@Override
public void onSaveInstanceState(Bundle outState) {
super.onSaveInstanceState(outState);
}
@Override
public void onDestroyView() {
mMediaController.setAnchorView(null);
mMediaController = null;
mMediaPlayer.setDisplay(null);
mSurfaceHolder.removeCallback(this);
mSurfaceHolder = null;
mSurfaceView = null;
super.onDestroyView();
}
@Override
public void onDestroy() {
mMediaPlayer.release();
mMediaPlayer = null;
try {
mVideoFd.close();
} catch (IOException ioe) {
Log.e(TAG, "Can't close asset file..", ioe);
}
mVideoFd = null;
super.onDestroy();
}
// MediaControler methods:
@Override
public void start() {
mMediaPlayer.start();
}
@Override
public void pause() {
mMediaPlayer.pause();
}
@Override
public int getDuration() {
return mMediaPlayer.getDuration();
}
@Override
public int getCurrentPosition() {
return mMediaPlayer.getCurrentPosition();
}
@Override
public void seekTo(int pos) {
mMediaPlayer.seekTo(pos);
}
@Override
public boolean isPlaying() {
return mMediaPlayer.isPlaying();
}
@Override
public int getBufferPercentage() {
return mBufferingPercent;
}
@Override
public boolean canPause() {
return true;
}
@Override
public boolean canSeekBackward() {
return true;
}
@Override
public boolean canSeekForward() {
return true;
}
@Override
public int getAudioSessionId() {
return mMediaPlayer.getAudioSessionId();
}
}
The if block in the onPause method is not being hit.
Update:
After doing a bit more debugging, removing the interaction with the SurfaceHolder causes the problem to go away. In other words, if I don't setDisplay on the MediaPlayer the audio will work fine during the configuration change: no pause, no skip. It would seem there is some timing issue with setting the display on the MediaPlayer that is confusing the player.
Additionally, I have found that you must hide() the MediaController before you remove it during the configuration change. This improves stability but does not fix the skipping issue.
Another update:
If you care, the Android media stack looks like this:
MediaPlayer.java -> android_media_MediaPlayer.cpp -> MediaPlayer.cpp -> IMediaPlayer.cpp -> MediaPlayerService.cpp -> BnMediaPlayerService.cpp -> IMediaPlayerService.cpp -> *ConcreteMediaPlayer* -> *BaseMediaPlayer* (Stagefright, NuPlayerDriver, Midi, etc) -> *real MediaPlayerProxy* (AwesomePlayer, NuPlayer, etc) -> *RealMediaPlayer* (AwesomePlayerSource, NuPlayerDecoder, etc) -> Codec -> HW/SW decoder
Upon examining AwesomePlayer, it appears this awesome player takes the liberty of pausing itself for you when you setSurface():
status_t AwesomePlayer::setNativeWindow_l(const sp<ANativeWindow> &native) {
mNativeWindow = native;
if (mVideoSource == NULL) {
return OK;
}
ALOGV("attempting to reconfigure to use new surface");
bool wasPlaying = (mFlags & PLAYING) != 0;
pause_l();
mVideoRenderer.clear();
shutdownVideoDecoder_l();
status_t err = initVideoDecoder();
if (err != OK) {
ALOGE("failed to reinstantiate video decoder after surface change.");
return err;
}
if (mLastVideoTimeUs >= 0) {
mSeeking = SEEK;
mSeekTimeUs = mLastVideoTimeUs;
modifyFlags((AT_EOS | AUDIO_AT_EOS | VIDEO_AT_EOS), CLEAR);
}
if (wasPlaying) {
play_l();
}
return OK;
}
This reveals that setting the surface will cause the player to destroy whatever surface was previously being used as well as the video decoder along with it. While setting a surface to null should not cause the audio to stop, setting it to a new surface requires the video decoder to be reinitialized and the player to seek to the current location in the video. By convention, seeking will never take you further than you request, that is, if you overshoot a keyframe when seeking, you should land on the frame you overshot (as opposed to the next one).
My hypothesis, then, is that the Android MediaPlayer does not honor this convention and jumps forward to the next keyframe when seeking. This, coupled with a video source that has sparse keyframes, could explain the jumping I am experiencing. I have not looked at AwesomePlayer's implementation of seek, though. It was mentioned to me that jumping to the next keyframe is something that needs to happen if your MediaPlayer is developed with streaming in mind since the stream can be discarded as soon as it has been consumed. Point being, it might not be that far fetch to think the MediaPlayer would choose to jump forward as opposed to backwards.
Final Update:
While I still don't know why the playback skips when attaching a new Surface as the display for a MediaPlayer, thanks to the accepted answer, I have gotten the playback to be seamless during rotation.