Editing android VideoView frames

Question

Environment:

Nexus 7 Jelly Bean 4.1.2

Problem:

I'm trying to make a Motion Detection application that works with RTSP using VideoView.

I wish that there was something like an onNewFrameListener

videoView.onNewFrame(Frame frame)

I've tried to get access to the raw frames of an RTSP stream via VideoView but couldn't find any support for that in the Android SDK.

I found out that VideoView encapsulates the Android's MediaPlayer class.

So i dived into the media_jni lib to try and find a way to access the raw frames, But couldn't find the byte buffer or whatever that represents a frame.

Question:

Anyone has an idea where or how can i find this buffer and get access to it ?

Or any other idea of implementing a Motion Detection over a VideoView ?

Even if it's sais that i need to recompile the AOSP.

Olag

Does your app target on Android 4.1 and up only? – Leaven

Yes, but if i need an earlier version i can manage that too. – Olag

Answer 1

You can extend the VideoView and override its draw(Canvas canvas) method.

• Set your bitmap to the canvas received through draw.
• Call super.draw() which will get the frame drawn onto your bitmap.

• Access the frame pixels from the bitmap.

  class MotionDetectorVideoView extends VideoView {
  public Bitmap mFrameBitmap;
  ...
      @Override
      public void draw(Canvas canvas) {
          // set your own member bitmap to canvas..
          canvas.setBitmap(mFrameBitmap);
          super.draw(canvas);
          // do whatever you want with mFrameBitmap. It now contains the frame.
          ...
          // Allocate `buffer` big enough to hold the whole frame.
          mFrameBitmap.copyPixelsToBuffer(buffer);
          ...
      }
  }
  

I don't know whether this will work. Avoid doing heavy calculation in draw, start a thread there.

Doctrinal

How can i get raw pixels of the drawn canvas ? couldn't find anything in their API about it. – Olag

There is public void copyPixelsToBuffer (Buffer dst) in Bitmap class. documentation here : developer.android.com/reference/android/graphics/… – Doctrinal

Yes, but how to get the Bitmap from the Canvas? – Olag

You already have the bitmap reference in mFrameBitmap.. Create the bitmap in constructor.. See the edit above.. – Doctrinal

So you say that when i call super.draw(canvas) it draws on the current bitmap and not replacing with a new one ? And I'm able to call NDK calculation inside draw ? – Olag

Haven't you tried it yet? Im not sure about that. Can do NDK calc if it doesnt slow down or flicker the video.. – Doctrinal

Tried it.. onDraw doesn't even get called. – Olag

What do you mean by "nothing"? – Doctrinal

I'm on debug mode, breakpoint inside the public void draw(Canvas canvas) with @Override before, and the debugger doesn't get there.. – Olag

Doesn't work ... Set Bitmap is a not supported operation for Video view... Because it's not a simple canvas , it's HardwareCanvas ! java.lang.UnsupportedOperationException at android.view.HardwareCanvas.setBitmap(HardwareCanvas.java:39) at com.zemingo.MyVideoView.dispatchDraw(MyVideoView.java:38) – Collop

Answer 2

In your case I would use the Camera Preview instead the VideoView, if you are working with live motion, not recorded videos. You can use a Camera Preview Callback to catch everyframe captured by your camera. This callback implements :

onPreviewFrame(byte[] data, Camera camera)
Called as preview frames are displayed.

Which I think it could be useful for you.

http://developer.android.com/reference/android/hardware/Camera.PreviewCallback.html

Tell if that is what you are searching for.

Good luck.

Hilarity