Send Android h264 capture over a rtp stream

Question

I'm writing a rtp video streamer for android that reads h264 coded data from an Android local socket and packetize it. The thing is that I did it but I keep getting black frames in the client side (Voip).

The communication goes like this: Android -> Asterisk -> Jitsi (Osx) (and reverse)

There are a few things that I haven't understood yet:

1) Android's mediarecorder gives me a raw h264 stream, How can I know when a NAL starts / ends based on that stream? It doesn't have any 0x000001 pattern but it does have a 0x0000 (which I'm assuming is a separator)

EDIT:

Adding more information. These are 2 (first, second) different reads of the input buffer (in order). If I got it right the first 4 bytes should be used to get the NALU length and the 5th one (index 4) is the NALU header.

I'll copy here the byte's values for future usage:

1) 0 0 12 114 101 -72 4 25 -15 -1 -121 -53 .....

   length = (114&0xFF) + (12&0xFF)*256 + (0&0xFF)
   length -> 3186

   forbidden = 101 & 0x80
   forbidden -> 0

   nri = 101 & 0x60
   nri -> 96

   nal_unit_type = 101 & 0x1F
   nal_unit_type -> 5


2) 0 0 1 -93 97 -32 32 103 -14 93 -1 .... 

   length = (-93&0xFF) + (1&0xFF)*256 + (0&0xFF)
   length -> 419

   forbidden = 97 & 0x80
   forbidden -> 0

   nri = 97 & 0x60
   nri -> 96

   nal_unit_type = 97 & 0x1F
   nal_unit_type -> 1

is this correct?

2) How can I get the NALu timestamp and its length from that stream?

3) For some reason the packets are being marked (Even when I unset the marker). (In case you check the pcap file) [FIXED: I wasn't using the same SSCR for every packet]

Here's a pcap capture of the stream coming from asterisk (wich comes from Android). The Android device is an Asus Transform Prime with Android ICS.

I'm sending the packetization-mode (1) and profile-level-id (42801e) in the sdp, I've also tried sending the sprops (sps: Z0KAHpWgUHxA, psp: aM48gA==) parameter but nothing changed.

Cheers.

Garibald

Answer 1

1. There is no "android h264 stream format". Packetization should follow RFC6184 if RTP is used as the transport protocol. The start code is not present of the RTP payload format. What is contained in an RTP packet, depends entirely on the packetization mode which specifies if NAL units may be aggregated, fragmented, etc. Read the RFC section on packetization modes for more info. Packetization mode is usually communicated via SDP.

2. Timestamp is part of RTP header. Length of a NAL unit again depends on packetization mode.

3. What do you mean by marked? Do you mean that the RTP marker bit is set? If so, this again must follow the rules laid out in the RFC.

Overliberal

Android sends a raw h264 stream that has to be parsed (AFAIU) but how can I know where a NAL should start / end? That's my point and I think I'm missing something. – Garibald

Are you sure that it isn't using RTP? – Overliberal

AFAIK it isn't using RTP. I haven't found any documentation saying that either. – Garibald