Combining an audio and video stream using gstreamer [closed]

Question

I am streaming an mp4(mpeg-4) file from one device to another using gstreamer over RTP stream. Basically I am splitting up the mp4 file into its audio and video file and then sending it all to the other device where it gets streamed. Now, I want to save the mp4 file to disk in the other device, but my problem is that I am able to save the audio and video files seperately and it cannot be played individually.

I am confused on how to combine both the audio and video rtp streams to form my mp4 file back and save it to a file in the other device.

Here are the command line codes :

Sender(Server)

gst-launch-0.10 -v filesrc location=/home/kuber/Desktop/sample.mp4 \
             ! qtdemux name=d \
             ! queue \
             ! rtpmp4vpay \
             ! udpsink port=5000 \
             d. \
             ! queue \
             ! rtpmp4gpay \
             ! udpsink port=5002

Reciever(client)

gst-launch-0.10 udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)MP4V-ES, profile-level-id=(string)243, config=(string)000001b0f3000001b50ee040c0cf0000010000000120008440fa282fa0f0a21f, payload=(int)96, ssrc=(uint)4291479415, clock-base=(uint)4002140493, seqnum-base=(uint)57180" \
             ! rtpmp4vdepay \
             ! ffdec_mpeg4 \
             ! xvimagesink sync=false \
             udpsrc port=5002 caps="application/x-rtp, media=(string)audio, clock-rate=(int)32000, encoding-name=(string)MPEG4-GENERIC, encoding-params=(string)2, streamtype=(string)5, profile-level-id=(string)2, mode=(string)AAC-hbr, config=(string)1290, sizelength=(string)13, indexlength=(string)3, indexdeltalength=(string)3, payload=(int)96, ssrc=(uint)501975200, clock-base=(uint)4248495069, seqnum-base=(uint)37039"\
             ! rtpmp4gdepay \
             ! faad \
             ! alsasink sync=false

Saraband

one solution that I thought of was to remove the encoder in the reciever and the sender namely : rtpmp4gpay and rtpmp4gdepay and faad. this would result in streaming of the audio and video file instead of encoded streams and then using qtmux in the reciever to merge both the audio and video stream, but I am getting errors on incompatibility of qtmux with udpsrc. Where exactly should I make that change? – Saraband

gst-launch-0.10 udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)MP4V-ES, profile-level-id=(string)243, config=(string)000001b0f3000001b50ee040c0cf0000010000000120008440fa282fa0f0a21f, payload=(int)96, ssrc=(uint)4291479415, clock-base=(uint)4002140493, seqnum-base=(uint)57180" ! rtpmp4vdepay ! queue ! qtmux name=muxer udpsrc port=5002 caps = "application/x-rtp, media=(string)audio, clock-rate=(int)32000, encoding-name=(string)MPEG4-GENERIC, encoding-params=(string)2, – Saraband

streamtype=(string)5, profile-level-id=(string)2, mode=(string)AAC-hbr, config=(string)1290, sizelength=(string)13, indexlength=(string)3, indexdeltalength=(string)3, payload=(int)96, ssrc=(uint)501975200, clock-base=(uint)4248495069, seqnum-base=(uint)37039" ! rtpmp4gdepay ! queue ! muxer. muxer. ! queue ! filesink location=/home/Desktop/finaldemo.mp4 – Saraband

So im using the above but it now shows internal data flow error. ? – Saraband

Answer 1

You can try the following pipeline to mux audio and video into a single file. Pipeline for this is as follows:

gst-launch-0.10 udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)MP4V-ES, profile-level-id=(string)243, config=(string)000001b0f3000001b50ee040c0cf0000010000000120008440fa282fa0f0a21f, payload=(int)96, ssrc=(uint)4291479415, clock-base=(uint)4002140493, seqnum-base=(uint)57180" \
             ! rtpmp4vdepay \
             ! ffdec_mpeg4 \
             ! mux. \
         udpsrc port=5002 caps="application/x-rtp, media=(string)audio, clock-rate=(int)32000, encoding-name=(string)MPEG4-GENERIC, encoding-params=(string)2, streamtype=(string)5, profile-level-id=(string)2, mode=(string)AAC-hbr, config=(string)1290, sizelength=(string)13, indexlength=(string)3, indexdeltalength=(string)3, payload=(int)96, ssrc=(uint)501975200, clock-base=(uint)4248495069, seqnum-base=(uint)37039"\
             ! rtpmp4gdepay \
             ! faad \
             ! mux. 
         matroskamux name=mux 
             ! filesink location=video.mp4 

Caltanissetta

i have unable to build pipeline from this syntex – Hyden