Playing an incoming RTP stream with GStreamer

Question

I'm developing a GStreamer application and struggling bit with implementing a player for incoming RTP streams. I am trying to construct a pipeline around the gstrtpbin element. I'm trying to model the pipeline using a gst-launch construction:

VIDEO_CAPS="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264"

gst-launch -v udpsrc caps=$VIDEO_CAPS port=4444 \
              ! gstrtpbin .recv_rtp_sink_0 \
              ! rtph264depay ! ffdec_h264 ! xvimagesink

When I launch the script GStreamer reports these errors:

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
/GstPipeline:pipeline0/GstRtpBin:rtpbin0/GstRtpSession:rtpsession0: ntp-ns-base = 3469468914024449000
New clock: GstSystemClock
/GstPipeline:pipeline0/GstRtpBin:rtpbin0/GstRtpSession:rtpsession0.GstPad:recv_rtp_sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
/GstPipeline:pipeline0/GstRtpBin:rtpbin0.GstGhostPad:recv_rtp_sink_0: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
/GstPipeline:pipeline0/GstRtpBin:rtpbin0.GstGhostPad:recv_rtp_sink_0.GstProxyPad:proxypad0: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
/GstPipeline:pipeline0/GstRtpBin:rtpbin0/GstRtpSession:rtpsession0.GstPad:recv_rtp_src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
/GstPipeline:pipeline0/GstRtpBin:rtpbin0/GstRtpSsrcDemux:rtpssrcdemux0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
/GstPipeline:pipeline0/GstRtpBin:rtpbin0/GstRtpJitterBuffer:rtpjitterbuffer0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
/GstPipeline:pipeline0/GstRtpBin:rtpbin0/GstRtpJitterBuffer:rtpjitterbuffer0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
/GstPipeline:pipeline0/GstRtpBin:rtpbin0/GstRtpPtDemux:rtpptdemux0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
/GstPipeline:pipeline0/GstRtpBin:rtpbin0.GstGhostPad:recv_rtp_src_0_960476599_33.GstProxyPad:proxypad1: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)33
ERROR: from element /GstPipeline:pipeline0/GstUDPSrc:udpsrc0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2378): gst_base_src_loop (): /GstPipeline:pipeline0/GstUDPSrc:udpsrc0:
streaming task paused, reason not-linked (-1)
Execution ended after 209381685 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
/GstPipeline:pipeline0/GstRtpBin:rtpbin0.GstGhostPad:recv_rtp_src_0_960476599_33: caps = NULL
/GstPipeline:pipeline0/GstRtpBin:rtpbin0/GstRtpPtDemux:rtpptdemux0.GstPad:src_33: caps = NULL
/GstPipeline:pipeline0/GstRtpBin:rtpbin0/GstRtpPtDemux:rtpptdemux0.GstPad:sink: caps = NULL
/GstPipeline:pipeline0/GstRtpBin:rtpbin0/GstRtpJitterBuffer:rtpjitterbuffer0.GstPad:sink: caps = NULL
/GstPipeline:pipeline0/GstRtpBin:rtpbin0/GstRtpJitterBuffer:rtpjitterbuffer0.GstPad:src: caps = NULL
/GstPipeline:pipeline0/GstRtpBin:rtpbin0/GstRtpSsrcDemux:rtpssrcdemux0.GstPad:src_960476599: caps = NULL
/GstPipeline:pipeline0/GstRtpBin:rtpbin0/GstRtpSsrcDemux:rtpssrcdemux0.GstPad:sink: caps = NULL
/GstPipeline:pipeline0/GstRtpBin:rtpbin0/GstRtpSession:rtpsession0.GstPad:recv_rtp_src: caps = NULL
/GstPipeline:pipeline0/GstRtpBin:rtpbin0/GstRtpSession:rtpsession0.GstPad:recv_rtp_sink: caps = NULL
/GstPipeline:pipeline0/GstRtpBin:rtpbin0.GstGhostPad:recv_rtp_sink_0: caps = NULL
/GstPipeline:pipeline0/GstUDPSrc:udpsrc0.GstPad:src: caps = NULL
Setting pipeline to NULL ...
Freeing pipeline ...

I should mention that it works with a playbin and SDP file. For example this file:

v=0
o=- 1188340656180883 1 IN IP4 127.0.0.1
s=Session streamed by GStreamer
i=server.sh
t=0 0
a=tool:GStreamer
a=type:broadcast
m=video 4444 RTP/AVP 96
c=IN IP4 127.0.0.1
a=rtpmap:96 H264/90000

Can be used to play the stream like this:

gst-launch -vvv playbin uri=file://`pwd`/stream.sdp

For completeness: I am using VLC to send data. This is the command:

vlc -I rc /usr/local/movies/sample.mp4 \
    --screen-fps=10 :screen-caching=100 \
    --sout='#transcode{vcodec=h264,venc=x264{bframes=0,keyint=40},vb=512}:\
                   rtp{mux=ts,dst=127.0.0.1,port=4444}'

Would someone help me understand why the gst-launch script fails? The error "reason not-linked" makes me think that the link between gstrtpbin and the rtph264depay elements is broken. But I don't know how to fix it.

Edit
Following RAOF's suggestions I fixed a few errors in my command. However I am using ffdec_h264 and autovideosink because on my Windows system I don't have the fluh264dec and xvimage sink plugins installed:

gst-launch-0.10 udpsrc port=4444 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264" ! .recv_rtp_sink_0 gstrtpbin ! rtpmp2tdepay ! mpegtsdemux ! ffdec_h264 ! autovideosink

Which leads to new errors:

0:00:00.743000000   516   024070A8 ERROR                 ffmpeg .:0:: non-existing PPS referenced
0:00:00.744000000   516   024070A8 ERROR                 ffmpeg .:0:: non-existing PPS referenced
0:00:00.745000000   516   024070A8 ERROR                 ffmpeg .:0:: decode_slice_header error
0:00:00.745000000   516   024070A8 ERROR                 ffmpeg .:0:: no frame!
0:00:00.812000000   516   024070A8 ERROR                 ffmpeg .:0:: non-existing PPS referenced
0:00:00.813000000   516   024070A8 ERROR                 ffmpeg .:0:: non-existi
...
ERROR: from element /GstPipeline:pipeline0/GstUDPSrc:udpsrc0: Internal data flow
 error.
Additional debug info:
..\Source\gstreamer\libs\gst\base\gstbasesrc.c(2378): gst_base_src_loop (): /Gst
Pipeline:pipeline0/GstUDPSrc:udpsrc0:
streaming task paused, reason not-negotiated (-4)
Execution ended after 4790000000 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

I'm still trying to figure out how to fix this. If you can help, then feel free to do so.

Edit2
I tested again using the SDP solution and observed that the "non-existing PPS" errors also occur, but the video does play. On the other hand, the fatal "internal data flow error" only shows when using the custom pipeline solution. I suspect that the "non-existing PPS" errors are caused by the x264 encoder. The "internal data flow error" must be caused by an error in my pipeline, or perhaps a bug in some Windows plugin. I'll so some further research...

Answer 1

As far as I can tell, you've got two problems there:

Firstly, it seems the order of sink specification is important: rather than ... ! gstrtpbin .recv_rtp_sink_0 ! ... you need to have ... ! .recv_rtp_sink_0 gstrtpbin ! ....

Secondly, vlc is sending an MPEG2 transport stream - you've got mux=ts in the rtp streaming output descriptor - but you're trying to depayload a raw h264 stream. You need to depayload the ts stream, and then demux it to get at the h264 stream data.

So, finally, the pipeline

gst-launch-0.10 -v udpsrc port=4444 \
caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264" \
! .recv_rtp_sink_0 gstrtpbin ! rtpmp2tdepay \
! mpegtsdemux ! fluh264dec ! xvimagesink

works for me, using the TS demuxer (mpegtsdemux) and h264 decoder (flutsdemux).