iMX6-to-PC h.264 Video Streaming

[dmitryv]

Hello All,

I'm trying to stream video from iMX6Q SABRE-SBP board to PC with gsteamer. I'm using Linux BSP 4.1.0 from Freescale Web with all necessary codecs installed. On PC side I have VLC player. I can successfully stream h.263 @ 1080p via 'tcpserversink' but I cannot do it for h.264 @ any resolution.

I'm aware about the bug discussed on the link below so locally I can get both h.263 and h.264 compressed files at 1080p.

gstreamer vpuenc H.264 encoder (codec=6) output not compressed

Similar problem has been discussed in the topic below but it ended up with streaming trough UDP (not clear successfuly or not).

H246 Network Streaming - iMX6 SabreLite - gstreamer

So far for me this works:

gst-launch mfw_v4lsrc device=/dev/video1 capture-mode=0 ! queue ! vpuenc codec=h263 bitrate=1000000 ! tcpserversink host=192.168.0.1 port=5000

This is not:

gst-launch mfw_v4lsrc device=/dev/video1 capture-mode=0 ! queue ! vpuenc codec=avc bitrate=1000000 ! tcpserversink host=192.168.0.1 port=5000

VLC captures h.264 stream but does not recognize it:

Output of gst-launch with debug:

MFW_GST_V4LSRC_PLUGIN 3.0.7 build on Oct  8 2013 08:22:08.

Setting pipeline to PAUSED ...

[INFO]  Product Info: i.MX6Q/D/S

vpuenc versions :smileyhappy:

        plugin: 3.0.7

        wrapper: 1.0.35(VPUWRAPPER_ARM_LINUX Build on Oct  8 2013 07:45:40)

        vpulib: 5.4.16

        firmware: 2.3.10.40778

/GstPipeline:pipeline0/MFWGstV4LSrc:mfwgstv4lsrc0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

0:00:01.350511025  3936    0x17050 WARN                     bin gstbin.c:2384:gst_bin_do_latency_func:<pipeline0> failed to query latency

New clock: GstSystemClock

0:00:01.417785358  3936 0x2bc06c30 WARN                  vpuenc vpuenc.c:1391:gst_vpuenc_sink_event: Unsupport newsegment format 2

/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1

/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1

/GstPipeline:pipeline0/GstVpuEnc:vpuenc0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1

[INFO]  chromaInterleave 0, mapType 0, linear2TiledEnable 0

/GstPipeline:pipeline0/GstVpuEnc:vpuenc0.GstPad:src: caps = video/x-h264, width=(int)640, height=(int)480, framerate=(fraction)30/1, framed=(boolean)true

/GstPipeline:pipeline0/GstTCPServerSink:tcpserversink0.GstPad:sink: caps = video/x-h264, width=(int)640, height=(int)480, framerate=(fraction)30/1, framed=(boolean)true

0:00:01.464051691  3936 0x2bc06c30 WARN                basesink gstbasesink.c:3626:gst_base_sink_chain_unlocked:<tcpserversink0> warning: Internal data flow problem.

0:00:01.464184024  3936 0x2bc06c30 WARN                basesink gstbasesink.c:3626:gst_base_sink_chain_unlocked:<tcpserversink0> warning: Received buffer without a new-segment. Assuming timestamps start from 0.

WARNING: from element /GstPipeline:pipeline0/GstTCPServerSink:tcpserversink0: Internal data flow problem.

Additional debug info:

gstbasesink.c(3626): gst_base_sink_chain_unlocked (): /GstPipeline:pipeline0/GstTCPServerSink:tcpserversink0:

Received buffer without a new-segment. Assuming timestamps start from 0.

0:00:01.466439691  3936    0xaffd0 WARN                     bin gstbin.c:2380:gst_bin_do_latency_func:<pipeline0> did not really configure latency of 0:00:00.000000000

0:00:10.838372025  3936    0x19258 WARN                GST_POLL gstpoll.c:1035:gst_poll_fd_has_closed: 0x95dc8: couldn't find fd !

0:00:10.838653025  3936    0x19258 WARN                GST_POLL gstpoll.c:1084:gst_poll_fd_has_error: 0x95dc8: couldn't find fd !

0:00:10.838765358  3936    0x19258 WARN                GST_POLL gstpoll.c:1112:gst_poll_fd_can_read_unlocked: 0x95dc8: couldn't find fd !

0:00:10.838865025  3936    0x19258 WARN                GST_POLL gstpoll.c:1184:gst_poll_fd_can_write: 0x95dc8: couldn't find fd !

'videotestsrc' shows the same behavior.

Did someone succeed to stream h.264 to PC?

--

Dmitry

[Raybiztech · Accepted solution]

Hi,

             I succeed to stream h.264 to PC. Instead of tcp i used rtp to stream h.264 over network. i connected ov5642 camera to IMX6 board. And i followed the below steps to stream.

On IMX6 board run the following commands to achieve streaming :

>> CLIENT_IP=<ip address of client machine>

>> gst-launch mfw_v4lsrc fps-n=30 ! vpuenc codec=6 ! queue ! rtph264pay ! udpsink host=$CLIENT_IP port=5000 -v

On client machine to play it through VLC follow the following steps :

>> Open text editor and copy the following content into that :

     v=0

     o=- 1223457093460663 1 IN IP4 127.0.0.1

     s=RTSP Server

     i=Codec00

     t=0 0

     a=tool:LIVE555 Streaming Media v2008.07.24

     a=type broadcast

     b=AS:21

     a=control:*

     a=source-filter: incl IN IP4 127.0.0.1

     a=rtcp-unicast: reflection

     m=video 5000 RTP/AVP 96   

     a=rtpmap:96 H264/90000

     a=framerate=30

     a=fmtp:96 profile-level-id=42001e; sprop-parameter-sets=Z0JAHqaAoD2QAA\\=\\=\\,aM4wpIAA\;

>> Save the file as H264.sdp

>>  Play the file (H264.sdp) with vlc media player

          Now you can see camera video on the client machine   

Thanks & Regards,

Sasidhar

[amassa]

Hi...again.

Ok, I was able to get this up and running (not sure why exactly)...tried the steps again. But I also see a large delay in the video. Is there any fix for this?

Thanks.

===

Hi,

I've tried running this on my i.MX6 SABRE-sdp board and setting up VLC as stated on my windows machine, however, I do not get the video displaying. The output below is on the SABRE board.

Any help is appreciated.

Thanks.

root@imx6qdlsolo:~# gst-launch mfw_v4lsrc fps-n=30 ! vpuenc codec=6 ! queue ! rtph264pay ! udpsink host=$CLIENT_IP port=5000 -v

MFW_GST_V4LSRC_PLUGIN 3.0.11 build on Feb 20 2015 10:35:33.

Setting pipeline to PAUSED ...

[INFO]  Product Info: i.MX6Q/D/S

vpuenc versions :smileyhappy:

        plugin: 3.0.11

        wrapper: 1.0.46(VPUWRAPPER_ARM_LINUX Build on Feb 20 2015 10:07:23)

        vpulib: 5.4.23

        firmware: 3.1.1.46056

/GstPipeline:pipeline0/MFWGstV4LSrc:mfwgstv4lsrc0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

/GstPipeline:pipeline0/GstVpuEnc:vpuenc0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1

[INFO]  ringBufferEnable 0, chromaInterleave 0, mapType 0, linear2TiledEnable 0

/GstPipeline:pipeline0/GstVpuEnc:vpuenc0.GstPad:src: caps = video/x-h264, width=(int)640, height=(int)480, framerate=(fraction)30/1, framed=(boolean)true

/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-h264, width=(int)640, height=(int)480, framerate=(fraction)30/1, framed=(boolean)true

/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-h264, width=(int)640, height=(int)480, framerate=(fraction)30/1, framed=(boolean)true

/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0.GstPad:sink: caps = video/x-h264, width=(int)640, height=(int)480, framerate=(fraction)30/1, framed=(boolean)true

/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, sprop-parameter-sets=(string)\"Z0JAHqaAoD2QAA\\=\\=\\,aM48gAA\\=\", payload=(int)96, ssrc=(uint)2123007203, clock-base=(uint)1161442818, seqnum-base=(uint)48058

/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0: timestamp = 1161446828

/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0: seqnum = 48058

/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, sprop-parameter-sets=(string)\"Z0JAHqaAoD2QAA\\=\\=\\,aM48gAA\\=\", payload=(int)96, ssrc=(uint)2123007203, clock-base=(uint)1161442818, seqnum-base=(uint)48058

WARNING: from element /GstPipeline:pipeline0/GstUDPSink:udpsink0: Internal data flow problem.

Additional debug info:

/home/jenkins/jobs/3.10.17-1.0.X/workspace/temp_build_dir/build_all/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r3/gstreamer-0.10.36/libs/gst/base/gstbasesink.c(3638): gst_base_sink_chain_unlocked (): /GstPipeline:pipeline0/GstUDPSink:udpsink0:

Received buffer without a new-segment. Assuming timestamps start from 0.

[jigneshmehta]

Hi Raybiztech,

I am trying to stream H264 video from IMX6 to PC, with your above given solution. I am streaming over wifi 802.11 and I am facing very much delay in receiving the video over client machine and also the quality is very poor.

The video on client side appears with a delay of about 5 to 10 seconds and sometimes it hangs and do not resume. Here the distance between IMX6 and PC is about 2 feet. We want to reduce the delay with better quality. This is quiet urgent, so any suggestion would be helpful.

Jignesh Mehta