iMX6 Quad plus / gstreamer-1.0 / RAW mode limitation

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

iMX6 Quad plus / gstreamer-1.0 / RAW mode limitation

1,409 Views
gabrielgrange
Contributor III

We made video test with a specific board with iMX6 Quad Plus and Ethernet Gigabit link. It works under 3.14.52 kernel from yocto Jethro Linux distrib.

We tested our board with a PC that generate video flow thanks to a Windows gstreamer version (from https://gstreamer.freedesktop.org/data/pkg/windows/1.10.2 ).

The configuration is as follow:

pastedImage_8.png

In PC side, we use gstreamer Windows version to generate video flow (from https://gstreamer.freedesktop.org/data/pkg/windows/1.10.2 )

We start this command from C:\gstreamer\1.0\x86_64\bin directory:

gst-launch-1.0.exe videotestsrc ! videoconvert ! videoscale ! timeoverlay ! video/x-raw,sampling=YCbCr-4:2:0,width=720, height=576,framerate=60/1 ! queue ! rtpvrawpay -v ! udpsink host=192.10.2.83 port=1234

Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-
raw, width=(int)720, height=(int)576, framerate=(fraction)60/1, format=(string)I
420, sampling=(string)YCbCr-4:2:0, pixel-aspect-ratio=(fraction)1/1, interlace-m
ode=(string)progressive
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-
raw, width=(int)720, height=(int)576, framerate=(fraction)60/1, format=(string)I
420, sampling=(string)YCbCr-4:2:0, pixel-aspect-ratio=(fraction)1/1, interlace-m
ode=(string)progressive
/GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:src: caps = video/x-raw,
width=(int)720, height=(int)576, framerate=(fraction)60/1, format=(string)I420,
sampling=(string)YCbCr-4:2:0, pixel-aspect-ratio=(fraction)1/1, interlace-mode=
(string)progressive
/GstPipeline:pipeline0/GstTimeOverlay:timeoverlay0.GstPad:src: caps = video/x-ra
w, width=(int)720, height=(int)576, framerate=(fraction)60/1, format=(string)I42
0, sampling=(string)YCbCr-4:2:0, pixel-aspect-ratio=(fraction)1/1, interlace-mod
e=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw,
width=(int)720, height=(int)576, framerate=(fraction)60/1, format=(string)I420,
sampling=(string)YCbCr-4:2:0, pixel-aspect-ratio=(fraction)1/1, interlace-mode=
(string)progressive
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, width=(i
nt)720, height=(int)576, framerate=(fraction)60/1, format=(string)I420, sampling
=(string)YCbCr-4:2:0, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)p
rogressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw
, width=(int)720, height=(int)576, framerate=(fraction)60/1, format=(string)I420
, sampling=(string)YCbCr-4:2:0, pixel-aspect-ratio=(fraction)1/1, interlace-mode
=(string)progressive
/GstPipeline:pipeline0/GstTimeOverlay:timeoverlay0.GstPad:video_sink: caps = vid
eo/x-raw, width=(int)720, height=(int)576, framerate=(fraction)60/1, format=(str
ing)I420, sampling=(string)YCbCr-4:2:0, pixel-aspect-ratio=(fraction)1/1, interl
ace-mode=(string)progressive
/GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:sink: caps = video/x-raw
, width=(int)720, height=(int)576, framerate=(fraction)60/1, format=(string)I420
, sampling=(string)YCbCr-4:2:0, pixel-aspect-ratio=(fraction)1/1, interlace-mode
=(string)progressive
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x
-raw, width=(int)720, height=(int)576, framerate=(fraction)60/1, format=(string)
I420, sampling=(string)YCbCr-4:2:0, pixel-aspect-ratio=(fraction)1/1, interlace-
mode=(string)progressive
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw, width=(in
t)720, height=(int)576, framerate=(fraction)60/1, format=(string)I420, sampling=
(string)YCbCr-4:2:0, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)pr
ogressive
/GstPipeline:pipeline0/GstRtpVRawPay:rtpvrawpay0.GstPad:src: caps = application/
x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sa
mpling=(string)YCbCr-4:2:0, depth=(string)8, width=(string)720, height=(string)5
76, colorimetry=(string)BT601-5, payload=(int)96, ssrc=(uint)2884138433, timesta
mp-offset=(uint)3456393280, seqnum-offset=(uint)24060, a-framerate=(string)60
/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = application/x-rtp
, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, samplin
g=(string)YCbCr-4:2:0, depth=(string)8, width=(string)720, height=(string)576, c
olorimetry=(string)BT601-5, payload=(int)96, ssrc=(uint)2884138433, timestamp-of
fset=(uint)3456393280, seqnum-offset=(uint)24060, a-framerate=(string)60
/GstPipeline:pipeline0/GstRtpVRawPay:rtpvrawpay0.GstPad:sink: caps = video/x-raw
, width=(int)720, height=(int)576, framerate=(fraction)60/1, format=(string)I420
, sampling=(string)YCbCr-4:2:0, pixel-aspect-ratio=(fraction)1/1, interlace-mode
=(string)progressive
/GstPipeline:pipeline0/GstRtpVRawPay:rtpvrawpay0: timestamp = 3456393280
/GstPipeline:pipeline0/GstRtpVRawPay:rtpvrawpay0: seqnum = 24060
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

In iMX6 side, we use these shell commands :

ifconfig enp1s0 192.10.2.83 netmask 255.255.252.0

then

gst-launch-1.0 udpsrc uri=udp://192.10.2.83:1234 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW,sampling=YCbCr-4:2:0, depth=(string)8, width=(string)720, height=(string)576, payload=(int)96" ! queue ! rtpvrawdepay !  autovideosink -v

Setting pipeline to PAUSED ...

display(/dev/fb0) resolution is (1024x768).

====== OVERLAYSINK: 4.0.8 build on Jul 13 2016 15:19:55. ======

display(/dev/fb0) resolution is (1024x768).

display(/dev/fb0) resolution is (1024x768).

Pipeline is live and does not need PREROLL ...

/GstPipeline:pipeline0/GstUDPSrc:udpsrc0.GstPad:src: caps = "application/x-rtp\,\ media\=\(string\)video\,\ clock-rate\=\(int\)90000\,\ encoding-name\=\(string\)RAW\,\ sampling\=\(string\)YCbCr-4:2:0\,\ depth\=\(string\)8\,\ width\=\(string\)720\,\ height\=\(string\)576\,\ payload\=\(int\)96"

/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = "application/x-rtp\,\ media\=\(string\)video\,\ clock-rate\=\(int\)90000\,\ encoding-name\=\(string\)RAW\,\ sampling\=\(string\)YCbCr-4:2:0\,\ depth\=\(string\)8\,\ width\=\(string\)720\,\ height\=\(string\)576\,\ payload\=\(int\)96"

/GstPipeline:pipeline0/GstRtpVRawDepay:rtpvrawdepay0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)720\,\ height\=\(int\)576\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)0/1"

Setting pipeline to PLAYING ...

/GstPipeline:pipeline0/GstRtpVRawDepay:rtpvrawdepay0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)720\,\ height\=\(int\)576\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)0/1"

New clock: GstSystemClock

/GstPipeline:pipeline0/GstImxIpuVideoTransform:imxipuvideotransform0.GstPad:src: caps = "video/x-raw\,\ width\=\(int\)720\,\ height\=\(int\)576\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)0/1\,\ format\=\(string\)I420"

/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = "video/x-raw\,\ width\=\(int\)720\,\ height\=\(int\)576\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)0/1\,\ format\=\(string\)I420"

/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstOverlaySink:autovideosink0-actual-sink-overlay.GstPad:sink: caps = "video/x-raw\,\ width\=\(int\)720\,\ height\=\(int\)576\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)0/1\,\ format\=\(string\)I420"

/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink: caps = "video/x-raw\,\ width\=\(int\)720\,\ height\=\(int\)576\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)0/1\,\ format\=\(string\)I420"

/GstPipeline:pipeline0/GstImxIpuVideoTransform:imxipuvideotransform0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)720\,\ height\=\(int\)576\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)bt601\,\ framerate\=\(fraction\)0/1"

/GstPipeline:pipeline0/GstRtpVRawDepay:rtpvrawdepay0.GstPad:sink: caps = "application/x-rtp\,\ media\=\(string\)video\,\ clock-rate\=\(int\)90000\,\ encoding-name\=\(string\)RAW\,\ sampling\=\(string\)YCbCr-4:2:0\,\ depth\=\(string\)8\,\ width\=\(string\)720\,\ height\=\(string\)576\,\ payload\=\(int\)96"

 

  • As a result we obtain a wrong iMX6 video display with many abnormal black lines despite CPU usage never exceed 100% and Ethernet bandwith is under 39Mb/sec.
  • Is something is wrong in my iMX6 RAW decoding or is there are gstreamer limit with RAW format?

 

Regards,

Gabriel GRANGE

0 Kudos
3 Replies

803 Views
gabrielgrange
Contributor III

We complete our tests with iMX6 quad plus without udp transfert. We operate with CPU frequency at 792MHz.

If we generate 720x576x60fps YUV4.2.0 raw video flow (35Mo/s bandwith is need in this mode) :

gst-launch-1.0 -v videotestsrc pattern=ball ! video/x-raw,sampling=YCbCr-4:2:0,width=720, height=576,framerate=60/1 ! queue ! rtpvrawpay ! udpsink host=127.0.0.1 port=1234  &

Then if we read and display in local the video flow :

gst-launch-1.0 udpsrc uri=udp://127.0.0.1:1234 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW,sampling=YCbCr-4:2:0, depth=(string)8, width=(string)720, height=(string)576, colorimetry=(string)BT601-5, payload=(int)96" ! queue ! rtpvrawdepay ! imxipuvideosink

We have a display problem that don't seem link with CPU charge.

The result is better if we add a large buffer to receive data flow, it takes several ten seconds to generate display problem :

gst-launch-1.0 udpsrc buffer-size=900000000 uri=udp://127.0.0.1:1234 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW,sampling=YCbCr-4:2:0, depth=(string)8, width=(string)720, height=(string)576, colorimetry=(string)BT601-5, payload=(int)96" ! queue2 ! rtpvrawdepay ! decodebin ! imxipuvideosink

I don't understand why we have a problem with iMX6 performance or software gstreamer-1.0 management. How can i improve my research ?

Regards,

Gabriel Grange

0 Kudos

803 Views
Bio_TICFSL
NXP TechSupport
NXP TechSupport

Hi Gabriel,

For gstreamer mechanism and some useful samples for streaming raw videos and images, please check:

Yocto/gstreamer/streaming – Gateworks 

Hope this helps

0 Kudos

803 Views
gabrielgrange
Contributor III

Thanks for your answer.
I read these samples but i didn't found specific point to manage RAW format decoding with iMX6 or a way to improve iMX6 display process.

Regards,

Gabriel

0 Kudos