gst launch window results in black screen

[virti_choksi]

Hello, 

I want to stream a video over a network.

I am using the following:

1. i.MX6Dual Processor as server

2. x86 System as receiver with Ubuntu 18

3. UVC Camera connected to i.MX6Dual Processor

When i run the gst pipeline, gst launch window pops up but video doesn't play it shows black screen. So i changed some parameters like: resolution, framerate,etc  but it didn't work I also tried to remove the parameter but after removing parameters it didn't show gst launch window or any error.

I have checked the camera whether it takes input or not but it is working fine.

Can you please help with the pipeline to stream a video and avoid black screen?

[brian14]

Hi @virti_choksi,

To continue with this support case, could you please share the following requirements:

What network are you using to stream this video?
Server GStreamer pipeline.
Client GStreamer pipeline.

With this information, we will be able to review the problems using GStreamer.

Best regards, Brian.

[virti_choksi]

Hello Brian_Ibarra

Thank you for the response 

I'm using LAN network 

server GStreamer pipeline:

gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! videoscale ! videorate ! "video/x-raw, width=720,height=576, format=I420, framerate=30"  ! rtpvrawpay ! udpsink host=192.168.0.105 port=5000 sync=false

client GStreamer pipeline:

 gst-launch-1.0 udpsrc port=5001 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)720, height=(string)576, payload=(int)96, a-framerate=(string)30" ! rtpvrawdepay ! videoconvert ! queue ! xvimagesink sync=false

[brian14]

Hi @virti_choksi,

Could you try using the following pipelines for server and client?

Server
gst-launch-1.0 -v v4l2src ! video/x-raw,width=720,height=576,framerate=30/1 ! videoconvert ! x264enc ! rtph264pay ! udpsink host=<IP_ADDRESS_CLIENT> port=5000

Client
gst-launch-1.0 udpsrc port=5000 ! application/x-rtp, payload=96 ! rtph264depay ! avdec_h264 ! autovideosink

Please let me know if this pipeline worked.

[virti_choksi]

Hello @brian14 ,

I tried your given pipeline but i got the following error on the server side 

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock             ----------> it was stuck here for few seconds          

-------> After few seconds it shows error 
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../../../../gstreamer-1.10.4/libs/gst/base/gstbasesrc.c(2950): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:05.794467334
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

[brian14]

Hi @virti_choksi,

Could you try with the following pipeline for the server side?

Server

gst-launch mfw_v4lsrc fps-n=30 ! vpuenc codec=6 ! queue ! rtph264pay ! udpsink host=<IP_ADDRESS_CLIENT> port=5000 –v

If this pipeline doesn't function as we expected, please share your BSP version.

Have a great day.

Best regards, Brian.

[virti_choksi]

Hello @brian14

Thank you for the response

I tried the given pipeline but it didn't work 

There was the following error :

WARNING: erroneous pipeline: no element "mfw_v4l2src"

And the BSP version I'm using is pyro.

Also there is an update now I'm able to stream but video is stucked at first frame and sometimes frame changes but it takes 5-10 seconds and video seems like images I also tried to change sink element and framerate but it didn't work 

server pipeline:

gst-launch-1.0 v4l2src ! videoconvert ! videoscale ! videorate ! "video/x-raw,width=720,height=576,format=I420,framerate=25/1" ! rtpvrawpay ! udpsink host=<Client IP Address> port=5000 sync=false async=false

Receiver pipeline:

gst-launch-1.0 udpsrc buffer-size=622080 port=5000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=YCbCr-4:2:0,depth=(string)8,width=(string)720, height=(string)576, payload=(int)96, a-framerate=25" ! rtpvrawdepay ! autovideosink window-width=720 window-height=576 sync=true

also I tried the same pipeline with the loopback IP address and it worked properly 

I have also tried another pipeline 

server GStreamer pipeline:

gst-launch-1.0 v4l2src ! videoconvert ! videoscale ! videorate ! "video/x-raw,width=720,height=576,format=I420,framerate=25/1" ! x264enc ! rtph264pay ! udpsink host=<Client ip address > port=5000 sync=false async=false

client GStreamer pipeline:

gst-launch-1.0 -v udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! autovideosink sync=false

OR

gst-launch-1.0 udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! vaapih264dec ! decodebin ! videoconvert ! autovideosink sync=false

In this case I tried following method:

1. I tried the same pipeline x86 system as sender and receiver both using loopback ip and it worked well but video was delayed by few seconds 
2. I also tried the same pipeline using two x86 system it worked well but video was delayed by few seconds 
3. But when i use the above pipeline for i.MX6 Dual processor as sender and x86 system as receiver it is stucked at the first frame.

So how can I acheive continuous streaming with low latency ?

[brian14]

Hi @virti_choksi, 

Thank you for your reply with the requested information.

First of all, please update the BSP using the following link:

Embedded Linux for i.MX Applications Processors | NXP Semiconductors

Then, please use the following reference to try with the gstreamer decode and get better results:

GStreamer i.MX6 Decoding - NXP Community

Please let me know if this works.

[virti_choksi]

Hello @brian14,

Thank you for the response 

I cannot update my BSP version because this version of BSP is required in my project.

I got the pipeline which works for me but there is latency and resolution issue. 

Our goal is to acheive 1080p resolution and latency as low as possible.  

I have attached one file below with the verbose logs.

The pipelines are as below :

Trail-1 : stretched pixels and High Latency

Transmit : 

gst-launch-1.0 v4l2src ! videoconvert ! x264enc tune=zerolatency byte-stream=true bitrate=4000  threads=2 ! rtph264pay config-interval=1 pt=96 ! udpsink host=192.168.0.106 port=5000

Receive :

gst-launch-1.0 -v udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" !  ! rtph264depay ! decodebin ! videoconvert ! autovideosink sync=false

Trail-2 :  Clear pixels and High Latency 

Transmit :  gst-launch-1.0 -v v4l2src ! videoconvert ! x264enc pass=qual quantizer=20 tune=zerolatency byte-stream=true bitrate=4000  threads=2 ! rtph264pay config-interval=1 pt=96 ! udpsink host=192.168.0.109 port=5000

Receive : gst-launch-1.0 -v udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! autovideosink sync=false

Trail-3 : reduced latency and stretched pixels 

Transmit :  gst-launch-1.0 v4l2src ! videoconvert ! x264enc pass=qual quantizer=20 speed-preset=ultrafast tune=zerolatency byte-stream=true bitrate=4000  threads=2 intra-refresh=true ! rtph264pay config-interval=1 pt=96 ! udpsink host=192.168.0.109 port=5000

Receive : gst-launch-1.0 udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! autovideosink sync=false

[brian14]

Hi @virti_choksi, 

Thank you for your detailed reply I have read it and it seems we are forgetting to use the hardware acceleration through VPU encoding.
Could you please try to add this element in the pipeline (vpuenc codec=6 bitrate=$BITRATE)?

Another test I want to try is to use a video with less information and verify if you achieve the required latency.

If this test works as we expected, we need to review the memory and make sure it is enough to correctly stream 1080p video.

Have a great day.

Best regards, Brian.

[virti_choksi]

Hello @brian14,

My system don't have vpuenc element so instead of vpuenc i have tried pipeline with imxvpuenc_h264 it is able to achieve low latency (i.e. ~350 ms)but the resolution is quit pixelated and disturbed .

So what do so suggest that i get clear 1080p video stream with low latency.

Transmit :

gst-launch-1.0  v4l2src  device=/dev/video0  !  image/jpeg, width=1920, height=1080, framerate=30/1 ! jpegdec ! videoscale ! videoconvert  ! queue ! imxvpuenc_h264  bitrate=256 ! rtph264pay ! udpsink host=<Client IP Address> port=5000  sync=false

Receive :

gst-launch-1.0 -vc udpsrc port=5000 ! application/x-rtp, payload=96 ! rtpjitterbuffer ! rtph264depay ! avdec_h264 ! fpsdisplaysink  sync=false async=false --verbose