H264 Encoder to WebRTC Problems

[jcc273]

Hello,

I am having an issue getting video to properly display via webRTC and the problem seems to be the h264 encoding done by imxvpuenc_h264.  My setup is I am using gstreamer to stream RTP to a UDP sink and then using Janus Gateway to do the webRTC that can be viewed by the user when the connect to a webpage running on the device.

Now what happens is if i do this:

gst-launch-1.0 videotestsrc is-live=true ! openh264enc ! rtph264pay ! capssetter caps="application/x-rtp,profile-level-id=(string)42e01f" ! udpsink host=127.0.0.1 port=8080 &

Everything works perfectly!  The video displays on the page with no issues, but as you can see i'm using the openh264enc and doing the encoding via software.  If I swap out the encoder and use the imx's gstreamer hardware encoder like this:

gst-launch-1.0 videotestsrc is-live=true ! imxvpuenc_h264 ! rtph264pay ! capssetter caps="application/x-rtp,profile-level-id=(string)42e01f" | udpsink host=127.0.0.1 port=8080 &

Things don't work as well.  It partially works, so if i start the pipeline above AFTER i open the webpage on my client the video shows and plays fine, BUT if the pipeline has been started before I open the page or i have a short network blip the video doesn't play it just freezes at the current frame (or no frame if page loaded after stream is started).  It's like there is some data sent at the very start of the video or something that if i miss it, it won't play.  I tried doing something like this:

gst-launch-1.0 videotestsrc is-live=true ! imxvpuenc_h264 ! fdsink | ffmpeg -f h264 -i - -c:v copy -flags global_header -bsf dump_extra -f rtp rtp://127.0.0.1:8080 &

To have ffmpeg put the headers in but that didn't make any difference.

Any thoughts about what the issue with the video encoded by the hardware encoder might be?  Or if there is a better place to ask this question?  Since i can fix the issue by simply not using the hardware encoder it would seem to be a problem with that, so this seemed like the best place to start.

[abeeshmp_dct]

Please check the config-interval property of rtph264pay component, set that to say 1 , so that you will get the ID frames for every 1 second and that might solve your issue.

[igorpadykov]

Hi Jarrod

imxvpuenc_h264 is part of gstreamer-imx plugins, issues may be posted on

Issues · Freescale/gstreamer-imx · GitHub 

Alternatively one can try nxp gstreamer plugins (imx-gst1.0-plugins) using linux

documentation (Linux Guide) on link:

i.MX Software and Development Tools | NXP 

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[jcc273]

Thanks Igor!  I am going to give the other gstreamer stuff a try over the weekend and see how it goes and if it doesn't help then i'll try the github for gstreamer-imx