AnsweredAssumed Answered

gstreamer1.0 on imx6 causes errors/segfaults with rtsp h264 streaming video

Question asked by Aaron Dowdy on Dec 19, 2017
Latest reply on Mar 1, 2018 by Joan Xie

I've been trying for a while now to create a new feature for my Yocto(krogoth)/Qt 5.6 based application to render a video camera stream over network IP on an embedded linux target using an iMX6 quad.  I have a 'working' pipeline, but after awhile playing a stream fine, I see the video freeze onscreen, then sometime after freeze, various segfaults depending on nuances of the configuration I've played with (different between QtMultimedia, qt-gstreamer). Several segfaults seem to indicate an exception trying to allocate memory, buried at a very deep abstraction underneath my application. Details below, but first high-level stuff.

 

I am using some packs of gstreamer1.0 plugins pulled in from 'poky' layer, as well as gstreamer1.0-plugins-imx from 'meta-fsl-arm' layer. Here are some details of recipes in question

 

Yocto layer -> recipes (Krogoth, qt5.6):

poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.6.3.bb

poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_1.6.3.bb

(bbappend + uvch264, because without it gstreamer reports no valid h264 decoder)

meta-fsl-arm/recipes-multimedia/gstreamer/gstreamer1.0-plugins-imx_0.12.2.bb

meta-qt5/recipes-qt/qt5/qtmultimedia_git.bb

 

Note - I've had to do some configuration of these recipes to get it to handle h264 video in bbappends.

 

App runs for up to 1 hour, then segfault behavior:

Some random amount of time before crash... video frame appears to freeze, Qt reports it's playing fine, then...

GLib-ERROR **: ../../glib-2.46.2/glib/gmem.c:100: failed to allocate 65611 bytes

coredump file name
core.udpsrc32:src.1511917795.1201

 

Notes -- confirmed nothing else in the application can be causing a memory allocation problem. Also, QtMultimedia doesn't appear to give me any interface to Gstreamer pipeline itself so I can't currently get a dot file and see what it looks like. I even looked at the QtMultimedia source, but they didn't appear to manage any pipeline handle needed for this debug feature.

 

Sadly for me, QtMultimedia is the most promising way to integrate the video into the app since it's otherwise not trivial to bring the decompressed video into QtQuick QML from scratch, and I cannot change requirement away from Qt+QtQuick2 for the application side. I tried out VLC-Qt and VLC-Qml -- both don't even give me the decompressed stream at all to start with, and fail long before the screen renders video. 

Beyond QtMultimedia, The most promising way I tried was qt-gstreamer... but there's very little support and the project seems abandoned. Tragically, there seems to be even less support out there for QtMultimedia, so I decided to try it anyway.


Layer: meta-qt5-extra/recipes-misc/recipes-multimedia/qt-gstreamer/qt-gstreamer_1.2.0.bb

 

This too fails, slightly differently. I can get demo for qt-gstreamer 'qmlplayer2' up and running, but soon after the video stops playing and I get some error messages from Gstreamer (no segfault).

 

(qmlplayer2:2201): GStreamer-CRITICAL **: gst_object_unref: assertion 'object != NULL' failed
"Internal data flow error."

 

If i try to 'play' the stream again, I get the same error once more, but thereafter it just says that GStreamer is 'missing a plugin' and won't start again.

 

I also get some interesting output in dmesg from the kernel in qt-gstreamer approach. Coinciding with the gst_object_unref error, dmesg states: 
mxc_vpu 2040000.vpu_fsl: Physical memory allocation error!

 

This has led me to trying to debug CMA/DMA configuration in the kernel.

uname -a
Linux ado 4.1.38-agj+gd385596 #1 SMP PREEMPT Mon Dec 18 22:26:52 MST 2017 armv7l GNU/Linux

 

Some posts online have mentioned that this type of problem could be a CMA configuration problem, so I tried that

Most of my reference is here, but it's hard to navigate past just turning on CONFIG_CMA_DEBUG in kernel

gstreamer-imx/faq.md at master · Freescale/gstreamer-imx · GitHub 

 

Note about imx-gpu-viv... I am using this version, which is slightly newer than the version noted on the page as having a memory leak. Wondering if this is the latest?

meta-fsl-arm/kernel-module-imx-gpu-viv_5.0.11.p8.4+fslc.bb at master · Freescale/meta-fsl-arm · GitHub 

 

DMA/CMA debugging:

...

Turned on CMA debugging parameter, I can now see cma allocs and frees in dmesg, and it was revealed that the kernel is reserving 320MB of space for CMA, as it's specified here:

 

ENGR00295570 ARM: imx_v7_defconfig: enlarge the CMA size from 256MB to 320MB (ea578e2b) · Commits · ARM / qmx6_kernel · … 

 

This seems like an OK value, and I'm not sure what else I haven't been successful at increasing this buffer size because this freescale config file appears to override any local changes I try to make in Yocto config... still trying to sort that out but nevertheless I think 320 should at least be big enough for a single 1080p video stream, so I don't think this is my problem.

 

I'm at the point where I am trying to figure out how I can print out the full contents of my test pipeline in both QtMultimedia and qt-gstreamer approaches, because backtraces and debug printouts have not yielded any path to victory here... and I really want to know how to verify the plugins in use.

 

Any and all help with this problem would be supremely honored. Thank you, community! - AD

 

PS - I will probably update this post later when I have more info.

Outcomes