gstreamer1.0 on imx6 causes errors/segfaults with rtsp h264 streaming video

cancel
Showing results for 
Search instead for 
Did you mean: 

gstreamer1.0 on imx6 causes errors/segfaults with rtsp h264 streaming video

1,418 Views
adowdy
Contributor II

I've been trying for a while now to create a new feature for my Yocto(krogoth)/Qt 5.6 based application to render a video camera stream over network IP on an embedded linux target using an iMX6 quad.  I have a 'working' pipeline, but after awhile playing a stream fine, I see the video freeze onscreen, then sometime after freeze, various segfaults depending on nuances of the configuration I've played with (different between QtMultimedia, qt-gstreamer). Several segfaults seem to indicate an exception trying to allocate memory, buried at a very deep abstraction underneath my application. Details below, but first high-level stuff.

I am using some packs of gstreamer1.0 plugins pulled in from 'poky' layer, as well as gstreamer1.0-plugins-imx from 'meta-fsl-arm' layer. Here are some details of recipes in question

Yocto layer -> recipes (Krogoth, qt5.6):

poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.6.3.bb

poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_1.6.3.bb

(bbappend + uvch264, because without it gstreamer reports no valid h264 decoder)

meta-fsl-arm/recipes-multimedia/gstreamer/gstreamer1.0-plugins-imx_0.12.2.bb

meta-qt5/recipes-qt/qt5/qtmultimedia_git.bb

Note - I've had to do some configuration of these recipes to get it to handle h264 video in bbappends.

App runs for up to 1 hour, then segfault behavior:

Some random amount of time before crash... video frame appears to freeze, Qt reports it's playing fine, then...

GLib-ERROR **: ../../glib-2.46.2/glib/gmem.c:100: failed to allocate 65611 bytes

coredump file name
core.udpsrc32:src.1511917795.1201

Notes -- confirmed nothing else in the application can be causing a memory allocation problem. Also, QtMultimedia doesn't appear to give me any interface to Gstreamer pipeline itself so I can't currently get a dot file and see what it looks like. I even looked at the QtMultimedia source, but they didn't appear to manage any pipeline handle needed for this debug feature.

Sadly for me, QtMultimedia is the most promising way to integrate the video into the app since it's otherwise not trivial to bring the decompressed video into QtQuick QML from scratch, and I cannot change requirement away from Qt+QtQuick2 for the application side. I tried out VLC-Qt and VLC-Qml -- both don't even give me the decompressed stream at all to start with, and fail long before the screen renders video. 

Beyond QtMultimedia, The most promising way I tried was qt-gstreamer... but there's very little support and the project seems abandoned. Tragically, there seems to be even less support out there for QtMultimedia, so I decided to try it anyway.


Layer: meta-qt5-extra/recipes-misc/recipes-multimedia/qt-gstreamer/qt-gstreamer_1.2.0.bb

This too fails, slightly differently. I can get demo for qt-gstreamer 'qmlplayer2' up and running, but soon after the video stops playing and I get some error messages from Gstreamer (no segfault).

(qmlplayer2:2201): GStreamer-CRITICAL **: gst_object_unref: assertion 'object != NULL' failed
"Internal data flow error."

If i try to 'play' the stream again, I get the same error once more, but thereafter it just says that GStreamer is 'missing a plugin' and won't start again.

I also get some interesting output in dmesg from the kernel in qt-gstreamer approach. Coinciding with the gst_object_unref error, dmesg states: 
mxc_vpu 2040000.vpu_fsl: Physical memory allocation error!

This has led me to trying to debug CMA/DMA configuration in the kernel.

uname -a
Linux ado 4.1.38-agj+gd385596 #1 SMP PREEMPT Mon Dec 18 22:26:52 MST 2017 armv7l GNU/Linux

Some posts online have mentioned that this type of problem could be a CMA configuration problem, so I tried that

Most of my reference is here, but it's hard to navigate past just turning on CONFIG_CMA_DEBUG in kernel

gstreamer-imx/faq.md at master · Freescale/gstreamer-imx · GitHub 

Note about imx-gpu-viv... I am using this version, which is slightly newer than the version noted on the page as having a memory leak. Wondering if this is the latest?

meta-fsl-arm/kernel-module-imx-gpu-viv_5.0.11.p8.4+fslc.bb at master · Freescale/meta-fsl-arm · GitH... 

DMA/CMA debugging:

...

Turned on CMA debugging parameter, I can now see cma allocs and frees in dmesg, and it was revealed that the kernel is reserving 320MB of space for CMA, as it's specified here:

ENGR00295570 ARM: imx_v7_defconfig: enlarge the CMA size from 256MB to 320MB (ea578e2b) · Commits · ... 

This seems like an OK value, and I'm not sure what else I haven't been successful at increasing this buffer size because this freescale config file appears to override any local changes I try to make in Yocto config... still trying to sort that out but nevertheless I think 320 should at least be big enough for a single 1080p video stream, so I don't think this is my problem.

I'm at the point where I am trying to figure out how I can print out the full contents of my test pipeline in both QtMultimedia and qt-gstreamer approaches, because backtraces and debug printouts have not yielded any path to victory here... and I really want to know how to verify the plugins in use.

Any and all help with this problem would be supremely honored. Thank you, community! - AD

PS - I will probably update this post later when I have more info.

Labels (4)
0 Kudos
5 Replies

380 Views
marco_sinig1
Contributor I

Hi  Aaron

what did you do at the end?
I am having similar issue and I am not able to find out a proper fix.

Thanks
M

0 Kudos

380 Views
adowdy
Contributor II

Marco --

You could try to make your own branch of qtmultimedia in Qt 5.6, and do away with playbin (too much of a generalized black box that didn't work for me). Try keeping QtMultimedia's video sink intact and hook up a custom pipeline to their sink. My own demo custom pipeline looked something like this (but implemented in C++ interface instead of commandline)

gst-launch-1.0 rtspsrc debug=1 user-id="username" user-pw="mypass" location="rtsp://192.168.0.123:554/mp4video" ! rtph264depay ! h264parse ! imxvpudec ! <hook up to qtmulti video sink here in your code>

Note that this is a specific solution for h264 video, because playbin didn't work at all for me.

Not the cleanest solution, but I wasn't prepared to dive into playbin debugging just to get something working.

I hope this helps a bit

0 Kudos

380 Views
joanxie
NXP TechSupport
NXP TechSupport

how about export QT_EGLFS_IMX6_NO_FB_MULTI_BUFFER=4?? any helpful?

0 Kudos

380 Views
adowdy
Contributor II

Joan -

Thank you for responding. I was never notified that someone responded to my post. I am still trying to solve this problem, but I realized I was likely not specific enough in my initial posting.

I'm using wayland platform, not EGLFS, and with QtMultimedia I can now see what's going on in the pipeline but I still see segfaults when streaming for less than an hour of time.

Setting that environment variable doesn't appear to change anything, unfortunately. I can't find anywhere in the QtMultimedia source code that ever references anything like that either.

I attached an image I was able to extract from the pipeline that Qt builds... I can't figure out how QtMultimedia's video sink works. It doesn't appear to use any of the gst-imx plugins, nor does it appear to go to an appsrc.

playbin_71.png

0 Kudos

380 Views
joanxie
NXP TechSupport
NXP TechSupport

maybe you can refer to the link as below. many discussion there

https://community.nxp.com/docs/DOC-330047

0 Kudos