I've been trying for a while now to create a new feature for my Yocto(krogoth)/Qt 5.6 based application to render a video camera stream over network IP on an embedded linux target using an iMX6 quad. I have a 'working' pipeline, but after awhile playing a stream fine, I see the video freeze onscreen, then sometime after freeze, various segfaults depending on nuances of the configuration I've played with (different between QtMultimedia, qt-gstreamer). Several segfaults seem to indicate an exception trying to allocate memory, buried at a very deep abstraction underneath my application. Details below, but first high-level stuff.
I am using some packs of gstreamer1.0 plugins pulled in from 'poky' layer, as well as gstreamer1.0-plugins-imx from 'meta-fsl-arm' layer. Here are some details of recipes in question
Yocto layer -> recipes (Krogoth, qt5.6):
(bbappend + uvch264, because without it gstreamer reports no valid h264 decoder)
Note - I've had to do some configuration of these recipes to get it to handle h264 video in bbappends.
App runs for up to 1 hour, then segfault behavior:
Some random amount of time before crash... video frame appears to freeze, Qt reports it's playing fine, then...
GLib-ERROR **: ../../glib-2.46.2/glib/gmem.c:100: failed to allocate 65611 bytes
coredump file name
Notes -- confirmed nothing else in the application can be causing a memory allocation problem. Also, QtMultimedia doesn't appear to give me any interface to Gstreamer pipeline itself so I can't currently get a dot file and see what it looks like. I even looked at the QtMultimedia source, but they didn't appear to manage any pipeline handle needed for this debug feature.
Sadly for me, QtMultimedia is the most promising way to integrate the video into the app since it's otherwise not trivial to bring the decompressed video into QtQuick QML from scratch, and I cannot change requirement away from Qt+QtQuick2 for the application side. I tried out VLC-Qt and VLC-Qml -- both don't even give me the decompressed stream at all to start with, and fail long before the screen renders video.
Beyond QtMultimedia, The most promising way I tried was qt-gstreamer... but there's very little support and the project seems abandoned. Tragically, there seems to be even less support out there for QtMultimedia, so I decided to try it anyway.
This too fails, slightly differently. I can get demo for qt-gstreamer 'qmlplayer2' up and running, but soon after the video stops playing and I get some error messages from Gstreamer (no segfault).
(qmlplayer2:2201): GStreamer-CRITICAL **: gst_object_unref: assertion 'object != NULL' failed
"Internal data flow error."
If i try to 'play' the stream again, I get the same error once more, but thereafter it just says that GStreamer is 'missing a plugin' and won't start again.
I also get some interesting output in dmesg from the kernel in qt-gstreamer approach. Coinciding with the gst_object_unref error, dmesg states:
mxc_vpu 2040000.vpu_fsl: Physical memory allocation error!
This has led me to trying to debug CMA/DMA configuration in the kernel.
Linux ado 4.1.38-agj+gd385596 #1 SMP PREEMPT Mon Dec 18 22:26:52 MST 2017 armv7l GNU/Linux
Some posts online have mentioned that this type of problem could be a CMA configuration problem, so I tried that
Most of my reference is here, but it's hard to navigate past just turning on CONFIG_CMA_DEBUG in kernel
Note about imx-gpu-viv... I am using this version, which is slightly newer than the version noted on the page as having a memory leak. Wondering if this is the latest?
Turned on CMA debugging parameter, I can now see cma allocs and frees in dmesg, and it was revealed that the kernel is reserving 320MB of space for CMA, as it's specified here:
This seems like an OK value, and I'm not sure what else I haven't been successful at increasing this buffer size because this freescale config file appears to override any local changes I try to make in Yocto config... still trying to sort that out but nevertheless I think 320 should at least be big enough for a single 1080p video stream, so I don't think this is my problem.
I'm at the point where I am trying to figure out how I can print out the full contents of my test pipeline in both QtMultimedia and qt-gstreamer approaches, because backtraces and debug printouts have not yielded any path to victory here... and I really want to know how to verify the plugins in use.
Any and all help with this problem would be supremely honored. Thank you, community! - AD
PS - I will probably update this post later when I have more info.
You could try to make your own branch of qtmultimedia in Qt 5.6, and do away with playbin (too much of a generalized black box that didn't work for me). Try keeping QtMultimedia's video sink intact and hook up a custom pipeline to their sink. My own demo custom pipeline looked something like this (but implemented in C++ interface instead of commandline)
gst-launch-1.0 rtspsrc debug=1 user-id="username" user-pw="mypass" location="rtsp://192.168.0.123:554/mp4video" ! rtph264depay ! h264parse ! imxvpudec ! <hook up to qtmulti video sink here in your code>
Note that this is a specific solution for h264 video, because playbin didn't work at all for me.
Not the cleanest solution, but I wasn't prepared to dive into playbin debugging just to get something working.
I hope this helps a bit
Thank you for responding. I was never notified that someone responded to my post. I am still trying to solve this problem, but I realized I was likely not specific enough in my initial posting.
I'm using wayland platform, not EGLFS, and with QtMultimedia I can now see what's going on in the pipeline but I still see segfaults when streaming for less than an hour of time.
Setting that environment variable doesn't appear to change anything, unfortunately. I can't find anywhere in the QtMultimedia source code that ever references anything like that either.
I attached an image I was able to extract from the pipeline that Qt builds... I can't figure out how QtMultimedia's video sink works. It doesn't appear to use any of the gst-imx plugins, nor does it appear to go to an appsrc.