Camera usage with Qt QML

[ben_]

Hi

I'm trying to get the camera working in a QML project. The setup is a imx6DL with an embedded linux built with yocto. I'm using Qt 5.1.1.

The camera works with:

gst-launch mfw_v4lsrc ! mfw_v4lsink

QtMultimedia supports a camera interface with Camera/VideoOutput. I follow the simple examples but it doesn't work. When i'm trying to display the camera with QML. The camera/videooutput item in QML just displays nothing and the console shows:

(qt5-gui-integration:4156): GLib-GObject-WARNING **: invalid (NULL) pointer instance
(qt5-gui-integration:4156): GLib-GObject-CRITICAL **: g_signal_connect_data: assertion `G_TYPE_CHECK_INSTANCE (instance)' failed
(qt5-gui-integration:4156): GStreamer-CRITICAL **: gst_object_ref: assertion `object != NULL' failed
(qt5-gui-integration:4156): GStreamer-CRITICAL **: gst_object_sink: assertion `GST_IS_OBJECT (object)' failed
(qt5-gui-integration:4156): GStreamer-CRITICAL **: gst_element_get_bus: assertion `GST_IS_ELEMENT (element)' failed
(qt5-gui-integration:4156): GStreamer-CRITICAL **: gst_bus_add_watch_full: assertion `GST_IS_BUS (bus)' failed
(qt5-gui-integration:4156): GStreamer-CRITICAL **: gst_bus_set_sync_handler: assertion `GST_IS_BUS (bus)' failed
(qt5-gui-integration:4156): GStreamer-CRITICAL **: gst_object_ref: assertion `object != NULL' failed
(qt5-gui-integration:4156): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed
(qt5-gui-integration:4156): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed
(qt5-gui-integration:4156): GLib-GObject-CRITICAL **: g_object_get: assertion `G_IS_OBJECT (object)' failed
(qt5-gui-integration:4156): GStreamer-CRITICAL **: gst_element_set_state: assertion `GST_IS_ELEMENT (element)' failed
(qt5-gui-integration:4156): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed
(qt5-gui-integration:4156): GStreamer-CRITICAL **: gst_element_get_state: assertion `GST_IS_ELEMENT (element)' failed
(qt5-gui-integration:4156): GLib-GObject-CRITICAL **: g_object_get: assertion `G_IS_OBJECT (object)' failed
(qt5-gui-integration:4156): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed
(qt5-gui-integration:4156): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed
(qt5-gui-integration:4156): GStreamer-CRITICAL **: gst_element_set_state: assertion `GST_IS_ELEMENT (element)' failed
(qt5-gui-integration:4156): GStreamer-CRITICAL **: gst_element_set_state: assertion `GST_IS_ELEMENT (element)' failed
(qt5-gui-integration:4156): GStreamer-CRITICAL **: gst_bus_set_sync_handler: assertion `GST_IS_BUS (bus)' failed
(qt5-gui-integration:4156): GStreamer-CRITICAL **: gst_object_unref: assertion `object != NULL' failed
(qt5-gui-integration:4156): GLib-CRITICAL **: g_source_remove: assertion `tag > 0' failed

Now i found out that QtMultimedia uses various backends to support multiple platforms. On linux it uses the GStreamer backend. A new backend can also be written but i'd like to use the GStreamer backend.

A reason might be, that the gstreamer source is mfw_v4lsrc but usually the source is name v4l2src (this one doesn't work on the board). Maybe the GStreamer backend of QtMultimedia uses v4l2src and therefore can't open the camera. But i haven't found any documentation how to change that.

Has anyone the camera working with Qt QML (QtMultimedia)?

Does anyone know how to get it running?

Cheers

Ben

[LeonardoSandova]

To see which elements are taking from the factory (which gst elements are being instantiated), you can set the variable

GST_DEBUG=GST_ELEMENT_FACTORY:3

So set it, then run your QT test.

BTW, are you using Yocto to construct your image?

Leo

[ben_]

Yes i’m using yocto but Qt5.1.1 is built manually with the toolchain from yocto.

QML debugging is enabled. Only use this in a safe environment.

0:00:00.047916999 [331m 624[00m 0x12fd60 [32;01mINFO [00m [00;01;37;41m GST_ELEMENT_FACTORY ~/yocto/build-bp/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/gst/gstelementfactory.c:467:gst_element_factory_make:[00m no such element factory "camerabin2"!

(qt5-gui-integration:624): GLib-GObject-WARNING **: invalid (NULL) pointer instance

(qt5-gui-integration:624): GLib-GObject-CRITICAL **: g_signal_connect_data: assertion `G_TYPE_CHECK_INSTANCE (instance)' failed

(qt5-gui-integration:624): GStreamer-CRITICAL **: gst_object_ref: assertion `object != NULL' failed

(qt5-gui-integration:624): GStreamer-CRITICAL **: gst_object_sink: assertion `GST_IS_OBJECT (object)' failed

(qt5-gui-integration:624): GStreamer-CRITICAL **: gst_element_get_bus: assertion `GST_IS_ELEMENT (element)' failed

(qt5-gui-integration:624): GStreamer-CRITICAL **: gst_bus_set_sync_handler: assertion `GST_IS_BUS (bus)' failed

(qt5-gui-integration:624): GStreamer-CRITICAL **: gst_object_ref: assertion `object != NULL' failed

(qt5-gui-integration:624): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed

(qt5-gui-integration:624): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed

0:00:01.703821998 [331m 624[00m 0x12fd60 [32;01mINFO [00m [00;01;37;41m GST_ELEMENT_FACTORY ~/yocto/build-bp/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/gst/gstelementfactory.c:467:gst_element_factory_make:[00m no such element factory "subdevsrc"!

0:00:01.703929665 [331m 624[00m 0x12fd60 [32;01mINFO [00m [00;01;37;41m GST_ELEMENT_FACTORY ~/yocto/build-bp/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/gst/gstelementfactory.c:467:gst_element_factory_make:[00m no such element factory "wrappercamerabinsrc"!

(qt5-gui-integration:624): GLib-GObject-CRITICAL **: g_object_get: assertion `G_IS_OBJECT (object)' failed

(qt5-gui-integration:624): GStreamer-CRITICAL **: gst_element_set_state: assertion `GST_IS_ELEMENT (element)' failed

(qt5-gui-integration:624): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed

(qt5-gui-integration:624): GStreamer-CRITICAL **: gst_element_get_state: assertion `GST_IS_ELEMENT (element)' failed

(qt5-gui-integration:624): GLib-GObject-CRITICAL **: g_object_get: assertion `G_IS_OBJECT (object)' failed

(qt5-gui-integration:624): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed

(qt5-gui-integration:624): GLib-GObject-CRITICAL **: g_object_set: assertion `G_IS_OBJECT (object)' failed

(qt5-gui-integration:624): GStreamer-CRITICAL **: gst_element_set_state: assertion `GST_IS_ELEMENT (element)' failed

It looks like there is nothing there :smileywink:

camerabin2, subdevsrc and wrappercamerabinsrc ist missing.

Do I have to add them manually in the yocto build?

Cheers

Ben

[LeonardoSandova]

You need to dig into the QT source code and manually change the default elements (the whole pipeline instead of camerabin2). Once you have the patch, please share it to the meta-freescale community. OtavioSalvador I am not looking closely to the distribution list, but do you know if someone has work on this issue?

[ben_]

I got a step closer: the plugins for gstreamer 0.10 (camerabin2, …) were missing from the yocto build. Now I rebuilt an image with those plugins and now Qt can build a full gstreamer pipeline. It uses the autovideosrc and this uses v4l2src which is not correct.

Here is the Output:

QML debugging is enabled. Only use this in a safe environment.

0:00:00.100359333 00;01;37;41m GST_ELEMENT_FACTORY creating element "camerabin2" named "camerabin2"

0:00:00.101360333 00;01;37;41m GST_ELEMENT_FACTORY creating element "viewfinderbin" named "vf-bin"

0:00:00.106285999 00;01;37;41m GST_ELEMENT_FACTORY creating element "capsfilter" named "videobin-capsfilter"

0:00:00.107177332 00;01;37;41m GST_ELEMENT_FACTORY creating element "capsfilter" named "imagebin-capsfilter"

0:00:00.107428332 00;01;37;41m GST_ELEMENT_FACTORY creating element "capsfilter" named "viewfinderbin-capsfilter"

0:00:00.107960666 00;01;37;41m GST_ELEMENT_FACTORY creating element "capsfilter" named "audio-capsfilter"

0:00:00.111536999 00;01;37;41m GST_ELEMENT_FACTORY creating element "volume" named "audio-volume"

0:00:00.216327666 00;01;37;41m GST_ELEMENT_FACTORY no such element factory "subdevsrc"!

0:00:00.216451332 00;01;37;41m GST_ELEMENT_FACTORY creating element "wrappercamerabinsrc" named "camera_source"

0:00:00.217412999 00;01;37;41m GST_ELEMENT_FACTORY creating element "pipeline" named "preview-pipeline"

0:00:00.219247666 00;01;37;41m GST_ELEMENT_FACTORY creating element "appsrc" named "preview-appsrc"

0:00:00.220626666 00;01;37;41m GST_ELEMENT_FACTORY creating element "appsink" named "preview-appsink"

0:00:00.223330999 00;01;37;41m GST_ELEMENT_FACTORY creating element "ffmpegcolorspace" named "preview-csp"

0:00:00.229618333 00;01;37;41m GST_ELEMENT_FACTORY creating element "videoscale" named "preview-vscale"

0:00:00.237731666 00;01;37;41m GST_ELEMENT_FACTORY creating element "encodebin" named "video-encodebin"

0:00:00.241304666 00;01;37;41m GST_ELEMENT_FACTORY creating element "filesink" named "videobin-filesink"

0:00:00.242116333 00;01;37;41m GST_ELEMENT_FACTORY creating element "encodebin" named "image-encodebin"

0:00:00.247277666 00;01;37;41m GST_ELEMENT_FACTORY creating element "multifilesink" named "imagebin-filesink"

0:00:00.247831666 00;01;37;41m GST_ELEMENT_FACTORY creating element "queue" named "viewfinderbin-queue"

0:00:00.254614333 00;01;37;41m GST_ELEMENT_FACTORY creating element "oggmux" named "muxer"

0:00:00.259720000 00;01;37;41m GST_ELEMENT_FACTORY creating element "theoraenc"

0:00:00.279953666 00;01;37;41m GST_ELEMENT_FACTORY creating element "queue"

0:00:00.280539000 00;01;37;41m GST_ELEMENT_FACTORY creating element "capsfilter"

0:00:00.282204999 00;01;37;41m GST_ELEMENT_FACTORY creating element "queue"

0:00:00.283420333 00;01;37;41m GST_ELEMENT_FACTORY creating element "ffmpegcolorspace"

0:00:00.283727333 00;01;37;41m GST_ELEMENT_FACTORY creating element "videoscale"

0:00:00.284063999 00;01;37;41m GST_ELEMENT_FACTORY creating element "ffmpegcolorspace"

0:00:00.286209666 00;01;37;41m GST_ELEMENT_FACTORY creating element "videorate"

0:00:00.292783333 00;01;37;41m GST_ELEMENT_FACTORY creating element "vorbisenc"

0:00:00.294217333 00;01;37;41m GST_ELEMENT_FACTORY creating element "queue"

0:00:00.294771666 00;01;37;41m GST_ELEMENT_FACTORY creating element "capsfilter"

0:00:00.295700999 00;01;37;41m GST_ELEMENT_FACTORY creating element "queue"

0:00:00.298252332 00;01;37;41m GST_ELEMENT_FACTORY creating element "audiorate"

0:00:00.301270666 00;01;37;41m GST_ELEMENT_FACTORY creating element "audioconvert"

0:00:00.302380333 00;01;37;41m GST_ELEMENT_FACTORY creating element "audioconvert"

0:00:00.304142333 00;01;37;41m GST_ELEMENT_FACTORY creating element "audioresample"

0:00:00.475381666 00;01;37;41m GST_ELEMENT_FACTORY creating element "jifmux" named "muxer"

0:00:00.482562333 00;01;37;41m GST_ELEMENT_FACTORY creating element "vpuenc"

0:00:00.488153999 00;01;37;41m GST_ELEMENT_FACTORY creating element "queue"

0:00:00.488767332 00;01;37;41m GST_ELEMENT_FACTORY creating element "capsfilter"

0:00:00.489695000 00;01;37;41m GST_ELEMENT_FACTORY creating element "queue"

0:00:00.490935333 00;01;37;41m GST_ELEMENT_FACTORY creating element "ffmpegcolorspace"

0:00:00.491209666 00;01;37;41m GST_ELEMENT_FACTORY creating element "videoscale"

0:00:00.491508999 00;01;37;41m GST_ELEMENT_FACTORY creating element "ffmpegcolorspace"

0:00:00.619578000 00;01;37;41m GST_ELEMENT_FACTORY creating element "autoaudiosrc" named "audiosrc"

0:00:00.620081666 00;01;37;41m GST_ELEMENT_FACTORY creating element "fakesrc" named "tempsrc"

0:00:00.668404666 00;01;37;41m GST_ELEMENT_FACTORY creating element "ffmpegcolorspace" named "vfbin-csp"

0:00:00.668781332 00;01;37;41m GST_ELEMENT_FACTORY creating element "videoscale" named "vfbin->videoscale"

0:00:00.714416666 00;01;37;41m GST_ELEMENT_FACTORY creating element "autovideosrc" named "camerasrc-real-src"

0:00:00.714892666 00;01;37;41m GST_ELEMENT_FACTORY creating element "fakesrc" named "tempsrc"

0:00:00.721337666 00;01;37;41m GST_ELEMENT_FACTORY creating element "v4l2src" named "camerasrc-real-src-actual-src-v4l"

0:00:00.723816666 00;01;37;41m GST_ELEMENT_FACTORY creating element "v4l2src" named "camerasrc-real-src"

Now I have to search if it is possible to deactivate v4l2src or tell autovideosrc to take mfw_v4lsrc.

But I’m on training this week, I can only look into it next week.

Cheers

[dmitrypozhidaev]

Have you found a solution how to deactivate v4l2src or tell autovideosrc to take mfw_v4lsrc?

best regards