Qt5 Video playback on i.mx6q not working

[andreasmessersc]

I'm running some Qt5 video examples (qmlvideo) to playback a VP8 encoded video on a phyFlex-imx6-board from Phytec but the video frames are not shown on the display. The BSP yocto distro version is 2.1.2 and uses Qt-5.6.2. Qt is configured for the eglfs-backend and uses the Vivante fbdev GPU binaries (Galcore version 5.0.11.41671). As far as I could see, the video frames get decoded correctly by Gstreamer and are processed correctly by Qt until they reach the i.mx6 videonode plugin in qtmultimedia. It seems that the video frames in the end get lost in the glTexDirectVIV-Interface in the i.mx6 videonode plugin. If I disable the videonode plugin everything works fine.

I can observe the same behaviour on a SabreLITE board running the kernel (fsl 4.1.15) from Morty and Qt-5.8.0, Galcore version 5.0.11.41671.

Any ideas?

[Bio_TICFSL · Accepted solution]

This issue can be closed

[karina_valencia]

Andreas Messerschmid can you let us know the  progress in your side?

[karina_valencia]

Andreas Messerschmid based on the information provided yesterday, do you have an update?

[karina_valencia]

Andreas Messerschmid can you let us know the  progress in your side?

[andreasmessersc]

Hi Karina,

I need some more time to investigate the problem further as I am not able to work on the topic at the moment. 

Best regards 

Andreas

[chingling_wang]

Hi, Andrea,

I replied it a while ago. Did you get it?

I follow the link to enalbe it https://community.nxp.com/docs/DOC-330047,  I got it played using qmlvideo camiandes.webm.  It was vpu accelerated.

 am using FB backend. Qt is the one built with 4.1.14_2.0.0 BSP, which is 5.6.1

You keep on saying that glDirectTexVIV() has problems, so what error you get from this api call?

And, today, I tried again from scratch to prepare the sd card.  It works ok, the message printed shows that it is vpu hardware accelerated.

I built the 4.1.15-2.0.0 based on instruction here:

http://sw-stash.freescale.net/projects/IMX/repos/fsl-arm-yocto-bsp/browse?at=refs%2Fheads%2Fimx-4.1-... 

And I make sure gstreamer is built into qt based on this link https://community.nxp.com/docs/DOC-330047,

Then I play the attached video, it works ok

[andreasmessersc]

Hi Chingling,

the link you provided (http://sw-stash.freescale.net/projects/IMX/repos/fsl-arm-yocto-bsp/browse?at=refs%2Fheads%2Fimx-4.1-...) is not accessible.

The glDirectTexVIV-functions do not return any value as the return value is declared 'void'.

Regards

Andreas

[chingling_wang]

Although glDirectTexVIMap() returns void, but if something wrong, when you call glGetError() after this call, it will not return GL_NO_ERROR, it will return some error code.

Can you check the last parameter, which is physical address of decoded frame, what adrress you have passed to this api?

From your debugged info last Friday, it seems the frame is not decoded from vpu. Can you make sure you got valid decoded frame from vpu?

It seems you are using community's vpu driver and lib, not from us. I am not familiar with it.

[karina_valencia]

chinglingwang‌ do you have something else to add here?

[chingling_wang]

no.

[karina_valencia]

andreasmesserschmid‌ can you share the status of this thread?

[chingling_wang]

Hello,

This thread has been updated since I was on PTO. I didn't get your last update.  I can image your upset.

 I just tried to play it using gstreamer by various sink plugin,  all these commands works well on our board, I believe they will work on your board.

gst-launch-1.0 filesrc location=/home/root/caminandes.webm ! video/x-matroska ! matroskademux ! vpudec frame-drop=false output-format=1 ! queue max-size-buffers=2 ! overlaysink

gst-launch-1.0 filesrc location=/home/root/caminandes.webm ! video/x-matroska ! matroskademux ! vpudec frame-drop=false output-format=1 ! queue max-size-buffers=2 ! imxv4l2sink

gst-launch-1.0 filesrc location=/home/root/caminandes.webm ! video/x-matroska ! matroskademux ! vpudec frame-drop=false output-format=1 ! queue max-size-buffers=2 ! glimagesink

The glimagesink plugin should use opegles glDirectVIV() to do the rendering. it proves that this api should work. 

In your qt environment, which plugin was connected when you play the file?  you mentioned .mx6 videonode plugin., is it glimagesink plugin? what is the name of the plugin?  is it using the one in src/plugins/videonode? The principle of code here is about the same as glimagesink plugin, if glimagesink works,this one should work.  What error log do you get? Did you install all the required packet ot enable qtmultimedia(QML) with Gstreamer 1.0?

You may check the link here:https://community.nxp.com/docs/DOC-330047

I cannot play the video in qt multi media directory since these are not installed in our build. I need to rebuilt them in order to install all the required files.. This is the old version, our server has some connection issues, I cannot download the repo.  I will build it as soon as the connection is OK.

[andreasmessersc]

Hi,

the relevant qt plugin is the one in src/plugins/videonode/imx6. I did some cross checks with gstreamer, too. I was using gstreamer1.0-plugins-imx in a yocto build based on the Morty fslc yocto release for the sabrelite-Board. And I was able to play the video calling gstreamer in a fashion like this on the framebuffer:

gst-launch-1.0 filesrc location=/home/root/caminandes.webm ! video/x-matroska ! matroskademux ! imxvpudec ! queue max-size-buffers=2 ! imxeglvivsink

So the glDirectVIV()-Interface is working in principle on the framebuffer backend.  Anyway I couldn't figure out why the videonode plugin in Qt isn't working. When comparing the source code from gstreamer-imx with the videonode plugin from Qt one can see that the interface is used similarly but not identical. E. g. the textures are setup in different ways and glTexDirectVIVMap is called different. Do you have any hints? Is there any documentation available which shows the correct usage of the glTexDirectVIV-API?

[chingling_wang]

what is the bsp version 2.1.2?, I know we have version like 3.14.52,  4.1.15, but not 2.1.2,  is the full version name?

How you play your video?  Can you share me your command and video files so that I can reproduce on my board?

[andreasmessersc]

'uname -a' shows this linux kernel version on the sabrelite-Board: Linux nitrogen6x 4.1.15-2.0.0-ga+yocto+g651db0b

I run the qmlvideo example from path-to-qt5/examples/multimedia/video/qmlvideo/qmlvideo. Tested with Qt-5.6.2 and Qt-5.8.0. VP8-Demo-Video is attached.

[chingling_wang]

There is no video/qmlvideo installed in our yocto build.  I installed them manually.

I got to qtmultimedia/video/qmlvideo

when I run qmlvideo ./qmlvideo caminandes.webm, I got error:

EglFSVivIntegration will set environment variable FB_MULTI_BUFFER=2 to enable double buffering and vsync.
 If this is not desired, you can override this via: export QT_EGLFS_IMX6_NO_FB_MULTI_BUFFER=1
Unable to query physical screen size, defaulting to 100 dpi.
To override, set QT_QPA_EGLFS_PHYSICAL_WIDTH and QT_QPA_EGLFS_PHYSICAL_HEIGHT (in millimeters).
QFontDatabase: Cannot find font directory /usr/lib/fonts.

where did you install the fonts?  I cannot find in our yocto build

[andreasmessersc]

I use the fontconfig library to manage the system fonts. Maybe you can add it via yocto as well. Alternatively you could use 'export QT_QPA_FONTDIR=<path-to-fonts>' to point to your system fonts.

[chingling_wang]

I follow the link to enalbe it https://community.nxp.com/docs/DOC-330047,  I got it played using qmlvideo camiandes.webm.  It was vpu accelerated.

[andreasmessersc]

Hi,

which Qt-version did you use? Did you run it on X11 or the frambuffer backend?

[chingling_wang]

I am using FB backend. Qt is the one built with 4.1.14_2.0.0 BSP, which is 5.6.1

You keep on saying that glDirectTexVIV() has problems, so what error you get from this api call?