Qt Quick responsiveness and Open GL ES 2.0 conflict on iMX6q with Vivante

[andreatessadri]

I am developing an industrial video application on iMX6q equipped with Vivante GPU.

My problem is that, since I use directly the GPU for heavy image processing, I don't have a responsive GUI base on Qt Quick 2.

Qt Quick is based on OpenGL ES that is exactly the same API I am using for the video processing, so the GPU is quite never free to render Qt animations.

I am trying to solve the problem using semaphores in order to stop video processing when there is a touch on the GUI (touch screen display).

The result is as follow:

1) the touch is immediatly taken from the Qt Quick GUI (MouseArea onPressed) and I am able to stop video processing using sem_wait()

2) since the onPressed action triggers a state change for the button on the GUI, the related Transition animation is not fluent even if there is no more video processing ongoing

3) I used the SequentialAnimation to trigger a script at the end on the Transition in order to restart the video processing but it seems that when the script is invoked the animation was not finished

Here is the part of my QML code

MouseArea {
        id: mouseArea
        anchors.fill: parent
        hoverEnabled: true

        onPressed: {
            semaGuiEvents.wait();
            container.state = "active";
        }
        onReleased: {
            container.state = "";
        }
    }

states: State {
        name: "active";
        PropertyChanges { target: content; color: "#FCFFF5"; scale: 1.1 }
    }

    transitions: Transition {
        SequentialAnimation {
            NumberAnimation { properties: "scale"; duration: 100 }
            ScriptAction { script: releaseGuiSem(); }
        }
    }

    function releaseGuiSem() {
        semaGuiEvents.post();
    }

Since the Qt GUI is based on 2 threads (GUI thread + Rendering thread) I would like to know how to understand when the animation embedded in the transition is really finished.

Is there any signal to catch ?

Another issue is to understand when a transition animation starts.

In my case, sometimes, I don't see the part of the animation related to state transition "normal" -> "active" because the call to semaGuiEvents.wait() stops the GUI thread waiting for the video processing to stop also (the video application will release the semaphore and it will stop again letting the Qt to go).

Thanks

   Andrew (Milan, Italy)

[Bio_TICFSL]

Hi Andrea,

Which version of Yocto BSP are you using? Latest Freescale fs-limage-qt5 Build QT5 for X11, fame buffer and Wayland Backends. with this version you should be able to use QML and QuickControls, with GPU without stopping the video application. 

If you already using this version please provide more details and steps to reproduced the issue.

Regards

[andreatessadri]

I've just updated Yocto BSP to fido branch so, I guess, I am using the latest graphics libraries imx-gpu-viv.

The issue is still there, no changes.

I have also activated the GPU profiler and the result is that Vivante is 100% busy.

To reproduce it's enough to create a QML button including the code I have posted and the animation won't be fluent.

To load the GPU 100% I am doing the demosaicing of an image from a 10MP sensor according to the Malvar-He-Cutler algorithm http://graphics.cs.williams.edu/papers/BayerJGT09/bayer-jgt09.pdf

I don't think it is necessary to have the same algorithm to reproduce the issue. Every vertex+shader programs combination able to kill the GPU will be good.

Even adding a call QCoreApplication::processEvents between frames, in the application that is using the GPU at full load, it won't solve the problem.

The real issue for having the animation fluent is to keep the real-time. So, in case an animation step is to be performed when the GPU is not available, the result will be that such step won't be rendered on time.

This is why I wondering how to stop the GPU processing before the animation starts.

Looking forward

  Andrew