Qt multimedia application performance on iMX8M Plus EVK

[aekam_parmar]

Hello,

I am creating a media player application that can play streams from the camera, local files (MP4, MKV), and online streams.

I have got NXP i.MX8M Plus EVK board, and decided to use the Qt framework to develop the application.

I referred to i.MX Yocto Project's User Guide, which guides through the process to create the Linux image using Yocto. I followed all the steps to create a full image (imx-image-full, to include Qt6) for the imx8mp-lpddr4-evk machine, using fsl-imx-wayland distro.

After flashing the image to the board, I ran the video widget sample application provided in Qt6 to play an MP4 file with a resolution of 1920x1080. In an ideal state (video not being played) the CPU consumption remains around 3-5%. However, as soon as the video is played, the CPU consumption shoots to around 50%.

Also, I verified the CPU consumption by playing the same video using a gstreamer pipeline as mentioned in i.MX8 GStreamer User Guide.

gst-launch-1.0 filesrc location=<filename.mp4> ! qtdemux name=d d.video_0 ! queue ! h264parse ! vpudec ! queue ! waylandsink

and

gst-launch-1.0 filesrc location=<filename.mp4> ! qtdemux name=d d.video_0 ! queue ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! queue ! waylandsink

The CPU consumption with gstreamer, in both cases, remains around 5% which is 10th of the Qt application.

Here are the questions.

• Is this normal behavior with Qt multimedia applications on i.MX8M Plus board?
• If not then what could be the problem? If a single stream consumes around 50% then running 3 streams simultaneously from a single application can exhaust the core.
• Any pointers to verify the Yocto configuration? Maybe something related to Qt configuration.
• How to verify if the Qt application is using hardware decoders and the rendering system?

Looking forward to your response to resolve this problem.

Regards,

Aekam Parmar

[Bio_TICFSL]

Hello,

Yes it's normal when working with QT, but the 50% you should be using pure cpu and not gpu, with qtdemux your ensure that you are using qt and gpu.

Regards

[aekam_parmar]

I tried running a sample OpenCV application to play a video, to see if it is just Qt or other video processing modules as well that are reporting high CPU usage.

#include "opencv2/opencv.hpp"
#include <iostream>
 
using namespace std;
using namespace cv;
 
int main(){
 
  // Create a VideoCapture object and open the input file
  // If the input is the web camera, pass 0 instead of the video file name
  VideoCapture cap("video.mp4"); 
    
  // Check if camera opened successfully
  if(!cap.isOpened()){
    cout << "Error opening video stream or file" << endl;
    return -1;
  }
   
  while(1){
 
    Mat frame;
    // Capture frame-by-frame
    cap >> frame;
  
    // If the frame is empty, break immediately
    if (frame.empty())
      break;
 
    // Display the resulting frame
    imshow( "Frame", frame );
 
    // Press  ESC on keyboard to exit
    char c=(char)waitKey(25);
    if(c==27)
      break;
  }
  
  // When everything done, release the video capture object
  cap.release();
 
  // Closes all the frames
  destroyAllWindows();
   
  return 0;
}

Also, tried passing a gstreamer pipeline to the VideoCapture.

VideoCapture cap("filesrc video.mp4 ! qtdemux name=d d.video_0 ! queue ! h264parse ! vpudec ! queue ! appsink", CAP_GSTREAMER);

With both options, the application consumes more than 50% CPU.

I am not sure if OpenCV is built to use hardware components.

Please share your opinion on this.

[Zater]

Hello
can you please share the solution yoy have found  to decode the video stream on Qt using the hardware VPU

[aekam_parmar]

Thank you for the response.

1. As far as I know, the qtdemux is responsible to split the audio and the video from the stream, not more than that. Let me know how it could make sure that GPU is utilized.
2. Did you get a chance to run the mentioned sample application on the board? If yes, then please share the statistics that you observed.
3. If you ran the sample application and observed that the CPU usage was around 50%, then what are the recommendations from your side to make sure that the Qt application utilizes hardware components to process the stream?