Imx8mqevk GPU Applications.

[manizillion]

Hi ,

CAn I use video based OpenCV applications to run on the imx8 GPU,

Do I need OpenCL for it ?

If so can you share OpenCV applications which video streaming based ..

[manish_bajaj]

manizillion@gmail.com‌,

Can you elaborate when you say object detection application is very slow response on imx8mqevk?

What is slow? Inference speed is low or image capture is slow. What others feature are enabled in back ground?

-Manish

[manizillion]

Hi Manish,

I am trying to inlcude meta-ml meta-sdk layers into agl linux build. I am using these layers from latest L5.4.24_2.1.0 release on imx8mqevk. But i do get error for armnn do_compile errors  . How to solve these merge issues ?

Can i merge these builds into other build environment?

ERROR: armnn-19.08-r1 do_compile: Execution of '/home/administrator/jellyfish5.4/build/tmp/work/aarch64-agl-linux/armnn/19.08-r1/temp/run.do_compile.19578' failed with exit code 127:
/home/administrator/jellyfish5.4/build/tmp/work/aarch64-agl-linux/armnn/19.08-r1/temp/run.do_compile.19578: 1: eval: cmake: not found
WARNING: exit code 127 from a shell command.

ERROR: Logfile of failure stored in: /home/administrator/jellyfish5.4/build/tmp/work/aarch64-agl-linux/armnn/19.08-r1/temp/log.do_compile.19578

[manizillion]

Thanks Manish,

We were running a face detection algorithm with opencv using v4l2  mipi csi camera from nxp, Although the face detection is happening but response seen on the display is very slow. Application does not hang though.  Any issue with v4l and opencv compatability ? In background we have enabled the following in local.conf.

#Configure the OpenCV package:
IMAGE_INSTALL_append = "opencv python-opencv"

PACKAGECONFIG_remove_pn-opencv_mx8 = "python3"
PACKAGECONFIG_append_pn-opencv_mx8 = " dnn jasper openmp test neon python2 qt5 gtk "

#Add CMake for SDK’s cross-compile:

TOOLCHAIN_HOST_TASK_append += " nativesdk-cmake nativesdk-make"

IMAGE_INSTALL_append = " \
flatbuffers \
arm-compute-library \
tensorflow-lite \
armnn \
armnn-onnx \
stb \
"

PREFERRED_VERSION_opencv = "4.0.1%"

#Remove the OpenCL support from packages
PACKAGECONFIG_remove_pn-opencv_mx8 = "opencl"
PACKAGECONFIG_remove_pn-arm-compute-library = "opencl"

[manish_bajaj]

manizillion@gmail.com‌,

 There can various reason because of that performance might not be optimal. What is the output of top? Share your application with us.

-Manish

[manizillion]

Hi Manish,

Here is the app code below, how to make this piece of code run on gpu ? will it be faster if it runs on gpu?

// CPP program to detects face in a video

// Include required header files from OpenCV directory
#include "opencv2/objdetect.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include <iostream>

using namespace std;
using namespace cv;

// Function for Face Detection
void detectAndDraw( Mat& img, CascadeClassifier& cascade, CascadeClassifier& nestedCascade, double scale);
//void detectAndDisplay( Mat& frame, CascadeClassifier& fullbodycascade, CascadeClassifier& bodycascade);

int main( int argc, const char** argv )
{
// VideoCapture class for playing video for which faces to be detected
VideoCapture capture;
Mat frame, image;

// PreDefined trained XML classifiers with facial features
// CascadeClassifier upperbodycascade, fullbodycascade;
CascadeClassifier facecascade, eyecascade;
double scale=1;

// Load classifiers from "opencv/data/haarcascades" directory
//nestedCascade.load("haarcascade_eye_tree_eyeglasses.xml") ;

// Change path before execution
//fullbodycascade.load( "haarcascade_fullbody.xml" ) ;
//upperbodycascade.load( "haarcascade_upperbody.xml" ) ;
facecascade.load( "haarcascade_frontalface_alt.xml" ) ;
eyecascade.load( "haarcascade_eye.xml" ) ;

// Start Video..1) 0 for WebCam 2) "Path to Video" for a Local Video
capture.open(0);
if( !capture.isOpened() )
{
cout << "Camera port opening failed!" << endl;
exit(1);
}
else
{
cout << "Camera port opened successfully!" << endl;
}
// set resolution & frame rate (FPS)
capture.set(CAP_PROP_FRAME_WIDTH, 640);
capture.set(CAP_PROP_FRAME_HEIGHT,480);
//capture.set(CAP_PROP_FPS, 30);

while(1)
{
cout << "####reading frame" << endl;
capture >> frame;
if( frame.empty() )
break;
Mat frame1 = frame.clone();
//detectAndDisplay( frame1, fullbodycascade, upperbodycascade);
detectAndDraw(frame1, facecascade, eyecascade, scale);
waitKey(1);
}
capture.release();
return 0;
}

/*void detectAndDisplay( Mat& frame, CascadeClassifier& fullbodycascade, CascadeClassifier& upperbodycascade)
{
std::vector<Rect> upperbodies, fullbodies;
Mat frame_gray;

cvtColor(frame, frame_gray, COLOR_BGR2GRAY);
equalizeHist(frame_gray, frame_gray);

//-- Detect upperbody
upperbodycascade.detectMultiScale(frame_gray, upperbodies, 1.1, 3, 0 | CASCADE_SCALE_IMAGE, Size(30, 30));

cout << "upperbody size: " << upperbodies.size() << endl;
for (size_t i = 0; i < upperbodies.size(); i++)
{
rectangle(frame, upperbodies[i], Scalar(255, 0, 255), 1, 8, 0);

//-- Detect fullbody
fullbodycascade.detectMultiScale(frame_gray, fullbodies, 1.1, 3, 0 | CASCADE_SCALE_IMAGE, Size(30, 30));
cout << "full body size: " << fullbodies.size() << endl;
for (size_t i = 0; i < fullbodies.size(); i++)
{
rectangle(frame, fullbodies[i], Scalar(255, 0, 255), 1, 8, 0);
}
}

//-- Show what you got
imshow("pedestrian detection", frame);
}*/

void detectAndDraw( Mat& img, CascadeClassifier& cascade,
CascadeClassifier& nestedCascade, double scale)
{
vector<Rect> faces;
Mat gray; //smallImg;

cvtColor( img, gray, COLOR_BGR2GRAY ); // Convert to Gray Scale
//double fx = 1 / scale;

// Resize the Grayscale Image
//resize( gray, smallImg, Size(), fx, fx, INTER_LINEAR );
equalizeHist( gray, gray );

// Detect faces of different sizes using cascade classifier
cascade.detectMultiScale(gray, faces, 1.1,
3, 0|CASCADE_SCALE_IMAGE, Size(30, 30));

cout << "face size: " << faces.size() << endl;
// Draw circles around the faces
for ( size_t i = 0; i < faces.size(); i++ )
{
Rect r = faces[i];
Mat smallImgROI;
vector<Rect> nestedObjects;
Point center;
Scalar color = Scalar(255, 0, 0); // Color for Drawing tool
int radius;

double aspect_ratio = (double)r.width/r.height;
if( 0.75 < aspect_ratio && aspect_ratio < 1.3 )
{
center.x = cvRound((r.x + r.width*0.5)*scale);
center.y = cvRound((r.y + r.height*0.5)*scale);
radius = cvRound((r.width + r.height)*0.25*scale);
circle( img, center, radius, color, 3, 8, 0 );
}
else
rectangle( img, cv::Point(cvRound(r.x*scale), cvRound(r.y*scale)),
cv::Point(cvRound((r.x + r.width-1)*scale),
cvRound((r.y + r.height-1)*scale)), color, 3, 8, 0);
if( nestedCascade.empty() )
continue;
smallImgROI = gray( r );

// Detection of eyes int the input image
nestedCascade.detectMultiScale( smallImgROI, nestedObjects, 1.1, 3,
0|CASCADE_SCALE_IMAGE, Size(30, 30) );

// Draw circles around eyes
for ( size_t j = 0; j < nestedObjects.size(); j++ )
{
Rect nr = nestedObjects[j];
center.x = cvRound((r.x + nr.x + nr.width*0.5)*scale);
center.y = cvRound((r.y + nr.y + nr.height*0.5)*scale);
radius = cvRound((nr.width + nr.height)*0.25*scale);
circle( img, center, radius, color, 3, 8, 0 );
}
}

// Show Processed Image with detected faces
imshow( "Face Detection", img );
}

/*void detectAndDraw( Mat& image, CascadeClassifier& face_cascade)
{
// Detect faces
std::vector<Rect> faces;
face_cascade.detectMultiScale( image, faces, 1.1, 2, 0|CASCADE_SCALE_IMAGE, Size(30, 30) );

cout << "body size: " << faces.size() << endl;
// Draw circles on the detected faces
for( unsigned int i = 0; i < faces.size(); i++ )
{
Point center( faces[i].x + faces[i].width*0.5, faces[i].y + faces[i].height*0.5 );
ellipse( image, center, Size( faces[i].width*0.5, faces[i].height*0.5), 0, 0, 360, Scalar( 255, 0, 255 ), 4, 8, 0 );
}
imshow( "Detected Face", image );
}*/

[manish_bajaj]

nxf60449‌,

Please check this ticket.

-Manish

[Alifer_Moraes]

Hello,

Object Detection using Haar feature-based cascade classifiers is relatively slow by nature. There is no GPU optimization for it.

However, trying to resize frames used in the methods detectMultiScale to a smaller size may improve performance.

Alifer

[manish_bajaj]

Above example should work for i.MX8M too, you can refer C++ example from below link

https://source.codeaurora.org/external/imx/tensorflow-imx/tree/tensorflow/lite/examples/label_image

-Manish

[manizillion]

Hi Manish,

I see my object detection application with veryslow response on imx8mqevk , i have done the following configurations for vivante , do i miss anything to speedup camera apps ?

# Graphics libraries
PREFERRED_PROVIDER_virtual/egl ?= "imx-gpu-viv"
PREFERRED_PROVIDER_virtual/libgl ?= "imx-gpu-viv"
PREFERRED_PROVIDER_virtual/libgles1 ?= "imx-gpu-viv"
PREFERRED_PROVIDER_virtual/libgles2 ?= "imx-gpu-viv"

PREFERRED_PROVIDER_virtual/egl_imxgpu ?= "imx-gpu-viv"
PREFERRED_PROVIDER_virtual/libgl_imxgpu3d ?= "imx-gpu-viv"
PREFERRED_PROVIDER_virtual/libgles1_imxgpu3d ?= "imx-gpu-viv"
PREFERRED_PROVIDER_virtual/libgles2_imxgpu3d ?= "imx-gpu-viv"
PREFERRED_PROVIDER_virtual/libg2d ?= "imx-gpu-viv"
PREFERRED_PROVIDER_virtual/libg2d_imxdpu ?= "imx-dpu-viv"

[manish_bajaj]

Manivannan,

Please check below link. It have lot of sample application and enables GPU.

https://source.codeaurora.org/external/imxsupport/pyeiq

For TFLite we use NNAPI delegates to support GPU. For application based on TFLite use UseNNAPI(TRUE) to enable GPU.

-Manish

[manizillion]

thanks Manish,  Any reference available for imx8mq , looks like the link you shared is not for this imx8 chipset.

[manizillion]

thanks Manish,

Can you share the any cpp code base for the same GPU enable run. This was on py ..

[manish_bajaj]

You should be able to use Open CV based application on i.MX8MQ. Refer below link for sample example. 

https://source.codeaurora.org/external/imxsupport/pyeiq

[ullasbharadwaj]

Hello Manish,

Does it run on GPU? I am using i.MX8 qmmek but openCV application does not run on GPU.

Best Regards

Ullas Bharadwaj

[manish_bajaj]

ML using OpenCV framework is not accelerated on GPU. Though some one can create application on openCV. There are other component  like gstreamer and other component which can be hardware accelerated. Above link provide example of using OpenCV using Python.

-Manish 

[manizillion]

Manish,

how to port any application to run on GPU, any info on that will be helpfull