imx8: pyeiq facial detection & face eye detection not working with live camera(v4l2, ov5640)

cancel
Showing results for 
Search instead for 
Did you mean: 

imx8: pyeiq facial detection & face eye detection not working with live camera(v4l2, ov5640)

795 Views
Contributor III

pyeiq facial detection & face eye detection working fine with USB cam but same is not working on nxp ov5640 camera on imx8mq evk (5.4.24 bsp) target. Streaming is fine but false detection happening with wrong marking and that is also random. Its fps is around 17. What could be the issue for such behavior?

Labels (1)
0 Kudos
23 Replies

133 Views
NXP Employee
NXP Employee

vsuneja63@gmail.com‌,

I will suggest to first check on what port ov5640 is connected and are your able to get it running using standard gstreamer command, and then use that port. I don't think /dev/video2 is correct. Their are 2 MIPI CSI port in 8MQ and it should be either /dev/video0 or /dev/video1.

-Manish

0 Kudos

133 Views
Contributor III

Hi Manish,

Thanks!

ov5640 cam is on /dev/video0 only, /dev/video2 is for USB webcam. As i mentioned recent error is wrt recorded video (recorded by ov5640 cam), not with live stream. Not sure why its showing /de/video2 but i am passing recorded video only as input. On the connected CSI port i could see live streaming happening through ov5640 camera(with pyeiq app & gstreamer command), face-eye false detection happening randomly. Again all case scenarios are absolutely working with USB web cam but do not with ov5640. Expecting to reproduce live camera & recorded video scenarios with ov5640 cam on your end.

0 Kudos

133 Views
NXP Employee
NXP Employee

vsuneja63@gmail.com‌,

We have not tested and verified it on 8MQ. Are you able to get ML with live camera (using OV5460) working?

I would suggest to check where this /dev/video2 coming from. 

Can you share the script/command you are using? Working/nonworking...

-Manish

0 Kudos

131 Views
Contributor III

--video_fwk=opencv flag was suggested by Alifer, without this flag nothing work.

0 Kudos

133 Views
Contributor III

Hi Manish,

ML with live ov5640 camera is only streaming but detection not working. Tested again with recorded video of ov5640 camera not getting /dev/video2 but same error video device could not initialize. Tried with other youtube videos(just changed video source in the below command), detection working fine but streaming is little slow.

Sharing the error logs below:

root@imx8mqevk:/home/pyeiq# pyeiq --run face_and_eyes_detection --video_src=/home/test_h264.avi --video_fwk=opencv
#########################################
# #
# PyeIQ - face_and_eyes_detection #
# #
#########################################


====== AIUR: 4.5.5 build on Jul 30 2020 15:31:43. ======
Core: AVI_PARSER_03.06.07 build on May 31 2019 02:09:31
file: /usr/lib/imx-mm/parser/lib_avi_parser_arm_elinux.so.3.1
Track 00 [video]: Disabled
Codec: 0, SubCodec: 0
------------------------
[ WARN:0] global /usr/src/debug/opencv/4.2.0.imx-r0/git/modules/videoio/src/cap_gstreamer.cpp (1758) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module aiurdemux0 reported: GStreamer encountered a general stream error.
[ WARN:0] global /usr/src/debug/opencv/4.2.0.imx-r0/git/modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /usr/src/debug/opencv/4.2.0.imx-r0/git/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Your video device could not be initialized. Exiting...

0 Kudos

133 Views
NXP Employee
NXP Employee

Hi,

Which version of PyeIQ are you using? This error you got on the log was fixed on the new version we released last week (2.1.0), you can update your package with the following command:

# pip3 install eiq --upgrade

Regards,

Alifer

0 Kudos

131 Views
Contributor III

Hi Alifer,

Thanks!

Tried to upgrade but looks like its already up-to-date. As I mention issue is with ov5640 camera recorded videos only not with other videos. I request if you can investigate with same environment(imx8mqevk with ov5640 camera, recorded video & live camera) setup on your side. 

0 Kudos

131 Views
NXP Employee
NXP Employee

Hello,

I tried to run it on the same environment you said (i.MX 8MQ EVK - BSP 5.4.24-2.1.0 and OV5640). I didn't see random detection, but it wasn't detecting properly if the face was far from the camera, otherwise, it worked fine.

Regards,

Alifer

0 Kudos

131 Views
Contributor III

Hi Alifer,

Thanks!

I tested here again but same no detection happening, tried with new imx8mq board & new camera module(ov5640). No face-eye detection even if I am close to less than a feet. Not sure what could be the difference. Can we enable debug mode & verify the logs for detection? Or any other way if you can suggest.  

0 Kudos

133 Views
NXP Employee
NXP Employee

Hi vsuneja63@gmail.com‌,

PyeIQ was tested and validated only on i.MX 8MP and i.MX 8QM, we never ran tests on i.MX 8MQ, but it should work.

Could you give us more details about the error you are facing? Which lines are you using to run the demo? Do you have any log messages or screenshot to help us figure out what may be wrong?

Regards,

Alifer

0 Kudos

133 Views
Contributor III

Hi,


Thanks!


g2d support is not there in imx8mq evk. Does that could be the reason for this? Here are the logs:

The specified video_src was not found.
Searching for default video device...
Using /dev/video0 as video device
[ WARN:0] global /usr/src/debug/opencv/4.2.0.imx-r0/git/modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1

0 Kudos

133 Views
NXP Employee
NXP Employee

Hello,

As Manish said, PyeIQ is just a reference and a quick way to run demos and there are a series of factors that may impact performance and result, especially on i.MX 8MQ, we didn't validate PyeIQ on 8MQ.

I don't think this is related to imxvideoconvert_g2d, because it is not supported by i.MX8 MQ and you said you were able to stream video without error.

Which line are you using to run the demo? Could you try to use "--res=vga" with your line and see if the demo will work properly?

Regards,

Alifer

0 Kudos

133 Views
Contributor III

Hi,

Recorded video by v4l2 camera & passing that as video source throwing GStreamer warning. No Streaming at all. Here are the logs:

Using /dev/video2 as video device
Resolution not supported. Using 640x480 instead.

====== AIUR: 4.5.5 build on Jul 30 2020 15:31:43. ======
        Core: AVI_PARSER_03.06.07  build on May 31 2019 02:09:31
 file: /usr/lib/imx-mm/parser/lib_avi_parser_arm_elinux.so.3.1
    Track 00 [video]: Disabled
        Codec: 0, SubCodec: 0
------------------------
[ WARN:0] global /usr/src/debug/opencv/4.2.0.imx-r0/git/modules/videoio/src/cap_gstreamer.cpp (1758) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module aiurdemux0 reported: GStreamer encountered a general stream error.
[ WARN:0] global /usr/src/debug/opencv/4.2.0.imx-r0/git/modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /usr/src/debug/opencv/4.2.0.imx-r0/git/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Your video device could not be initialized. Exiting...

0 Kudos

133 Views
NXP Employee
NXP Employee

Hello,

Please, could you send every detail of your environment? Board specs, BSP version, camera model, etc, so I can reproduce those errors to try and find a solution?

In the meantime, could you try to run the demo with the flags --res=vga --video_fwk=opencv?

Thank you,

Alifer

0 Kudos

133 Views
Contributor III

Hi Alifer,

Thanks!

Tested v4l2 camera recorded video for face eye detection with the flags "--res=vga --video_fwk=opencv", but Same issue.

0 Kudos

133 Views
NXP Employee
NXP Employee

Hello,

Thank you for the update. I'm still investigating what may be causing this issue and as soon as I find something I'll let you know.

BR,

Alifer

0 Kudos

133 Views
Contributor III

Hi Alifer,

Thanks!

Did you get a chance to reproduce the scenario with ov5640 camera on imx8mq-evk? Its a blockage for me, looking forward for your response. 

0 Kudos

133 Views
NXP Employee
NXP Employee

vsuneja63@gmail.com‌,

Can you confirm if you are referring below error for 8MQ?

Using /dev/video2 as video device
Resolution not supported. Using 640x480 instead.

-Manish

 

0 Kudos

133 Views
Contributor III

Hi Manish,

Thanks!

Yes, this error is there if using recorded video by ov5640 camera as input. 

0 Kudos

133 Views
Contributor III

Hi Alifer,

Thanks, Any update!.

0 Kudos