opencv qr code detector fails with out of memory error

[nishanth_]

Hi, for qr code detection and decoding, I am using cv2.QRCodeDetector() from opencv-python version 4.2.0 on imx8m mini. I am using ov5647 based camera module.

The kernel version I am using is 5.4.24. I have included opencv in linux by adding an entry in local.conf file in yocto project.

Attached the source code for the qr code detection and decoding.

Now, there are two problems with this.

1. the program is not able to decode qr codes that embed large amount of date

for ex: this program fails to decode qr codes that contain more than 50 chars of data

2. after running for sometime, the above program fails with out of memory error. Below is the log for that error. Also attached the full logs.

OpenCV(4.2.0) /usr/src/debug/opencv/4.2.0.imx-r0/git/modules/core/src/alloc.cpp:73: error: (-4:Insufficient memory) Failed to allocate 2756250000 bytes in function 'OutOfMemoryError'

Please suggest any leads/ possible solutions. Thanks in advance

[Bio_TICFSL]

Hello nishanth,

Can you share changes you did in local.conf to enable opencv? Can you check why around 10 GB memory is getting allocated? I see multiple post online related to insufficient memory error opencv, One common issue seems to be using 32 bit version of OpenCV and Python and recommendation is to use 64 bits version.

Regards

[nishanth_]

Hi, 

I am using 64bit python. 

also, opencv is also 64 bit.

And, in the local.conf, this is how opencv is added, below is the snapshot.

And, regarding 10GB memory allocation, I am not sure how to check why that is being allocated. Please give me some info on how to check that

Thanks for your response.

[Bio_TICFSL]

We don't have OV5647 so instead I tried on OV5640 and changed the script a bit like below.

stream = cv2.VideoCapture('v4l2src ! video/x-raw, width=640, height=480 ! videoconvert ! appsink')

I am able to run it with out any issue. How long do I need to run to get the crash?

Regards

[nishanth_]

Hi,

In my case, that program never runs for more than 5 min.

In most of the cases when I keep pointing any qr code towards the camera, it crashed in less than 3 min.

Thanks

[Bio_TICFSL]

So do you have test setup to reproduce the issue? Can you try on our EVK and OV5640 camera and confirm if you can reproduce the issue?

Regards

[nishanth_]

Hi, 

I am unable to reproduce this issue on imx8mm evk because I am currently unable to read frames at all from ov5640 camera on evk. I get the below errors while trying to read frames from camera ov5640 on evk.

1. To make sure that camera is attached properly to evk, below are the confirmations.

2. When I try to read frames from opencv-python, I get the below error.

3. When I try to construct a gstreamer pipeline, I get the below error.

So, I am unable to verify this on evk currently.

And, we do have a custom hardware which is based on imx8mm SOM board. Now on our custom board, I am able to read frames from ov5640 camera. However, I get the similar memory errors while trying to scan qr codes from camera stream with this setup also.

Below is the source code.

root@imx8mmevk:~/qr_workspace# cat opencv_qr_scan_cam.py
import cv2

# src='v4l2src ! video/x-bayer, format=bggr, width=640, height=480 ! bayer2rgb ! videoconvert ! appsink'
src='v4l2src ! video/x-raw, width=640, height=480 ! videoconvert ! appsink'
camera = cv2.VideoCapture(src)
if not camera.isOpened():
print('unable to open camera')
exit()

qr = cv2.QRCodeDetector()

while True:
ok, frame = camera.read()
if not ok:
print('unable to read frame')
break

retval, points, straight_qrcode = qr.detectAndDecode(frame)
if retval:
print('retval : ', retval)
else:
print('.', end=' ', flush=True)

camera.release()

And this is the error that happens, when I run the above program.

root@imx8mmevk:~/qr_workspace# python3 opencv_qr_scan_cam.py
[ 966.385684] ov5640_mipi 3-003c: s_stream: 1
[ WARN:0] global /usr/src/debug/opencv/4.2.0.imx-r0/git/modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . [ 984.538185] ntpd invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
[ 984.547535] CPU: 0 PID: 611 Comm: ntpd Not tainted 5.4.24-00075-gd96031ef0f61-dirty #18
[ 984.555536] Hardware name: ioGenies i.MX8MM DDR4 Honeywell TEV board (DT)
[ 984.562321] Call trace:
[ 984.564772] dump_backtrace+0x0/0x140
[ 984.568433] show_stack+0x14/0x20
[ 984.571748] dump_stack+0xb4/0xf8
[ 984.575063] dump_header+0x44/0x1ec
[ 984.578551] oom_kill_process+0x1d4/0x1d8
[ 984.582560] out_of_memory+0x170/0x4e0
[ 984.586308] __alloc_pages_slowpath+0x954/0x9f8
[ 984.590836] __alloc_pages_nodemask+0x21c/0x280
[ 984.595366] alloc_pages_current+0x7c/0xe8
[ 984.599460] __page_cache_alloc+0x80/0xa8
[ 984.603468] pagecache_get_page+0x150/0x300
[ 984.607650] filemap_fault+0x534/0x940
[ 984.611400] ext4_filemap_fault+0x30/0x8b8
[ 984.615494] __do_fault+0x4c/0x188
[ 984.618894] __handle_mm_fault+0xb5c/0x10a0
[ 984.623074] handle_mm_fault+0xdc/0x1a8
[ 984.626910] do_page_fault+0x130/0x460
[ 984.630656] do_translation_fault+0x5c/0x78
[ 984.634837] do_mem_abort+0x3c/0x98
[ 984.638323] do_el0_ia_bp_hardening+0x38/0xb8
[ 984.642678] el0_ia+0x18/0x1c
[ 984.663517] Mem-Info:
[ 984.665816] active_anon:441657 inactive_anon:4209 isolated_anon:0
[ 984.665816] active_file:38 inactive_file:39 isolated_file:0
[ 984.665816] unevictable:0 dirty:0 writeback:0 unstable:0
[ 984.665816] slab_reclaimable:3933 slab_unreclaimable:9135
[ 984.665816] mapped:396 shmem:4341 pagetables:1930 bounce:0
[ 984.665816] free:7456 free_pcp:0 free_cma:0
[ 984.681104] skip frame 0
[ 984.698236] Node 0 active_anon:1766628kB inactive_anon:16836kB active_file:0kB inactive_file:148kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:1584kB dirty:0kB writeback:0kB shmem:17364kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes
[ 984.739256] Node 0 DMA32 free:29824kB min:30000kB low:37500kB high:45000kB active_anon:1766628kB inactive_anon:16836kB active_file:4kB inactive_file:56kB unevictable:0kB writepending:0kB present:2093056kB managed:2026972kB mlocked:0kB kernel_stack:3152kB pagetables:7720kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[ 984.774570] lowmem_reserve[]: 0 0 0
[ 984.778086] Node 0 DMA32: 261*4kB (UME) 89*8kB (UME) 105*16kB (UME) 84*32kB (UME) 43*64kB (UME) 17*128kB (UME) 5*256kB (UE) 3*512kB (ME) 0*1024kB 2*2048kB (UM) 1*4096kB (U) 1*8192kB (U) 0*16384kB 0*32768kB = 30252kB
[ 984.797240] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
[ 984.813633] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=32768kB
[ 984.822167] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[ 984.833642] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=64kB
[ 984.857950] 4394 total pagecache pages
[ 984.861711] 0 pages in swap cache
[ 984.865024] Swap cache stats: add 0, delete 0, find 0/0
[ 984.870254] Free swap = 0kB
[ 984.873132] Total swap = 0kB
[ 984.880413] 523264 pages RAM
[ 984.883307] 0 pages HighMem/MovableOnly
[ 984.887148] 16521 pages reserved
[ 984.890382] 163840 pages cma reserved
[ 984.894049] 0 pages hwpoisoned
[ 984.897100] Tasks state (memory values in pages):
[ 984.903060] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[ 984.917671] [ 369] 0 369 8553 471 73728 0 -250 systemd-journal
[ 984.926731] [ 385] 0 385 3319 390 53248 0 -1000 systemd-udevd
[ 984.937649] [ 404] 994 404 20408 89 57344 0 0 systemd-timesyn
[ 984.946707] [ 449] 0 449 552 27 45056 0 0 atd
[ 984.961638] [ 463] 0 463 785 75 49152 0 0 crond
[ 984.969827] [ 465] 997 465 1116 114 45056 0 -900 dbus-daemon
[ 984.982304] [ 500] 0 500 1941 774 53248 0 0 haveged
[ 984.990668] [ 503] 0 503 2133 162 53248 0 0 ofonod
[ 985.003316] [ 513] 993 513 926 57 49152 0 0 rpcbind
[ 985.011683] [ 524] 0 524 539 32 40960 0 0 syslogd
[ 985.023298] [ 579] 0 579 1925 126 57344 0 0 systemd-logind
[ 985.032270] [ 596] 0 596 2094 1672 53248 0 0 klogd
[ 985.042890] [ 599] 996 599 4079 94 53248 0 0 systemd-network
[ 985.057654] [ 602] 995 602 1926 81 53248 0 0 systemd-resolve
[ 985.066711] [ 604] 998 604 1209 77 49152 0 0 avahi-daemon
[ 985.075506] [ 605] 998 605 1178 62 49152 0 0 avahi-daemon
[ 985.084306] [ 606] 0 606 818 150 53248 0 0 rpc.statd
[ 985.092855] [ 611] 990 611 18299 122 57344 0 0 ntpd
[ 985.100956] [ 617] 0 617 1252 26 49152 0 0 agetty
[ 985.111347] [ 618] 0 618 1140 100 45056 0 0 login
[ 985.119536] [ 794] 0 794 2320 216 61440 0 0 systemd
[ 985.130117] [ 795] 0 795 2861 482 65536 0 0 (sd-pam)
[ 985.141636] [ 800] 0 800 1174 364 40960 0 0 sh
[ 985.149576] [ 926] 0 926 2491 185 57344 0 0 wpa_supplicant
[ 985.158556] [ 931] 0 931 717 72 40960 0 0 dropbear
[ 985.167004] [ 932] 0 932 888 110 45056 0 0 bash
[ 985.177636] [ 976] 0 976 855 59 45056 0 0 sh
[ 985.185564] [ 984] 0 984 243594 7070 610304 0 0 node
[ 985.193662] [ 1069] 0 1069 170875 2042 348160 0 0 node
[ 985.203756] [ 1120] 0 1120 717 73 49152 0 0 dropbear
[ 985.212212] [ 1121] 0 1121 888 110 45056 0 0 bash
[ 985.223182] [ 1174] 0 1174 342624 28302 1359872 0 0 node
[ 985.231286] [ 1181] 0 1181 218058 1967 344064 0 0 node
[ 985.240475] [ 1200] 0 1200 1722 149 53248 0 0 sh
[ 985.248406] [ 1279] 0 1279 5460873 26522 1249280 0 0 node
[ 985.256507] [ 1419] 0 1419 477 15 45056 0 0 sleep
[ 985.269636] [ 1437] 0 1437 633843 368434 3239936 0 0 python3
[ 985.277998] [ 1443] 0 1443 477 16 40960 0 0 sleep
[ 985.286185] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-0.slice/session-c2.scope,task=python3,pid=1437,uid=0
[ 985.305654] Out of memory: Killed process 1437 (python3) total-vm:2535372kB, anon-rss:1473736kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:3164kB oom_score_adj:0
[ 985.529015] oom_reaper: reaped process 1437 (python3), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
Killed

----------------------------------------------------------------------------------------------------------------------------------------

This is the current situation, I am trying to integrate qr code scanning into my application. I am not able to succeed until now, and I am trying this here and then for more than 30 days now. I have also tried with zbar library, but I am unable to run zbar library on imx8mm. I have also posted that in this forum and for that also I am not able to figure out any solution yet.

zbar library issue on imx8mm