IMX6Solo Android 5.0 - Out of memory and no killable processes...

[ennebi]

Hello

I am trying to prepare a custom Android 5.0 build with slight changes from Sabre board (Hummingboard from Solidrun).

I have a booting kernel that runs out of memory after starting to kill a number of processes. I have several "main" processes that are consuming RAM. When I lower the kernel "cma" parameter from 380MB to 250M and down to 64M, I see that more processes can be accomodated, but I in the case below I have 8 "main" processes each of them consuming about 30MB each. Can you please point me to what is this "main" process, and what I could look at?

request_suspend_state: wakeup (0->0) at 101799750344 (1970-01-01 00:02:28.868573678 UTC)
init: untracked pid 930 killed by signal 9
mediaserver invoked oom-killer: gfp_mask=0x2084d0, order=0, oom_score_adj=-1000
CPU: 0 PID: 932 Comm: mediaserver Tainted: G        W    3.10.53-88204-g4f0dd92-dirty #7
[<80014404>] (unwind_backtrace+0x0/0xf8) from [<80011964>] (show_stack+0x10/0x14)
[<80011964>] (show_stack+0x10/0x14) from [<807f0098>] (dump_header.isra.10+0x6c/0x188)
[<807f0098>] (dump_header.isra.10+0x6c/0x188) from [<800ac1f0>] (out_of_memory+0x29c/0x2d4)
[<800ac1f0>] (out_of_memory+0x29c/0x2d4) from [<800b07bc>] (__alloc_pages_nodemask+0x848/0x870)
[<800b07bc>] (__alloc_pages_nodemask+0x848/0x870) from [<800c64f8>] (__pte_alloc+0x24/0x144)
[<800c64f8>] (__pte_alloc+0x24/0x144) from [<800c9414>] (handle_mm_fault+0xb0/0xc4)
[<800c9414>] (handle_mm_fault+0xb0/0xc4) from [<80019180>] (do_page_fault+0x240/0x37c)
[<80019180>] (do_page_fault+0x240/0x37c) from [<800083c4>] (do_DataAbort+0x34/0x9c)
[<800083c4>] (do_DataAbort+0x34/0x9c) from [<8000deb4>] (__dabt_usr+0x34/0x40)
Exception stack(0x853effb0 to 0x853efff8)
ffa0:                                     767b7034 767b7134 767b7000 76c7cbdc
ffc0: 7ebc7650 7ebc7654 00000008 7ebc7640 00000000 00000000 767b7034 00000000
ffe0: 76c7cbc3 7ebc7628 76fe3551 76fe5372 800d0030 ffffffff
Mem-info:
DMA per-cpu:
CPU    0: hi:  186, btch:  31 usd:  24
active_anon:104052 inactive_anon:3 isolated_anon:0
 active_file:152 inactive_file:615 isolated_file:19
 unevictable:334 dirty:0 writeback:0 unstable:0
 free:662 slab_reclaimable:756 slab_unreclaimable:1631
 mapped:338 shmem:9 pagetables:423 bounce:0
 free_cma:616
DMA free:2860kB min:2636kB low:3292kB high:3952kB active_anon:416712kB inactive_anon:12kB active_file:748kB inactive_file:944kB unevictable:1336kB isolated(anon):0kB isolated(file):132kB present:523264kB managed:435032kB mlocked:0kB dirty:0kB writeback:0kB mapped:1312kB shmem:36kB slab_reclaimable:3024kB slab_unreclaimable:6524kB kernel_stack:904kB pagetables:1692kB unstable:0kB bounce:0kB free_cma:2492kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 587*4kB (EC) 24*8kB (UEC) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB 0*32768kB = 2540kB
362 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
131072 pages of RAM
889 free pages
5658 reserved pages
1918 slab pages
321 pages shared
0 pages swap cached
[ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
[  130]     0   130      655      150       4        0         -1000 ueventd
[  131]     0   131      654       59       4        0         -1000 watchdogd
[  142]  1036   142     2292      233       6        0         -1000 logd
[  143]     0   143      922       86       4        0         -1000 healthd
[  144]     0   144     1118       78       4        0         -1000 lmkd
[  145]  1000   145      812       67       3        0         -1000 servicemanager
[  146]     0   146     1933      163       5        0         -1000 vold
[  149]     0   149     1228      176       5        0         -1000 debuggerd
[  150]  1019   150     3239      260       8        0         -1000 drmserver
[  152]  1012   152      801       60       3        0         -1000 installd
[  153]  1017   153     1571      141       5        0         -1000 keystore
[  154]     0   154     1310      100       4        0         -1000 rild
[  157]  1023   157     2317       55       6        0         -1000 sdcard
[  159]  2000   159     1423       79       5        0         -1000 adbd
[  409]     0   409    30250    15637      43        0         -1000 main
[  449]     0   449    29226    14440      41        0         -1000 main
[  489]     0   489    29226    13917      42        0         -1000 main
[  531]     0   531    29226    13263      39        0         -1000 main
[  571]     0   571    29226    11503      36        0         -1000 main
[  612]     0   612    29226     8682      31        0         -1000 main
[  653]     0   653    29226     7539      29        0         -1000 main
[  692]     0   692    29226     6435      28        0         -1000 main
[  734]     0   734     9112     5292      20        0         -1000 dex2oat
[  772]     0   772     8593     5222      19        0         -1000 dex2oat
[  929]  1000   929     2699      179       6        0         -1000 surfaceflinger
[  931]     0   931     1103       92       4        0         -1000 netd
[  932]  1013   932     2142      198       5        0         -1000 mediaserver
[  933]     0   933     4671      170       7        0         -1000 app_process
[  941]     0   941      643       44       3        0         -1000 iptables
Kernel panic - not syncing: Out of memory and no killable processes...
CPU: 0 PID: 932 Comm: mediaserver Tainted: G        W    3.10.53-88204-g4f0dd92-dirty #7
[<80014404>] (unwind_backtrace+0x0/0xf8) from [<80011964>] (show_stack+0x10/0x14)
[<80011964>] (show_stack+0x10/0x14) from [<807ef26c>] (panic+0x8c/0x1e4)
[<807ef26c>] (panic+0x8c/0x1e4) from [<800ac1f8>] (out_of_memory+0x2a4/0x2d4)
[<800ac1f8>] (out_of_memory+0x2a4/0x2d4) from [<800b07bc>] (__alloc_pages_nodemask+0x848/0x870)
[<800b07bc>] (__alloc_pages_nodemask+0x848/0x870) from [<800c64f8>] (__pte_alloc+0x24/0x144)
[<800c64f8>] (__pte_alloc+0x24/0x144) from [<800c9414>] (handle_mm_fault+0xb0/0xc4)
[<800c9414>] (handle_mm_fault+0xb0/0xc4) from [<80019180>] (do_page_fault+0x240/0x37c)
[<80019180>] (do_page_fault+0x240/0x37c) from [<800083c4>] (do_DataAbort+0x34/0x9c)
[<800083c4>] (do_DataAbort+0x34/0x9c) from [<8000deb4>] (__dabt_usr+0x34/0x40)
Exception stack(0x853effb0 to 0x853efff8)
ffa0:                                     767b7034 767b7134 767b7000 76c7cbdc
ffc0: 7ebc7650 7ebc7654 00000008 7ebc7640 00000000 00000000 767b7034 00000000
ffe0: 76c7cbc3 7ebc7628 76fe3551 76fe5372 800d0030 ffffffff
Rebooting in 5 seconds..

Probably I am using a wrong init script (can you confirm that the above "main" processes are services initialized from init script?). However, I did not find in the documentation the proper lunch settings for an imx6solo android build. I am comparing the init scripts for dual/quad version with the ones provided for the soloX evk. But I see that the binary distribution files are avilable for both Solo and Dual/Quad Android versions. What kind of lunch settings could be required for IMX6SOLO board (I am recreating bootimage with kernel, dtb files and ramdisk)?

[ennebi · Accepted solution]

Main process in Android is the init script.

It was failing and restarting because of another bug shown in another post.

Init scripts have high priority and cannot be killed by Android memory management.

For those interested...