Larger Linux CMA Allocation?

[jared_sanson]

Hi all,

does anyone know whether it is possible to increase the CMA allocation size beyond 256MiB?

I have tried passing 'cma=384M' parameter to linux, but the kernel crashes:

Booting Linux on physical CPU 0x0
Linux version 3.14.44+g10c68cd (ad@ad-build) (gcc version 4.9.2 (GCC) ) #1 SMP Tue Aug 11 16:54:59 NZST 2015
CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
Machine model: SolidRun HummingBoard Solo/DualLite
cma: CMA: failed to reserve 384 MiB
Memory policy: Data cache writeback
CPU: All CPU(s) started in SVC mode.
PERCPU: Embedded 7 pages/cpu @dfbd8000 s8128 r8192 d12352 u32768
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 130048
Kernel command line: console=ttymxc0,115200 root=/dev/mmcblk0p2 rootwait rw
PID hash table entries: 2048 (order: 1, 8192 bytes)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 510972K/524288K available (5733K kernel code, 244K rwdata, 1912K rodata, 339K init, 392K bss, 13316K reserved)
Virtual kernel memory layout:
    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)
    vmalloc : 0xe0800000 - 0xff000000   ( 488 MB)
    lowmem  : 0xc0000000 - 0xe0000000   ( 512 MB)
    modules : 0xbf000000 - 0xc0000000   (  16 MB)
      .text : 0xc0008000 - 0xc077f7b4   (7646 kB)
      .init : 0xc0780000 - 0xc07d4fc0   ( 340 kB)
      .data : 0xc07d6000 - 0xc08132a0   ( 245 kB)
       .bss : 0xc08132a8 - 0xc08756a4   ( 393 kB)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
Hierarchical RCU implementation.
        RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=2.
RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
NR_IRQS:16 nr_irqs:16 16
L310 cache controller enabled
l2x0: 16 ways, CACHE_ID 0x410000c8, AUX_CTRL 0x32050000, Cache size: 512 kB
Switching to timer-based delay loop
sched_clock: 32 bits at 3000kHz, resolution 333ns, wraps every 1431655765682ns
Console: colour dummy device 80x30
Calibrating delay loop (skipped), value calculated using timer frequency.. 6.00 BogoMIPS (lpj=30000)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
CPU: Testing write buffer coherency: ok
CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Setting up static identity map for 0x10573420 - 0x10573478
CPU1: failed to boot: -38
Brought up 1 CPUs
SMP: Total of 1 processors activated (6.00 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
devtmpfs: initialized
VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 4
pinctrl core: initialized pinctrl subsystem
regulator-dummy: no parameters
NET: Registered protocol family 16
DMA: failed to allocate 131072 KiB pool for atomic coherent allocation
cpuidle: using governor ladder
cpuidle: using governor menu
CPU identified as i.MX6DL, silicon rev 1.1
Use WDOG1 as reset source
syscon 20c8000.anatop: regmap [mem 0x020c8000-0x020c8fff] registered
vdd1p1: 800 <--> 1375 mV at 1100 mV
vdd3p0: 2800 <--> 3150 mV at 3000 mV
vdd2p5: 2000 <--> 2750 mV at 2400 mV
vddarm: 725 <--> 1450 mV at 1150 mV
vddpu: 725 <--> 1450 mV at 1150 mV
vddsoc: 725 <--> 1450 mV at 1175 mV
syscon 20e0000.iomuxc-gpr: regmap [mem 0x020e0000-0x020e0037] registered
syscon 21bc000.ocotp-ctrl: regmap [mem 0x021bc000-0x021bffff] registered
hw-breakpoint: found 5 (+1 reserved) breakpoint and 1 watchpoint registers.
hw-breakpoint: maximum watchpoint size is 4 bytes.
imx6dl-pinctrl 20e0000.iomuxc: initialized IMX pinctrl driver
bio: create slab <bio-0> at 0
mxs-dma 110000.dma-apbh: initialized
brcm_osc_reg: 3300 mV
brcm_reg: 3300 mV
3P3V: 3300 mV
usb_h1_vbus: 5000 mV
usb_otg_vbus: 5000 mV
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
usbphy_nop1.11 supply vcc not found, using dummy regulator
usbphy_nop2.12 supply vcc not found, using dummy regulator
i2c i2c-0: IMX I2C adapter registered
i2c i2c-1: IMX I2C adapter registered
Linux video capture interface: v2.00
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
PTP clock support registered
imx-ipuv3 2400000.ipu: IPU DMFC NORMAL mode: 1(0~1), 5B(4,5), 5F(6,7)
mxc_mipi_csi2 21dc000.mipi_csi: i.MX MIPI CSI2 driver probed
mxc_mipi_csi2 21dc000.mipi_csi: i.MX MIPI CSI2 dphy version is 0x3130302a
MIPI CSI2 driver module loaded
Advanced Linux Sound Architecture Driver Initialized.
Switched to clocksource mxc_timer1
cfg80211: Calling CRDA to update world regulatory domain
NET: Registered protocol family 2
TCP established hash table entries: 4096 (order: 2, 16384 bytes)
TCP bind hash table entries: 4096 (order: 3, 32768 bytes)
TCP: Hash tables configured (established 4096 bind 4096)
TCP: reno registered
UDP hash table entries: 256 (order: 1, 8192 bytes)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
hw perfevents: enabled with ARMv7 Cortex-A9 PMU driver, 7 counters available
Bus freq driver module loaded
futex hash table entries: 512 (order: 3, 32768 bytes)
zbud: loaded
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
NFS: Registering the id_resolver key type
Key type id_resolver registered
Key type id_legacy registered
fuse init (API version 7.22)
msgmni has been set to 997
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
mxc_hdmi 20e0000.hdmi_video: Detected HDMI controller 0x13:0x1a:0xa0:0xc1
fbcvt: 1920x1080@60: CVT Name - 2.073M9
mxc_sdc_fb fb.23: registered mxc display driver hdmi
mxc_sdc_fb fb.23: Unable to allocate framebuffer memory
detected fb_set_par error, error code: -12
mxc_sdc_fb fb.23: Error fb_set_var ret:-12
mxc_sdc_fb: probe of fb.23 failed with error -12
imx-sdma 20ec000.sdma: no iram assigned, using external mem
imx-sdma 20ec000.sdma: initialisation failed with -12
imx-sdma: probe of 20ec000.sdma failed with error -12
Serial: IMX driver
2020000.serial: ttymxc0 at MMIO 0x2020000 (irq = 58, base_baud = 5000000) is a IMX
console [ttymxc0] enabled
21f0000.serial: ttymxc3 at MMIO 0x21f0000 (irq = 61, base_baud = 5000000) is a IMX
serial: Freescale lpuart driver
imx sema4 driver is registered.
[drm] Initialized drm 1.1.0 20060810
[drm] Initialized vivante 1.0.0 20120216 on minor 0
brd: module loaded
loop: module loaded
2188000.ethernet supply phy not found, using dummy regulator
pps pps0: new PPS source ptp0
fec: probe of 2188000.ethernet failed with error -12
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
usbcore: registered new interface driver usb-storage
2184800.usbmisc supply vbus-wakeup not found, using dummy regulator
ci_hdrc ci_hdrc.0: doesn't support gadget
ci_hdrc ci_hdrc.0: EHCI Host Controller
ci_hdrc ci_hdrc.0: new USB bus registered, assigned bus number 1
ci_hdrc ci_hdrc.0: can't setup: -12
ci_hdrc ci_hdrc.0: USB bus 1 deregistered
ci_hdrc ci_hdrc.0: can't start host role
(NULL device *): gadget not registered.
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
dma_pool_free ▒▒;܀▒+▒,   (null)/0 (bad dma)
Unable to handle kernel NULL pointer dereference at virtual address 00000023
pgd = c0004000
[00000023] *pgd=00000000
Internal error: Oops: 805 [#1] SMP ARM
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.14.44+g10c68cd #1
task: dc090000 ti: dc094000 task.ti: dc094000
PC is at dma_pool_destroy+0x30/0x15c
LR is at mutex_lock+0xc/0x58
pc : [<c00d5e20>]    lr : [<c0571b54>]    psr: 60000113
sp : dc095cb0  ip : 00000000  fp : 00000000
r10: ffffffff  r9 : dc10a810  r8 : dc3bfc40
r7 : 00000000  r6 : dc3d6810  r5 : dc3b8d00  r4 : dc03d010
r3 : 00200200  r2 : 00100100  r1 : ffffffff  r0 : 0000001f
Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel
Control: 10c5387d  Table: 10004059  DAC: 00000015
Process swapper/0 (pid: 1, stack limit = 0xdc094238)
Stack: (0xdc095cb0 to 0xdc096000)
5ca0:                                     c06aab34 dc3b8c9c 00000000 dc03d010
5cc0: fffffff4 dc3d6810 00000000 c03528b8 dc03d010 c034f360 dc3c0d38 c06cead4
5ce0: dc3c09d8 c0863e54 dc3d6810 c0863e54 dc3d6810 c0802218 fffffdfb 00000000
5d00: 00000000 c02cdec8 c02cde84 c0863e54 dc3d6810 00000000 c0802218 c02cc734
5d20: 00000000 dc3d6810 c02cc870 00000000 00000000 c02cad38 dc07c670 dc3b8938
5d40: dc3d6810 dc3d6810 dc3d6844 c02cc5f4 dc3d6818 dc3d6810 c07fbd90 c02cbc78
5d60: dc3d6818 dc3d6810 dc136810 c02ca100 dc3bfc40 c0034ce0 c0815340 dc3bfc40
5d80: c07e767c 00000002 dc3d6800 00000038 dc3d6810 00000000 dc3d6800 c02cdbb4
5da0: 02184000 dc095e1c dc136810 00000002 dc117500 c034f060 dc095e1c dc113a00
5dc0: dc1c3b10 dc136810 dc136800 c05b76e0 dfbf2bd4 ffffffff 00000000 c0355388
5de0: dc095df4 dc095e38 000000d3 c07c7b24 00000000 dfbf316c 00000001 00000000
5e00: dc136810 00000000 c08023e0 dc095e38 000000d3 c07c7b24 00000000 dc117540
5e20: 00000100 00000000 00000000 dc10a810 00000000 00003e84 00000003 c0354dc0
5e40: dc3bfb80 00000000 00000000 00001010 dc118af8 c0863e54 dc136810 c08023e0
5e60: fffffdfb 00000000 000000d3 c07c7b24 00000000 c02cdec8 c02cde84 c0863e54
5e80: dc136810 00000000 c08023e0 c02cc734 dc136810 c08023e0 dc136844 c07fbd90
5ea0: c08132c0 c02cc940 00000000 c08023e0 c02cc8b4 c02cac8c dc07c65c dc1191b4
5ec0: c08023e0 dc3b8b00 00000000 c02cbeec c06eb37c c07c7b1c c08023e0 c08023e0
5ee0: c07ae144 c07c7b1c c08132c0 c02ccf5c 00000000 dc094000 c07ae144 c000889c
5f00: dc011900 c06a4be8 dc0e1d80 c0576eec 00000000 00000000 000032c0 c01381fc
5f20: 00000000 c07e87e0 60000153 00000001 00000008 00000000 dfffcab4 c0049714
5f40: c06ebb00 00000006 00000006 dfffcab6 c07e87d0 c07d23e8 00000006 c07c7b1c
5f60: c08132c0 c08132c0 000000d3 c07c7b24 00000000 c0780c54 00000006 00000006
5f80: c0780514 00000000 00000000 c056c1e4 00000000 00000000 00000000 00000000
5fa0: 00000000 c056c1ec 00000000 c000e440 00000000 00000000 00000000 00000000
5fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
5fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
[<c00d5e20>] (dma_pool_destroy) from [<c03528b8>] (ci_hdrc_gadget_destroy+0x2c/0x38)
[<c03528b8>] (ci_hdrc_gadget_destroy) from [<c034f360>] (ci_hdrc_probe+0x208/0x5c4)
[<c034f360>] (ci_hdrc_probe) from [<c02cdec8>] (platform_drv_probe+0x44/0xa4)
[<c02cdec8>] (platform_drv_probe) from [<c02cc734>] (driver_probe_device+0x110/0x24c)
[<c02cc734>] (driver_probe_device) from [<c02cad38>] (bus_for_each_drv+0x64/0x98)
[<c02cad38>] (bus_for_each_drv) from [<c02cc5f4>] (device_attach+0x74/0x88)
[<c02cc5f4>] (device_attach) from [<c02cbc78>] (bus_probe_device+0x84/0xa8)
[<c02cbc78>] (bus_probe_device) from [<c02ca100>] (device_add+0x450/0x538)
[<c02ca100>] (device_add) from [<c02cdbb4>] (platform_device_add+0xb4/0x21c)
[<c02cdbb4>] (platform_device_add) from [<c034f060>] (ci_hdrc_add_device+0x1a8/0x1e8)
[<c034f060>] (ci_hdrc_add_device) from [<c0355388>] (ci_hdrc_imx_probe+0x4cc/0x708)
[<c0355388>] (ci_hdrc_imx_probe) from [<c02cdec8>] (platform_drv_probe+0x44/0xa4)
[<c02cdec8>] (platform_drv_probe) from [<c02cc734>] (driver_probe_device+0x110/0x24c)
[<c02cc734>] (driver_probe_device) from [<c02cc940>] (__driver_attach+0x8c/0x90)
[<c02cc940>] (__driver_attach) from [<c02cac8c>] (bus_for_each_dev+0x6c/0xa0)
[<c02cac8c>] (bus_for_each_dev) from [<c02cbeec>] (bus_add_driver+0x148/0x1f0)
[<c02cbeec>] (bus_add_driver) from [<c02ccf5c>] (driver_register+0x78/0xf8)
[<c02ccf5c>] (driver_register) from [<c000889c>] (do_one_initcall+0xf8/0x154)
[<c000889c>] (do_one_initcall) from [<c0780c54>] (kernel_init_freeable+0x138/0x1d8)
[<c0780c54>] (kernel_init_freeable) from [<c056c1ec>] (kernel_init+0x8/0xe8)
[<c056c1ec>] (kernel_init) from [<c000e440>] (ret_from_fork+0x14/0x34)
Code: e3a02c01 e3a03c02 e3402010 e3403020 (e5801004)
---[ end trace 0b4d159260ffe941 ]---
Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
random: noblocking pool is initialized

I have noticed the following:

1. "cma: CMA: failed to reserve 384 MiB"

2. "DMA: failed to allocate 131072 KiB pool for atomic coherent allocation"

3. A crash in ci_hdrc (probably a result of the above two errors?)

This occurs for any value above 256MiB.
Is it possible to set CMA higher than 256MiB? Is there a limit?

We need this memory for video capture and GPU processing.

Any help would be appreciated!

[igorpadykov]

Hi Jared

in general one can try to decrease memory used by vpu/gpu

Re: Re: Memory Reserved on i.MX6 for VPU

however with 512MB memory it may be problematic to allocate such size.

Best regards

igor

-----------------------------------------------------------------------------------------------------------------------

Note: If this post answers your question, please click the Correct Answer button. Thank you!

-----------------------------------------------------------------------------------------------------------------------

[jared_sanson]

Thanks,

We've found a way to use less memory and don't need to increase CMA now.

We need both V4L2 buffers (mxc_v4l2_capture.ko) and a framebuffer (vfb_phymem.ko), but calls to dma_alloc_coherent were failing. I think this was due to fragmentation of the DMA pool, not a lack of memory. I set up a separate dma pool to use by replacing GFP_KERNEL with GFP_ATOMIC in the above modules, forcing them to use the atomic pool. (although I'm not sure if this is ideal)

If you have any suggestions on how we could avoid fragmented DMA memory or set up a dedicated DMA pool please let me know

[vikaspatil]

Hi Jared Sanson,

If possible could you share what was your solution to use the less memory and where you changed GFP_KERNEL to GFP_ATOMIC?

Thanks & Regards,

Vikash