Custom IMX8MP Board GPU STATE DUMP / DMA Appears to be Stuck

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Custom IMX8MP Board GPU STATE DUMP / DMA Appears to be Stuck

Jump to solution
3,843 Views
Wobaffet
Senior Contributor I

Hello, We are working on our custom board with imx8mp mpu. We have 512MB LPDDR4 on our board. We have tested our RAM, patched our calibration code and built an image with 6.1.36 kernel.

On boot we are facing with register dump from GPU starting with this lines of console dumps stating DMA appears to be stuck at this address:

Welcome to NXP i.MX Release Distro 6.1-mickledore (mickledore)!

[    3.059815] systemd[1]: Hostname set to <imx8mp-lpddr4-evk>.
[    3.068457] systemd[1]: Initializing machine ID from random generator.
[    3.089195] GPU[0](ChipModel=0x7000 ChipRevision=0x6204):
[    3.095032] **************************
[    3.098794] ***   GPU STATE DUMP   ***
[    3.102549] **************************
[    3.106296]   axi      = 0x0000000F
[    3.109788]   idle     = 0x7FFFFFFE
[    3.113279]     FE not idle
[    3.116076]   AQ_HI_CLOCK_CONTROL  = 0x00040100
[    3.120610]   DMA appears to be stuck at this address:
[    3.125749]     0x97FC36E0
[    3.128475]   dmaLow   = 0xFFFFBFBF
[    3.131968]   dmaHigh  = 0xFFFEFFFE
[    3.135465]   dmaState = 0x00000900
[    3.138957]     command state       = 0 (PAR_IDLE_ST)
[    3.144012]     command DMA state   = 1 (CMD_START_ST)
[    3.149161]     command fetch state = 2 (FET_VALID_ST)
[    3.154304]     DMA request state   = 0 (REQ_IDLE_ST)
[    3.159386]     cal state           = 0 (CAL_IDLE_ST)
[    3.164463]     VE request state    = 0 (VER_IDLE_ST)
[    3.169540]   Debug registers:
[    3.172622]     RA[0] debug registers:

We've configured CMA for our board for about 100MiB, is this enough? Could this be causing the problem?

What else could be causing the problem? 

Full log is attached.

Thank you in advance!

Best Regards!

0 Kudos
1 Solution
3,441 Views
Dhruvit
NXP TechSupport
NXP TechSupport

Hi @Wobaffet,

I hope you are doing well.

Please try to enable the below node in the device tree.

&ldb_phy {
    status = "okay";
};

Thanks & Regards,
Dhruvit Vasavada

View solution in original post

0 Kudos
11 Replies
3,818 Views
Dhruvit
NXP TechSupport
NXP TechSupport

Hi @Wobaffet,

I hope you are doing well.

Please try increasing CMA size to 25 % of the Total DDR size.

Please make sure that you have also changed the address range in the gpu_reserved node in /arch/arm64/boot/dts/freescale/imx8mp.dtsi

as the default range will fall out of range in 512MB DRAM.

Thanks & Regards,
Dhruvit Vasavada

0 Kudos
3,741 Views
Wobaffet
Senior Contributor I

Hello Dhruvit,

Thank you for your reply,

I've made the following changes and like you've mentioned the dump is fixed but I think there are still some issues. Because I get the following log on boot-up and my display is still not working. Might be related to below issues.

 

 

 

[    8.995052] galcore: retry probe(nxp platform).
[    1.333584] of_reserved_mem_lookup() returned NULL
[   19.207576] platform 40000000.mix_gpu_ml: deferred probe pending
        resmem: reserved-memory {
                #address-cells = <2>;
                #size-cells = <2>;
                ranges;

                ocram: ocram@900000 {
                        no-map;
                        reg = <0 0x900000 0 0x70000>;
                };

/*
 *              Memory reserved for optee usage. Please do not use.
 *              This will be automaticky added to dtb if OP-TEE is installed.
 *              optee@56000000 {
 *                      reg = <0 0x56000000 0 0x2000000>;
 *                      no-map;
 *              };
 */
                /* global autoconfigured region for contiguous allocations */
                linux,cma {
                        compatible = "shared-dma-pool";
                        reusable;
                        size = <0 0x7270E00>;
                        alloc-ranges = <0 0x40000000 0 0x47270E00>;
                        linux,cma-default;
                };

                gpu_reserved: gpu_reserved@47270E00 {
                        no-map;
                        reg = <0x0 0x47270E00 0 0x04000000>;
                };

                dsp_reserved: dsp@4B270E00 {
                        reg = <0 0x4B270E00 0 0x01000000>;
                        no-map;
                };
                dsp_reserved_heap: dsp_reserved_heap@4C270E00 {
                        reg = <0 0x4C270E00 0 0xef0000>;
                        no-map;
                };
                dsp_vdev0vring0: vdev0vring0@4D160E00 {
                        reg = <0 0x4D160E00 0 0x8000>;
                        no-map;
                };
                dsp_vdev0vring1: vdev0vring1@4D168E00 {
                        reg = <0 0x4D168E00 0 0x8000>;
                        no-map;
                };
                dsp_vdev0buffer: vdev0buffer@4D170E00 {
                        compatible = "shared-dma-pool";
                        reg = <0 0x4D170E00 0 0x100000>;
                        no-map;
                };
        };        
mix_gpu_ml: mix_gpu_ml {
                compatible = "fsl,imx8mp-gpu", "fsl,imx8-gpu-ss";
                cores = <&gpu_3d>, <&ml_vipsi>, <&gpu_2d>;
                reg = <0x0 0x40000000 0x0 0x20000000>, <0x0 0xD270E00 0x0 0x4000000>;
                reg-names = "phys_baseaddr", "contiguous_mem";
                memory-region=<&gpu_reserved>;
                status = "disabled";
        };

 

 

 

Can you guess the faulty area?

Thank you!

Best Regards,

Gencay Akyüz

Tags (1)
0 Kudos
3,663 Views
Dhruvit
NXP TechSupport
NXP TechSupport

Hi @Wobaffet,

Please make sure that you have the mix_gpu_ml node enabled in your dts file.

& recommended GPU reserved size is 128 MB (0x8000000).

Thanks & Regards,
Dhruvit Vasavada

0 Kudos
3,638 Views
Wobaffet
Senior Contributor I

Hello,

Thank you for your reply. It seems that logs disappeared on the boot with some minor changes on the configuration, however on power-off, I see the below logs and I can't seem to reach any display-related modules. Here are my logs for modetest and related dmesg lines:

[    0.043459] platform 32e90000.lcd-controller: Fixed dependency cycle(s) with /soc@0/bus@32c00000/ldb@32ec005c
[    0.049636] platform 32c00000.bus:ldb@32ec005c: Fixed dependency cycle(s) with /panel0
[   12.774004] imx-drm display-subsystem: bound imx-lcdifv3-crtc.0 (ops lcdifv3_crtc_ops)
[   12.783599] platform imx-lcdifv3-crtc.0: deferred probe pending
root@imx8mp-lpddr4-evk:/# modetest -c
trying to open device 'i915'...failed
trying to open device 'amdgpu'...failed
trying to open device 'radeon'...failed
trying to open device 'nouveau'...failed
trying to open device 'vmwgfx'...failed
trying to open device 'omapdrm'...failed
trying to open device 'exynos'...failed
trying to open device 'tilcdc'...failed
trying to open device 'msm'...failed
trying to open device 'sti'...failed
trying to open device 'tegra'...failed
trying to open device 'imx-drm'...failed
trying to open device 'rockchip'...failed
trying to open device 'atmel-hlcdc'...failed
trying to open device 'fsl-dcu-drm'...failed
trying to open device 'vc4'...failed
trying to open device 'virtio_gpu'...failed
trying to open device 'mediatek'...failed
trying to open device 'meson'...failed
trying to open device 'pl111'...failed
trying to open device 'stm'...failed
trying to open device 'sun4i-drm'...failed
trying to open device 'armada-drm'...failed
trying to open device 'komeda'...failed
trying to open device 'imx-dcss'...failed
trying to open device 'mxsfb-drm'...failed
trying to open device 'simpledrm'...failed
trying to open device 'imx-lcdif'...failed
trying to open device 'imx-dcnano'...failed
no device found

Log for the poweroff: 

[ 1231.039157] systemd-shutdown[1]: Powering off.
[ 1231.043622] kvm: exiting hardware virtualization
[ 1231.074100] ------------[ cut here ]------------
[ 1231.078721] DMA-API: galcore 40000000.mix_gpu_ml: device driver frees DMA memory with different direction [device address=0x000000004d681000] [size=4096 bytes] [mapped with DMA_BIDIRECTIONAL] [unmapped with DMA_FROM_DEVICE]
[ 1231.098540] WARNING: CPU: 2 PID: 1 at kernel/dma/debug.c:1010 check_unmap+0x260/0x8fc
[ 1231.106381] Modules linked in:
[ 1231.109437] CPU: 2 PID: 1 Comm: systemd-shutdow Not tainted 6.1.36+g0eaea3f4cf21 #1
[ 1231.117094] Hardware name: NXP i.MX8MPlus EVK board (DT)

We are using an LVDS panel directly connected to the LVDS interface of the MPU. What could cause these errors? 

Thank you in advance!

0 Kudos
3,599 Views
Dhruvit
NXP TechSupport
NXP TechSupport

Hi @Wobaffet,

I hope you are doing well.

There seems to be some issue with kernel configuration or dts.

Please provide me with the full device tree, boot logs & kernel .config for further debugging.

Thanks & Regards,
Dhruvit Vasavada

0 Kudos
3,588 Views
Wobaffet
Senior Contributor I

Hello Dhruvit,

I've provided the kernel dts, dtsi, current boot log and the config file that I've patched.

Thank you in advance!

Best Regards.

0 Kudos
3,547 Views
Dhruvit
NXP TechSupport
NXP TechSupport

Hi @Wobaffet,

I hope you are doing well.

Please make sure that the below configurations are available in .config file.

CONFIG_IMX_LCDIFV3_CORE=y
CONFIG_IMX_DPU_CORE=y
CONFIG_DRM=y
CONFIG_DRM_IMX=y
CONFIG_DRM_IMX8MP_LDB=y

Thanks & Regards,
Dhruvit Vasavada

0 Kudos
3,467 Views
Wobaffet
Senior Contributor I

Hello Dhruvit, 

I've updated the config file but nothing is changed for my scenario.

CONFIG_TOUCHSCREEN_ILI210X=y
CONFIG_IMX_LCDIFV3_CORE=y
CONFIG_DRM_PANEL_LVDS=y
CONFIG_DRM_FSL_LDB=y
CONFIG_DRM_IMX_LCDIF=y
CONFIG_CRC_CCITT=y
CONFIG_IMX_SDMA=m
CONFIG_DRM=y
CONFIG_MXC_GPU_VIV=y
CONFIG_DRM_IMX=y
CONFIG_DRM_IMX8MP_LDB=y
CONFIG_IMX_DPU_CORE=y
#DMA DEBUG 
CONFIG_CMA_DEBUG=y
CONFIG_DMA_API_DEBUG=y
CONFIG_DMA_API_DEBUG_SG=y
#DMA SIZE CONFIG
CONFIG_CMA_SIZE_MBYTES=120
#TESTCASES
DRM_IMX_LCDIFV3=y
IMX_LCDIFV3_CORE=y

Any other suggestions?

Thank you in advance!

Best Regards

 

0 Kudos
3,442 Views
Dhruvit
NXP TechSupport
NXP TechSupport

Hi @Wobaffet,

I hope you are doing well.

Please try to enable the below node in the device tree.

&ldb_phy {
    status = "okay";
};

Thanks & Regards,
Dhruvit Vasavada

0 Kudos
3,367 Views
Wobaffet
Senior Contributor I

Hello Dhruvit,

Thank you for the configs and all the other help. LVDS display currently running.

But During boot up, the following log persists. Should CMA size > All the reserved mem combined? Could that be the reason for the warning or they are two different things? Thank you in advance!

[    9.525002] cma: cma_alloc(cma (____ptrval____), count 1000, align 8)
[    9.536858] cma: cma_alloc(): returned (____ptrval____)
[    9.537971] ------------[ cut here ]------------
[    9.542616] DMA-API:  (null): mapping sg segment longer than device claims to support [len=4096000] [max=65536]
[    9.552743] WARNING: CPU: 1 PID: 422 at kernel/dma/debug.c:1160 debug_dma_map_sg+0x2b0/0x350
[    9.561204] Modules linked in:
[    9.564260] CPU: 1 PID: 422 Comm: weston Not tainted 6.1.36+gae79fbd409ed #1
[    9.571309] Hardware name: NXP i.MX8MPlus EVK board (DT)
[    9.576618] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[    9.583578] pc : debug_dma_map_sg+0x2b0/0x350
[    9.587938] lr : debug_dma_map_sg+0x2b0/0x350
[    9.592298] sp : ffff80000ab03910
[    9.595609] x29: ffff80000ab03910 x28: 0000000000010000 x27: ffff00000c2b8300
[    9.602748] x26: ffffffffffffffff x25: ffff000010fdec00 x24: 0000000000000000
[    9.609885] x23: ffff800009d58be8 x22: 0000000000000001 x21: 0000000000000001
[    9.617023] x20: 0000000000000000 x19: ffff80000ab03ad0 x18: 0000000000000006
[    9.624160] x17: 7420736d69616c63 x16: 2065636976656420 x15: 6e61687420726567
[    9.631298] x14: 6e6f6c20746e656d x13: ffff800009d72588 x12: 0000000000000501
[    9.638435] x11: 00000000000001ab x10: ffff800009dca588 x9 : ffff800009d72588
[    9.645572] x8 : 00000000ffffefff x7 : ffff800009dca588 x6 : 80000000fffff000
[    9.652709] x5 : ffff000015ebaa18 x4 : ffff000015ebaa18 x3 : ffff000015ebdc20
[    9.659846] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff000010dc2c40
[    9.666984] Call trace:
[    9.669428]  debug_dma_map_sg+0x2b0/0x350
[    9.673439]  __dma_map_sg_attrs+0xa0/0x110
[    9.677536]  dma_map_sgtable+0x28/0x44
[    9.681284]  drm_gem_map_dma_buf+0x5c/0xd4
[    9.685381]  __map_dma_buf+0x2c/0xa0
[    9.688957]  dma_buf_map_attachment+0xa0/0x130
[    9.693402]  dma_buf_ioctl+0x27c/0x650
[    9.697152]  __arm64_sys_ioctl+0xac/0xf0
[    9.701077]  invoke_syscall+0x48/0x114
[    9.704828]  el0_svc_common.constprop.0+0xcc/0xec
[    9.709532]  do_el0_svc+0x2c/0xd0
[    9.712848]  el0_svc+0x2c/0x84
[    9.715902]  el0t_64_sync_handler+0xf4/0x120
[    9.720172]  el0t_64_sync+0x18c/0x190
[    9.723835] ---[ end trace 0000000000000000 ]---
[    9.728599] cma: cma_alloc(cma (____ptrval____), count 1000, align 8)
[    9.738238] cma: cma_alloc(): returned (____ptrval____)
[    9.742690] cma: cma_alloc(cma (____ptrval____), count 1000, align 8)
[    9.750538] cma: cma_alloc(): returned (____ptrval____)

 

0 Kudos
3,628 Views
Wobaffet
Senior Contributor I

my latest configuration to related parts are attached.

0 Kudos