imx7d crash on sdma_int_handler kernel

[864535720]

Hello everyone
my development board is IMX7DSABRE. Linux kernel version is 4.9.88
During the use, our kernel crashed, and the crash information is as follows:
Unable to handle kernel NULL pointer dereference at virtual address 00000002
pgd = a9b7c000
[00000002] * pgd = a8da4835, * pte = 00000000, * ppte = 00000000
Internal error: Oops: 17 [# 1] PREEMPT SMP ARM
Modules linked in:
CPU: 0 PID: 3144 Comm: KMonitor Not tainted 4.9.88 # 205
Hardware name: Freescale i.MX7 Dual (Device Tree)
task: a84fa680 task.stack: a8cf0000
PC is at sdma_int_handler + 0xd8 / 0x2f8
LR is at _raw_spin_lock + 0x10 / 0x50
pc: [<80450e0c>] lr: [<80923160>] psr: 60030193
sp: a8cf1ec0 ip: a987d0bc fp: 00000001
r10: a8394010 r9: a87a5300 r8: 00000000
r7: 00000000 r6: a8394dd0 r5: 0000000d r4: a8394d78
r3: 00000000 r2: 00000000 r1: 00000000 r0: 00000001
Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment none
Control: 10c53c7d Table: a9b7c06a DAC: 00000051
Process KMonitor (pid: 3144, stack limit = 0xa8cf0210)
Stack: (0xa8cf1ec0 to 0xa8cf2000)
1ec0: 00000001 a8394dd0 7eaed7e0 00002000 00000000 00000000 a8cf0000 a8353740
1ee0: a814e800 00000000 a8cf1f2c 00000040 a814e800 810903b8 00feb1f8 8017136c
1f00: a8cf1f4c 80923480 a814e800 a814e800 8100a880 00000000 00000001 a8cf1fb0
1f20: a8008000 80171418 00000000 00000000 00feb1f8 a814e800 a814e860 8017148c
1f40: a814e800 a814e860 8100a880 80174804 80174734 80f5bd80 00000000 00000040
1f60: 00000000 8017065c 00000130 80170b64 8101c000 810033f4 c080200c a8cf1fb0
1f80: c0802000 c0803000 7eaeccd8 801014c4 7eaecd74 76ce0678 60030010 ffffffff
1fa0: 10c53c7d 10c53c7d 7eaecce8 8010c0f0 7eaecd74 00000000 00000000 00000000
1fc0: 00feb440 7eaecd74 00feb448 7eaeccf8 7eaeccb0 7eaecce8 7eaeccd8 00feb1f8
1fe0: 00000000 7eaecc90 7eaecc64 76ce0678 600 30010 ffffffff 00000000 00000000
[<80450e0c>] (sdma_int_handler) from [<8017136c>] (__handle_irq_event_percpu + 0x88 / 0x118)
[<8017136c>] (__handle_irq_event_percpu) from [<80171418>] (handle_irq_event_percpu + 0x1c / 0x58)
[<80171418>] (handle_irq_event_percpu) from [<8017148c>] (handle_irq_event + 0x38 / 0x5c)
[<8017148c>] (handle_irq_event) from [<80174804>] (handle_fasteoi_irq + 0xd0 / 0x1a8)
[<80174804>] (handle_fasteoi_irq) from [<8017065c>] (generic_handle_irq + 0x24 / 0x34)
[<8017065c>] (generic_handle_irq) from [<80170b64>] (__handle_domain_irq + 0x7c / 0xec)
[<80170b64>] (__handle_domain_irq) from [<801014c4>] (gic_handle_irq + 0x48 / 0x8c)
[<801014c4>] (gic_handle_irq) from [<8010c0f0>] (__irq_usr + 0x50 / 0x80)
Exception stack (0xa8cf1fb0 to 0xa8cf1ff8)
1fa0: 7eaecd74 00000000 00000000 00000000
1fc0: 00feb440 7eaecd74 00feb448 7eaeccf8 7eaeccb0 7eaecce8 7eaeccd8 00feb1f8
1fe0: 00000000 7eaecc90 7eaecc64 76ce0678 600 30010 ffffffff
Code: e5992044 e5993050 e0822082 e0833102 (e5d32002)
--- [end trace e60c122438210011] ---
Kernel panic-not syncing: Fatal exception in interrupt
CPU1: stopping
CPU: 1 PID: 0 Comm: swapper / 1 Tainted: G D 4.9.88 # 205
Hardware name: Freescale i.MX7 Dual (Device Tree)
[<8010eb84>] (unwind_backtrace) from [<8010b328>] (show_stack + 0x10 / 0x14)
[<8010b328>] (show_stack) from [<803c00c8>] (dump_stack + 0x88 / 0x9c)
[<803c00c8>] (dump_stack) from [<8010d9cc>] (handle_IPI + 0x198 / 0x1ac)
[<8010d9cc>] (handle_IPI) from [<80101504>] (gic_handle_irq + 0x88 / 0x8c)
[<80101504>] (gic_handle_irq) from [<8010bd0c>] (__irq_svc + 0x6c / 0xa8)
Exception stack (0xa80adf58 to 0xa80adfa0)
df40: 00000001 80c22038
df60: 00000001 000000f5 00000000 ab730e70 1ada9825 00008ef5 1ada3506 00008ef5
df80: 80f5ce68 81003144 00000016 a80adfa8 801675f4 806926ac 20000013 ffffffff
[<8010bd0c>] (__irq_svc) from [<806926ac>] (cpuidle_enter_state + 0x11c / 0x264)
[<806926ac>] (cpuidle_enter_state) from [<8016781c>] (cpu_startup_entry + 0x148 / 0x21c)
[<8016781c>] (cpu_startup_entry) from [<801015ac>] (__enable_mmu + 0x0 / 0x14)
--- [end Kernel panic-not syncing: Fatal exception in interrupt

Are there friends who have the same problem as me.
Looking forward to your help

[dp_yanam]

Hi,

I also facing same issue in imx6 custom board with same kernel version 4.9.88.is the issue was resolved? If not please tell me how to debug these kind of issues.

Thanks and regards

K.durgaprasad

[864535720]

hello
We have not completely solved this problem at present. After investigation, we preliminarily judged that the problem may be related to the audio driver. At present, we are only doubtful and unsure. We can keep the information shared and see how to solve this problem.

[dp_yanam]

Hi, 

Thanks for your reply.

In my case, i using dummy codec driver contains only ssi clock configurations for interfacing imx6 with 24 bit ADC. the issue may be related sdma driver(imx-sdma.c). If you have any information on debugging these kind of issues please let me know. I too can try debug same kind of issue & let you know.

Regards,

K. durgaprasad.

[864535720]

Hello there
About the problem of sdma_int_handler I seem to find the problem.
You try the following：

unsigned long stattemp=0;
struct sdma_engine *sdmatemp = NULL;
static irqreturn_t sdma_int_handler(int irq, void *dev_id)
{
struct sdma_engine *sdma = dev_id;
unsigned long stat;
sdmatemp = sdma;
stat = readl_relaxed(sdma->regs + SDMA_H_INTR);
writel_relaxed(stat, sdma->regs + SDMA_H_INTR);
/* channel 0 is special and not handled here, see run_channel0() */
stat &= ~1;
stattemp = stat;

while (stat) {
int channel = fls(stat) - 1;
struct sdma_channel *sdmac = &sdma->channel[channel];
struct sdma_desc *desc;

spin_lock(&sdmac->vc.lock);
desc = sdmac->desc;
if (desc) {
if (sdmac->flags & IMX_DMA_SG_LOOP) {
if (sdmac->peripheral_type != IMX_DMATYPE_HDMI)
sdma_update_channel_loop(sdmac);
else
vchan_cyclic_callback(&desc->vd);
} else {
mxc_sdma_handle_channel_normal(sdmac);
vchan_cookie_complete(&desc->vd);
if (!list_empty(&sdmac->pending))
list_del(&desc->node);
sdma_start_desc(sdmac);
}
}
__clear_bit(channel, &stat);
stattemp = stat;
spin_unlock(&sdmac->vc.lock);
}

return IRQ_HANDLED;
}

static void sdma_update_channel_loop(struct sdma_channel *sdmac)
{
struct sdma_buffer_descriptor *bd;
struct sdma_desc *desc = sdmac->desc;
int error = 0;
enum dma_status old_status = sdmac->status;

/*
* loop mode. Iterate over descriptors, re-setup them and
* call callback function.
*/
while (desc) {
bd = &desc->bd[desc->buf_tail];

if (bd->mode.status & BD_DONE)
break;

if (bd->mode.status & BD_RROR) {
bd->mode.status &= ~BD_RROR;
sdmac->status = DMA_ERROR;
error = -EIO;
}

/*
* We use bd->mode.count to calculate the residue, since contains
* the number of bytes present in the current buffer descriptor.
*/

sdmac->chn_real_count = bd->mode.count;
bd->mode.status |= BD_DONE;
bd->mode.count = sdmac->period_len;
desc->buf_ptail = desc->buf_tail;
desc->buf_tail = (desc->buf_tail + 1) % desc->num_bd;

if (error)
sdmac->status = old_status;
/*
* The callback is called from the interrupt context in order
* to reduce latency and to avoid the risk of altering the
* SDMA transaction status by the time the client tasklet is
* executed.
*/
spin_unlock(&sdmac->vc.lock);
printk("%s %s %d\n", __FILE__, __func__, __LINE__);
dmaengine_desc_get_callback_invoke(&desc->vd.tx, NULL);
printk("%s %s %d\n", __FILE__, __func__, __LINE__);
spin_lock(&sdmac->vc.lock);
desc = sdmac->desc;
}
}

static void sdma_desc_free(struct virt_dma_desc *vd)
{
// dump_stack();

int channel = fls(stattemp) - 1;
struct sdma_channel *sdmac = &sdmatemp->channel[channel];
printk("%s %s %d\n", __FILE__, __func__, __LINE__);
struct sdma_desc *desc = container_of(vd, struct sdma_desc, vd);
if (desc) {
printk("%s %s %d\n", __FILE__, __func__, __LINE__);
sdma_free_bd(desc);
printk("%s %s %d\n", __FILE__, __func__, __LINE__);
udelay(1000);
kfree(desc);
sdmac->desc = NULL;
}
}

[igorpadykov]

Hi 其东 刘 

reason may be wrong image, please try

Linux Binary Demo Files - i.MX 6UltraLite, i.MX 6ULL, i.MX 7Dual

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[864535720]

Hi igorpadykov

   I downloaded the source code of Linux4.9.88 through yocto. The image generated by compiling the source code.
Then our chip specific model is: MCIMX7D7DVM10SD. The manual explains that its frequency is 1Ghz, but I get 1.2GHz through the command cat / sys / devices / system / cpu / cpu0 / cpufreq / cpuinfo_cur_freq

[igorpadykov]

please check Linux 4.9.88_2.0.0 BSP & Multimedia Codecs Documentation

and generate image for imx7dsabresd, as described in Yocto Guide

Best regards
igor

[864535720]

Hi igorpadykov

I have downloaded the source code through yocto. And compile and burn according to the manual. Later, during a long run, an sdma_int_handler error occurred. I want to know what is causing this error and how to resolve it.

[igorpadykov]

is it custom board or i.MX7D Sabre SD board ?

If i.MX7D Sabre SD board please try Linux Binary Demo Files - i.MX 6UltraLite, i.MX 6ULL, i.MX 7Dual

[864535720]

Hi igorpadykov

Both the custom board and the i.MX7D Saber SD board have sdma_int_handler issues. I can't use the image you gave me directly because we need to add and modify some features.

[igorpadykov]

had you tried  i.MX7D Sabre SD board with  Linux Binary Demo Files - i.MX 6UltraLite, i.MX 6ULL, i.MX   ?

[864535720]

Hi igorpadykov

I didn't try this you mentioned. I just want to know what caused the sdma_int_handler error and how to resolve it.