microSD card write hangs on P4080

[yogesh_ahiray]

Hi All,

We are using Kernel 3.0.34 ( from SDK 1.2) on our  P4080 board. We are using microSD card and have our filesystem on it.

The Write operation on SD card randomly hangs and eventually Kernel causes following exception, The same can be reproduce if we write continuously on uSD card with dd command.

INFO: task dd:2184 blocked for more than 120 seconds.

"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

dd              D 0ff1d604     0  2184   2163 0x00000000

Call Trace:

[c7401940] [00000003] 0x3 (unreliable)

[c7401a00] [c0007654] __switch_to+0x94/0xc0

[c7401a20] [c0607068] __schedule+0x268/0x610

[c7401b50] [c0607860] schedule+0x30/0xd0

[c7401b60] [c01ac8d4] do_get_write_access+0x2c4/0x470

[c7401bd0] [c01acc00] jbd2_journal_get_write_access+0x30/0x60

[c7401be0] [c0191658] __ext4_journal_get_write_access+0x38/0xa0

[c7401c00] [c0174010] ext4_reserve_inode_write+0x90/0xc0

[c7401c20] [c0174068] ext4_mark_inode_dirty+0x28/0x1b0

[c7401c50] [c0179718] ext4_dirty_inode+0x38/0x70

[c7401c60] [c010ca3c] __mark_inode_dirty+0x3c/0x220

[c7401c80] [c0117294] generic_write_end+0xe4/0x110

[c7401ca0] [c0177ad4] ext4_da_write_end+0x1a4/0x370

[c7401ce0] [c00a637c] generic_file_buffered_write+0xdc/0x230

[c7401d50] [c00a6ef8] __generic_file_aio_write+0x238/0x4d0

[c7401dc0] [c00a7218] generic_file_aio_write+0x88/0x150

[c7401e10] [c016cff4] ext4_file_write+0x134/0x330

[c7401e60] [c00e2c68] do_sync_write+0xa8/0x130

[c7401ef0] [c00e35a8] vfs_write+0xa8/0x1a0

[c7401f10] [c00e391c] sys_write+0x4c/0x90

[c7401f40] [c000f340] ret_from_syscall+0x0/0x3c

--- Exception: c01 at 0xff1d604

    LR = 0x100044c8

Kernel panic - not syncing: hung_task: blocked tasks

Call Trace:

[e6743ea0] [c0007d64] show_stack+0x44/0x160 (unreliable)

[e6743ee0] [c060a4d0] panic+0xac/0x1ec

[e6743f40] [c008f254] watchdog+0x254/0x270

[e6743fa0] [c0065fdc] kthread+0x7c/0x80

[e6743ff0] [c000e208] kernel_thread+0x4c/0x68

More Information::::

After looking into uSD card messages ( enabled SD/MMC debug logs) the SD card continuously responds with following messages and never completes its write operation,

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

sdhci [sdhci_irq()]: *** mmc0 got interrupt: 0x00000001

mmc0: req done (CMD13): 0: 00000c00 00000000 00000000 00000000

mmc0: starting CMD13 arg aaaa0000 flags 00000195

I am using Kingston 8GB and tried other cards like Toshiba, Lexar etc they all fails at some point of time.

Any help will be highly appreciated, this is causing us major block now.

Thanks

Yogesh

[yipingwang]

There is a similar know issue in SDK 1.2 release, a call trace can occur when test SD performance with ext4 filesystem, please refer to the following workaround.

1. Use ext2 filesystem.

2. Increase the timeout value (in kernel configuration) and decrease the dirty_ratio in proc file system.

menuconfig:

Kernel hacking

(xxx) Default timeout for hung task detection (in seconds)

proce file system:

echo xx > /proc/sys/vm/dirty_ratio

echo xx > /proc/sys/vm/dirty_background_ratio

In addition you could update the software to SDK 1.4(Kernel 3.8), if you would like to.

[yogesh_ahiray]

Hi Yiping,

Thank you for the reply. I tried the above workarounds but no help. Once the card entered in that state it is not coming out of that, even I reboot the board. Only power cycle brings card out of that state. Can you please explain what is the exact problem here.

Thanks

Yogesh

[yipingwang]

Please refer to the know issue of SDK 1.2 release, this problem can be found in 64 bit platform more frequently,

this workaround has been verified by the testing team, it should work. In addition, this problem has been fixed in SDK 1.4 release.

SDK-227665

A call trace can occur when running "iozone" with ext4 filesystem to test SD performance on P5040DS 64-bit.

Use ext2 filesystem. Increase the timeout value (in kernel configuration) and decrease the dirty_ratio in proc file system. menuconfig:Kernel hacking(xxx) Default timeout for hung task detection (in seconds) proce file system:echo xx > /proc/sys/vm/dirty_ratio echo xx > /proc/sys/vm/dirty_background_ratio