Hung task exception using dm-crypt

[fridgemagnet]

Using SDK-V2.0-20160527 with a T1043D4RDB, I have set up an encrypted partition on a SATA drive using cryptsetup. However attempting to write a file of more than about 40MBytes generates a hung task exception and lock up of the command:

root@192:/mnt/sda2# dd if=/dev/zero of=test32.dat bs=5M count=400
INFO: task kworker/u8:0:6 blocked for more than 120 seconds.
      Tainted: G           O    4.1.8-rt8+gbd51baf #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kworker/u8:0    D 00000000     0     6      2 0x00000800
Workqueue: kcryptd kcryptd_crypt [dm_crypt]
Call Trace:
[e605bc70] [e600120c] 0xe600120c (unreliable)
[e605bd30] [c07030b0] __schedule+0x1f0/0x4b0
[e605bd60] [c07033b0] schedule+0x40/0xf0
[e605bd70] [c0706014] schedule_timeout+0x144/0x1a0
[e605bdb0] [c0703f98] wait_for_common+0xd8/0x150
[e605bde0] [fa05c048] crypt_convert+0x438/0x530 [dm_crypt]
[e605be30] [fa05e488] kcryptd_crypt+0x248/0x450 [dm_crypt]
[e605be80] [c0047e68] process_one_work+0x158/0x350
[e605bea0] [c0048198] worker_thread+0x138/0x4d0
[e605bef0] [c004d518] kthread+0xb8/0xd0
[e605bf40] [c000f278] ret_from_kernel_thread+0x5c/0x64
INFO: task kworker/u8:2:660 blocked for more than 120 seconds.
      Tainted: G           O    4.1.8-rt8+gbd51baf #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kworker/u8:2    D 00000000     0   660      2 0x00000800
Workqueue: kcryptd kcryptd_crypt [dm_crypt]
Call Trace:
[e63e3c70] [e600120c] 0xe600120c (unreliable)
[e63e3d30] [c07030b0] __schedule+0x1f0/0x4b0
[e63e3d60] [c07033b0] schedule+0x40/0xf0
[e63e3d70] [c0706014] schedule_timeout+0x144/0x1a0
[e63e3db0] [c0703f98] wait_for_common+0xd8/0x150
[e63e3de0] [fa05c048] crypt_convert+0x438/0x530 [dm_crypt]
[e63e3e30] [fa05e488] kcryptd_crypt+0x248/0x450 [dm_crypt]
[e63e3e80] [c0047e68] process_one_work+0x158/0x350
[e63e3ea0] [c0048198] worker_thread+0x138/0x4d0
[e63e3ef0] [c004d518] kthread+0xb8/0xd0
[e63e3f40] [c000f278] ret_from_kernel_thread+0x5c/0x64
INFO: task kworker/u8:3:668 blocked for more than 120 seconds.
      Tainted: G           O    4.1.8-rt8+gbd51baf #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kworker/u8:3    D 00000000     0   668      2 0x00000800
Workqueue: kcryptd kcryptd_crypt [dm_crypt]
Call Trace:
[e62d1c70] [e600120c] 0xe600120c (unreliable)
[e62d1d30] [c07030b0] __schedule+0x1f0/0x4b0
[e62d1d60] [c07033b0] schedule+0x40/0xf0
[e62d1d70] [c0706014] schedule_timeout+0x144/0x1a0
[e62d1db0] [c0703f98] wait_for_common+0xd8/0x150
[e62d1de0] [fa05c048] crypt_convert+0x438/0x530 [dm_crypt]
[e62d1e30] [fa05e488] kcryptd_crypt+0x248/0x450 [dm_crypt]
[e62d1e80] [c0047e68] process_one_work+0x158/0x350
[e62d1ea0] [c0048198] worker_thread+0x138/0x4d0
[e62d1ef0] [c004d518] kthread+0xb8/0xd0
[e62d1f40] [c000f278] ret_from_kernel_thread+0x5c/0x64
INFO: task kworker/u8:4:764 blocked for more than 120 seconds.
      Tainted: G           O    4.1.8-rt8+gbd51baf #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kworker/u8:4    D 00000000     0   764      2 0x00000800
Workqueue: kcryptd kcryptd_crypt [dm_crypt]
Call Trace:
[e64b3d30] [c07030b0] __schedule+0x1f0/0x4b0
[e64b3d60] [c07033b0] schedule+0x40/0xf0
[e64b3d70] [c0706014] schedule_timeout+0x144/0x1a0
[e64b3db0] [c0703f98] wait_for_common+0xd8/0x150
[e64b3de0] [fa05c048] crypt_convert+0x438/0x530 [dm_crypt]
[e64b3e30] [fa05e488] kcryptd_crypt+0x248/0x450 [dm_crypt]
[e64b3e80] [c0047e68] process_one_work+0x158/0x350
[e64b3ea0] [c0048198] worker_thread+0x138/0x4d0
[e64b3ef0] [c004d518] kthread+0xb8/0xd0
[e64b3f40] [c000f278] ret_from_kernel_thread+0x5c/0x64
INFO: task kjournald:1586 blocked for more than 120 seconds.
      Tainted: G           O    4.1.8-rt8+gbd51baf #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kjournald       D 00000000     0  1586      2 0x00000800
Call Trace:
[e6b2fbc0] [c0058934] try_to_wake_up+0x1f4/0x380 (unreliable)
[e6b2fc80] [c07030b0] __schedule+0x1f0/0x4b0
[e6b2fcb0] [c07033b0] schedule+0x40/0xf0
[e6b2fcc0] [c0706014] schedule_timeout+0x144/0x1a0
[e6b2fd00] [c0702e84] io_schedule_timeout+0x84/0xc0
[e6b2fd20] [c0703d24] bit_wait_io+0x34/0x70
[e6b2fd30] [c07038ec] __wait_on_bit+0xac/0x110
[e6b2fd50] [c07039b4] out_of_line_wait_on_bit+0x64/0x80
[e6b2fd90] [c01a63c8] journal_write_superblock+0x128/0x1b0
[e6b2fdd0] [c01a7a40] journal_update_sb_log_tail+0x40/0x90
[e6b2fdf0] [c01a3870] journal_commit_transaction+0x1160/0x12f0
[e6b2fea0] [c01a687c] kjournald+0xdc/0x270
[e6b2fef0] [c004d518] kthread+0xb8/0xd0
[e6b2ff40] [c000f278] ret_from_kernel_thread+0x5c/0x64

This problem does not occur when reading files from the encrypted partition, neither does it occur if I disable all but one of the cores so it appears to be an SMP issue.

[_andev_]

Hi!

Any progress with this issue? I've the same with a more recent kernel, the 4.9.11-1.0.0+g427eb83.

I have a read-only and encrypted filesystem and sometimes reading files from it my process hungs.

I've been able to trace the same than you:

[  366.562305] INFO: task kworker/u4:1:91 blocked for more than 120 seconds.
[  366.562392]       Tainted: G           O    4.9.11-1.0.0+g427eb83 #3
[  366.562434] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  366.562494] kworker/u4:1    D    0    91      2 0x00000000
[  366.562543] Workqueue: kcryptd kcryptd_crypt
[  366.562584] [<8064e578>] (__schedule) from [<8064e8e8>] (schedule+0x50/0xb4)
[  366.562613] [<8064e8e8>] (schedule) from [<806516c8>] (schedule_timeout+0x1d4/0x21c)
[  366.562636] [<806516c8>] (schedule_timeout) from [<8064f3a4>] (wait_for_common+0x90/0x140)
[  366.562660] [<8064f3a4>] (wait_for_common) from [<8050b004>] (crypt_convert+0x3d8/0x4d8)
[  366.562688] [<8050b004>] (crypt_convert) from [<8050cdf4>] (kcryptd_crypt+0x338/0x3b4)
[  366.562736] [<8050cdf4>] (kcryptd_crypt) from [<801414b0>] (process_one_work+0x1d8/0x3f0)
[  366.562763] [<801414b0>] (process_one_work) from [<8014213c>] (worker_thread+0x38/0x558)
[  366.562788] [<8014213c>] (worker_thread) from [<80146f44>] (kthread+0xdc/0xf4)
[  366.562818] [<80146f44>] (kthread) from [<80107638>] (ret_from_fork+0x14/0x3c)

Any hint is welcome!

[_andev_]

Maybe this entry is related:

https://community.nxp.com/thread/476928