Debugging Linux Kernel over JTAG with J-Link

[jonahpetri]

We've been working through getting a working setup for debugging the linux kernel via JTAG, using Segger's J-Link.  There's a few hints on this forum and elsewhere, but not a complete guide, so here's what works for us:

First: if you just go and try debugging the kernel, you'll probably find that you either:

• can't connect at all once linux has booted, or
• if you connect at u-boot, and then let linux load, you get disconnected around the time that init runs, or
• if you get lucky and manage to connect when linux is running, you get disconnected shortly afterwards.

This issue is caused by linux going into a low power idle state.  In this state, linux will gate the ARM clock domain, causing the JTAG to lose connection whenever the OS is executing the idle task. There are two ways around this. The first is less invasive, but slightly more limited.

Method 1 (easy method): Disable 'clock off' wait state in the cpuidle driver

On the IMX6 you can disable the clock off wait state through sysfs. Navigate to the cpuidle driver for the cpu (cpu0 in this case as we are on the solo lite).

cd /sys/devices/system/cpu/cpu0/cpuidle/state1

cat desc

The output of 'cat desc' should be 'Clock off'. If this is not the case, go up a folder and check the other states until you find the one with the 'Clock off' desc. Now, you simply need to disable this wait state

echo 1 > disable

Now you should be able to attach with JTAG and debug to your heart's content.

Method 2 (harder method): Disable gating of the ARM clock domain in the source

The above method is good for most cases of debugging, unless something needs to be debugged during startup. In this case you will need to modify the source to stop the IMX from ever gating the ARM clock domain. You can apply the following patch to disable the clock gating.

diff --git a/arch/arm/mach-imx/pm-imx6.c b/arch/arm/mach-imx/pm-imx6.c

index e1a45e2..feadccb 100644

--- a/arch/arm/mach-imx/pm-imx6.c

+++ b/arch/arm/mach-imx/pm-imx6.c

@@ -552,8 +552,8 @@ int imx6q_set_lpm(enum mxc_cpu_pwr_mode mode)

        case WAIT_CLOCKED:

                break;

        case WAIT_UNCLOCKED:

-               val |= 0x1 << BP_CLPCR_LPM;

-               val |= BM_CLPCR_ARM_CLK_DIS_ON_LPM;

                break;

        case STOP_POWER_ON:

                val |= 0x2 << BP_CLPCR_LPM;

J-Link setup

We have been using the Segger J-Link to debug ARM platforms from a linux host. Download the Segger J-Link tools for Linux from the Segger website. After you have done one of the two above methods you can now attach the J-Link GDB server. Note that by default your user wont have permissions to access the J-Link USB device. You can either setup a udev rule to set the permissions properly, or simply run the gdb server as root. We will do the latter as we are lazy. Note the noir and noreset options. noir will not init the working register and noreset won't reset the cpu core. See the J-Link documentation included with the download for more info.

sudo ./JLinkGDBServer -if JTAG -endian little -device MCIMX6L8 -select USB -noir -noreset

Helpful kernel configuration options for debugging

Before diving right into debugging, note there are some useful kernel configuration options that will make your life a lot easier. Below is a list of kernel configuration options you should set:

CONFIG_DEBUG_INFO=y # this will compile in debugging symbols and info

GDB client connection and debugging!

Finally, with a GDB server running that is connected to your target, you can connect with a GDB client. Note that you can point GDB to your linux binary for the debugging information. Also, you should use your cross compile GDB for this target. If you are using the Yocto Poky SDK (as we are), you can simply source the SDK and use '$GDB' in place of gdb-[CROSS-COMPILE] below. Note that [CROSS-COMPILE] should be replaced with your proper cross compile tuple (e.g. gdb-arm-linux-eabi).

$ gdb-[CROSS-COMPILE] /path/to/linux/build/vmlinux

(gdb) target remote localhost:2331

Enjoy your debugging session!  Any questions or comments, please post them below.

[arnoutdiels]

Just a quick node, in case you are working with an EVK for the IMX6UL, don't forget to disable the sai2 in the device tree, which pinmuxes away the jtag in favor of SAI pins.

[skrap]

One follow-up on this: the i.MX7 has a further low-power state beyond "Clock off", so you should probably do something like this for "method 1" above:

#!/bin/sh
for i in /sys/devices/system/cpu/cpu*/cpuidle/state[123456789]/disable; do 
        echo 1 > $i
        echo $i
done

The [123456789] above is just trying to avoid state0, which is WFI and is OK for JTAG debugging.

If you want to debug using "method 2" above, the easiest thing may be to just turn off CONFIG_CPU_IDLE in your kernel config.  I haven't verified, but that should do the trick.

[dry]

The setup doesn't appear to work on iMX7D, Sabre board, with default Linux & image as provided by NXP, when you try to connect to the M4 core.  The entire SoC, or A7 core(s), get reset, when J-Link tries to connect.

Any idea?

[skrap]

We took JTAG off of our board pretty early on in the dev cycle, so I can't easily verify for you.  But, one thing you can try is breaking into u-boot, and then attaching at that point.  If you _can_ attach to u-boot, but can't attach to linux, then I'd guess it's either some necessary clock which Linux is either disabling, or changing frequency.  Or, it might be something in the pinmux which is changing.

Does that help?

[dry]

Well yea, with u-boot it's not an issue, sorry I should have mentioned it.

It is Linux magic.

Already removed disabling of unused clocks,  halt instructions (wfi), and cpu frequency set to off (all from linux boot command line). So that is all similar  / same as you have found above.

But the thing still resets ... Is there some hardware protection mechanism which can get enabled in software, which would exclude J-Link connection to the M4 core, and reset the SoC when trying?

I can connect fine to A7_0 core.. halt/go Linux. But cannot access M4 SCR register from debugger, to say, enable M4 this way.  (And any attempt to then connect to M4 resets the entire thing, sorry repeating).

Urg.

[skrap]

I've not tried to use the M4, so I am not going to be of too much help.  I do note that there are M4-specific J-Link configurations here: https://wiki.segger.com/IMX7D

[dry]

Yea thanks, I think i've learned every word of that connect script by now.

But may be i need to double check if it's being used at all, so thanks for pointing it out

[skrap]

All right, best of luck!  If I have any other ideas, I'll post them here.  I hope if you find a solution you'll also add it!

[skrap]

There's a hardware watchdog... that might do it, but under normal configurations it's either disabled or ticks every 10 seconds or so...

[dry]

Yea I've set nowatchdog parameter too, and even proceeded to remove all from kernel config & rebuild ..

The system resets way too quick for 10 secs.. it's almost immediately on connect attempt.

[sanjaymishra]

Thanks for putting the details up.