Debugging a bare-metal uSDHC driver, written in Rust for the i.MX 8M nano

[npashi]

I've been working on adding Rust support for the i.MX 8M nano and managed to get the board up and running along with a working uart driver - code here

The next step in my plan is to add a uSDHC driver for the nano. Here's a link to the initial implementation

The implementation keeps throwing some i.MX specific uSDHC command errors. I was hoping someone here can shed some light on this.

The sequence of steps I used are as follows

1. enable the usdhc2 clocks via the ccm peripheral,
2. set mux state for the uSDHC2 peripheral via the iomuxc peripheral
3. initialize uSDHC driver and send a couple of commands to verify we have working driver

So far, steps 1 and 2 seem to work (i think)

Here's the debug log output. From what I gather by looking at the contents of the uSDHC's interrupt status field (i.e. a value of 0x00048001), the command (i.e. SEND_IF_CMD) was successfully sent and a response was received, however the response returned contains an error - command end bit error. Not entirely sure why this is happening.

Also the present status register has the SDOFF bit set (SD clock gated off internally). Again not sure why this is happening. The reference manual doesn't provide more information about these bits or the errors.

Any inputs would be greatly appreciated. Please let me know if more information is required.

[    0.000004] imx8mn-rs version 0.1.0
[    0.003516] Booting on: i.MX 8M Nano EVK
[    0.007340] Current privilege level: EL3
[    0.011254] Exception handling state:
[    0.014966]       Debug:  Masked
[    0.018166]       SError: Unmasked
[    0.021464]       IRQ:    Masked
[    0.024758]       FIQ:    Masked
[    0.027968] Drivers loaded:
[    0.030739]       1. i.MX8M Uart2
[    0.034045] Chars written: 382
[    0.036995] Sd host circuit reset..
[    0.040462] Prescaler = 4, Divisor = 15, Freq Set = 400000
[    0.045966] Sd: sending command, CMD_NAME: "GO_IDLE_STATE", CMD_CODE: 0x00000000, CMD_ARG: 0x00000000
[    0.056378] Sd: sending command, CMD_NAME: "SEND_IF_COND", CMD_CODE: 0x08020000, CMD_ARG: 0x000001aa
[    0.066602] Sd Error: Cmd response returned an error, PresentStatus: 0xf0048088, intStatus: 0x00048001, Resp0: 0x00000000, timeDiff: 0

[    0.078923] Sd host circuit reset..
[    0.082293] Prescaler = 4, Divisor = 15, Freq Set = 400000
[    0.087788] Sd: sending command, CMD_NAME: "GO_IDLE_STATE", CMD_CODE: 0x00000000, CMD_ARG: 0x00000000
[    0.098218] Sd: sending command, CMD_NAME: "APP_CMD", CMD_CODE: 0x17000000, CMD_ARG: 0x00000000
[    0.108176] Sd: sending command, CMD_NAME: "APP_SEND_OP_COND", CMD_CODE: 0x09020000, CMD_ARG: 0x00ff8000
[    0.119630] Sd Error: Cmd response returned an error, PresentStatus: 0xf0048088, intStatus: 0x00048001, Resp0: 0x00000000, timeDiff: 0

[    0.131839] Sd Error: ACMD41 returned non-timeout error SdError

[    0.137829] uSDHC: PRES_STATE: 0xf0048088, SYS_CTRL: 0x000e02e0, INT_STATUS: 0x00048001

[    0.146063] uSDHC: CMD "APP_SEND_OP_COND", resp: SdError, CMD_RSP3: 0x00000000, CMD_RSP2: 0x00000000, CMD_RSP1: 0x00000000, CMD_RSP0: 0x00000000

[    0.159382] failed to initialize
[    0.162494]

[    0.164107]  ... wait forever

[npashi]

Update:

Just thought I'd add this here, in case anyone finds this useful or can figure out what I'm missing.

During uSDHC initialization, I use an identification clock of 400Khz and a 3.3 volts setting. I'm currently testing an 8GB card.

Here is the list of steps in my initialization routine

1. Reset the entire host controller by setting the RSTA bit. Just to be sure I also set the RSTC + RSTD bits
2. set sd-clock to 400khz i.e. the prescaler field in the SYS_CTRL register is set to 0x20 and divisor field is set to 0xf.
3. Note - I'm using PLL1_DIV2 as my source root clock i.e. SYSTEM_PLL1_DIV2_CLK or 800Mhz/2 for uSDHC2. (note: I have not configured the source clock, I use it as-is. I assume this is OK).
   
4. enable relevant IRQ bits via the interrupt status enable register
5. set the initialize active field in SYS_CTRL register and wait until it is cleared
6. set the little-endian and four-bit wide fields in the PROT_CTRL register
7. set data timeout value field in the SYS_CTRL to its maximum i.e. 0xf
8. at this point, I assume the card is initialized
9. and send CMD0 wait 2ms and send a CMD8
10. It fails to initialize after CMD8 is sent. Looking at the command completion bit in the interrupt status register, we can conclude that a response was received but it contains errors.

For pinmux settings - I use the values from the imx8mn nano device-tree, as listed below

• clock line     - <0xd4 0x33c 0x00 0x00 0x00 0x190>
• cmd line       - <0xd8 0x340 0x00 0x00 0x00 0x1d0>
• data0 line     - <0xdc 0x344 0x00 0x00 0x00 0x1d0>
• data1 line     - <0xe0 0x348 0x00 0x00 0x00 0x1d0>
• data2 line     - <0xe4 0x34c 0x00 0x00 0x00 0x1d0>
• data3 line     - <0xe8 0x350 0x00 0x00 0x00 0x1d0>
• cd line          - <0x38 0x2a0 0x00 0x01 0x00 0x1d0>;
• reset line      - <0xec 0x354 0x00 0x05 0x00 0x41>;

with some modifications.

• I have applied a drive strength of 6 instead of 1.
• the reset line is set is set to GPIO output mode before setting the mux-modes and pad config for all the other pins. Once done, we clear it. If this is not done, we get a sd-timeout error instead of sd-command errors.

[npashi]

I think I may have (finally) found the fix. I had to remove the entire reset logic for it to work i.e.

• remove hardware reset or power-cycling logic (this was done by muxing the uSDHC2’s RESET line to gpio-mode and toggling gpio pin 19 to power-cycle the sd-card) and
• remove software reset logic (performed by setting RSTA)
• and set the reset line’s mux-mode to uSDHC-mode.

and it magically started working.

I still have to no idea why this works? I hope NXP can explain why this works? But until then, my best guess is that the i.MX 8M Nano EVK board deviates from both the SD spec and the reference manual.

If anyone wants to test this. Please clone the latest (master) commit, build and flash. You should see debug output below

[ 0.000005] imx8mn-rs version 0.1.0
[ 0.003617] Booting on: [i.MX](https://i.mx/) 8M Nano EVK
[ 0.007682] Current privilege level: EL3
[ 0.011755] Exception handling state:
[ 0.015561] Debug: Masked
[ 0.018894] SError: Unmasked
[ 0.022413] IRQ: Masked
[ 0.025746] FIQ: Masked
[ 0.028993] Drivers loaded:
[ 0.031841] 1. i.MX8M Uart2
[ 0.035289] Chars written: 382
[ 0.038436] uSDHC2 supports 1.8v, 3.0v, 3.3v ...
[ 0.043311] NO RESET, PRES_STAT: 0xff8d8088
...
[ 0.050676] NO RSTA, PRES_STAT: 0xff8d8088
...
[ 0.082844] Sd clock stablized in 37us
[ 0.086777] Prescaler = 64, Divisor = 16, Freq = 390625 Hz
...
[ 0.108591] before CMD0 sent, PRES_STAT: 0xff8d8088
[ 0.113783] Sd: sending command, CMD_NAME: "GO_IDLE_STATE", CMD_CODE: 0x00000000, CMD_ARG: 0x00000000
[ 0.126803] Sd: sending command, CMD_NAME: "SEND_IF_COND", CMD_CODE: 0x081a0000, CMD_ARG: 0x000001aa
[ 0.137920] Sd: sending command, CMD_NAME: "APP_CMD", CMD_CODE: 0x371a0000, CMD_ARG: 0x00000000
[ 0.148645] Sd: sending command, CMD_NAME: "APP_SEND_OP_COND", CMD_CODE: 0x29020000, CMD_ARG: 0x50ff8000
[ 0.560598] Sd: sending command, CMD_NAME: "APP_CMD", CMD_CODE: 0x371a0000, CMD_ARG: 0x00000000
[ 0.571475] Sd: sending command, CMD_NAME: "APP_SEND_OP_COND", CMD_CODE: 0x29020000, CMD_ARG: 0x50ff8000
[ 0.583475] Sd: sending command, CMD_NAME: "ALL_SEND_CID", CMD_CODE: 0x02090000, CMD_ARG: 0x00000000
[ 0.595085] Sd: sending command, CMD_NAME: "SEND_REL_ADDR", CMD_CODE: 0x03020000, CMD_ARG: 0x00000000
[ 0.606431] Sd: sending command, CMD_NAME: "SEND_CSD", CMD_CODE: 0x09010000, CMD_ARG: 0x00010000
[ 0.617763] CSD Contents : 00 40 0e 00 32 5b 59 00 003b 83 7f 80 0a 40 00
[ 0.624947] cemmc_structure=1, spec_vers=0, taac=0x0E, nsac=0x00, tran_speed=0x32,ccc=0x05B5, read_bl_len=0x09, read_bl_partial=0b, write_blk_misalign=0b,read_blk_mi
salign=0b, dsr_imp=0b, sector_size =0x7F, erase_blk_en=1b
[ 0.645753] CSD 2.0: ver2_c_size = 0x3BFF, card capacity: 7987527680 bytes or 7.99GiB
[ 0.654006] wp_grp_size=0x0000000b, wp_grp_enable=0b, default_ecc=00b, r2w_factor=010b, write_bl_len=0x09, write_bl_partial=0b, file_format_grp=0, copy=0b, perm_writ
e_protect=0b, tmp_write_protect=0b, file_format=0b ecc=00b

[npashi]

Some context for why the (above) fix works: I zeroed-in on the issue with the following observation which was consistent across multiple rounds of testing

• I looked at (a working execution trace of) U-boot's register values and saw the PRES_STAT register (always) has DLSL and CLSL bits set. This makes sense as the pad (or pin) configuration is set to 0x1d0 (taken from the device-tree) or more precisely is as follows.
  • Dse::DseX1,
  • Fsel::Fast,
  • Ode::Disabled,
  • Pue::PullUp,
  • Hys::Enabled,
  • Pe::Enabled,
• they should always be pulled high. In u-boot's case, the PRES_STAT register always has CLSL and DLSL bits set and contains the following value - 0xff888088
• but in my case, applying the same pin configuration results in a PRES_STAT value of 0xff8d8088 and upon performing a software RESET i.e. setting the RSTA bit in SYS_CTRL and waiting for it to be cleared, I saw that CLSL and DLSL lines are pulled low. PRES_STAT value changes to 0xf00d8088. This happens consistently. 

[   0.000005] imx8mn-rs version 0.1.0
[   0.003618] Booting on: i.MX 8M Nano EVK
[   0.007667] Current privilege level: EL3
[   0.011687] Exception handling state:
[   0.015349]      Debug: Masked
[   0.018745]      SError: Unmasked
[   0.022187]      IRQ:   Masked
[   0.025411]      FIQ:   Masked
[   0.028761] Drivers loaded:
[   0.031530]      1. i.MX8M Uart2
[   0.034922] Chars written: 382
[   0.038160] uSDHC2 has support for 1.8v, 3.0v, 3.3v ...
[   0.044572] after HW RESET, PRES_STAT: 0xff8d8088
[   0.049441] checking register value
[   0.051909] after RSTA cleared, PRES_STAT: 0xf40d8088
[   0.057188] Sd host circuit reset in 3us
[   0.061136] after Sd host cicuit reset, PRES_STAT: 0xf00d8088
...
[   0.087812] Sd clock stablized in 38us
[   0.091674] Prescaler = 64, Divisor = 16, Freq = 390625 Hz
...
[   0.118224] Sd: sending command, CMD_NAME: "GO_IDLE_STATE", CMD_CODE: 0x00000000, CMD_ARG: 0x00000000
[   0.131208] Sd: sending command, CMD_NAME: "SEND_IF_COND", CMD_CODE: 0x081a0000, CMD_ARG: 0x000001aa
[   0.145884] Error: we got a response for the last cmd but it contains errors, decode contents of interrupt status register for details
 VendSpec: 0x30007809, SysCtrl: 0x000f20ff, ProtCtrl: 0x08800022, PresentStatus: 0xf00d8088, intStatus: 0x000c8001, Resp0: 0x00000000, Resp1: 0x00000000, Resp2: 0x00000000, Resp3: 0x00000000, CC_bit set in 4131us
[   0.177955] SdError: Send interface condition command (CMD8) returned an error
[   0.185646] failed to initialize
[   0.189029]
[   0.190608] ... wait forever

[npashi]

Update:

• I pushed the latest version of my implementation to GitHub.
  • usdhc driver impl
  • clock control impl
  • pin-mux config impl
• After including changes in latest version, I noticed the following - when I set the command index enable bit in the CMD_XFR_TYP register, I now get 2 errors.
  1. Command index error: which means the response did not contain the command index the host sent.
  2. Command end bit error: which occurs when detecting that the end field of a command response is 0. 

Given the above - I can summarise the issue as the following (with some degree of certainty)

• SD commands from the host are not going through to the sd-card. But the host controller seems to believe that it did send the command and it also acknowledges having received a response. But the response is (probably) always a string of zeroes.

Some extra context: just in case this is required

• Build and test instructions:
  • clone the git repo from here
  • navigate to the path- rustBoot/boards/bootloaders/imx8mn
  • assuming you have rust nightly installed and the aarch64-unknown-none-softfloat target added, simply run `cargo build --release` to compile and build the bootloader.
  • once done you can either use a jtag programmer or some other method to load the code on to the device and run it. Debug logs will be output to the default serial peripheral at 0x30890000.
  • The compiled elf binary will be located at the following path- boards/target/aarch64-unknown-none-softfloat/release/imx8mn-rs

[npashi]

Update: I think I made some headway but the error still seems baffling.  

So, I used the (exact) same pad-ctrl settings as the ones from the linux device tree in `imx8mn.dtsi` and I am now able to detect and report if an sd-card is inserted. But I still cant figure out what's causing the command error (i.e. CEBE bit error).

[   71.987092] imx8mn-rs version 0.1.0
[   71.990506] Booting on: i.MX 8M Nano EVK
[   71.994481] Current privilege level: EL3
[   71.998294] Exception handling state:
[   72.002008]       Debug:  Masked
[   72.005139]       SError: Unmasked
[   72.008524]       IRQ:    Masked
[   72.011811]       FIQ:    Masked
[   72.014926] Drivers loaded:
[   72.017677]       1. i.MX8M Uart2
[   72.020958] Chars written: 382
[   72.023983] Sd host circuit reset..
[   72.027439] card inserted...
[   72.030367] Prescaler = 4, Divisor = 15, Freq Set = 400000
[   72.035865] Sd: sending command, CMD_NAME: "GO_IDLE_STATE", CMD_CODE: 0x00000000, CMD_ARG: 0x00000000
[   72.049859] Sd: sending command, CMD_NAME: "ALL_SEND_CID", CMD_CODE: 0x02010000, CMD_ARG: 0x00000000
[   72.065140] Sd Error: Cmd response returned an error, VendSpec: 0x2000f800, ProtCtrl: 0x00800040, PresentStatus: 0xf00d8088, intStatus: 0x00048001, Re
sp0: 0x00000000, timeDiff: 5062

[   72.081495] uSDHC: PRES_STATE: 0xf00d8088, SYS_CTRL: 0x000e02ef, INT_STATUS: 0x00048001

[   72.089641] uSDHC: CMD "ALL_SEND_CID", resp: SdError, CMD_RSP3: 0x00000000, CMD_RSP2: 0x00000000, CMD_RSP1: 0x00000000, CMD_RSP0: 0x00000000

[   72.102677] failed to initialize
[   72.105781]

[   72.107302]  ... wait forever

On a side note, I noticed that one of the pins in the device tree is referenced as uSDHC2's voltage regulator (i.e. pad  iomuxc_sw_mux_ctl_pad_sd2_reset_b) but its pinmux configuration uses a mux mode that doesn't make sense. The device-tree contains a mux mode of 0x5 for this pin instead of a 0. Would anyone know why? 

When I set the pin’s mux-mode to 0x5, the uSDH controller simply times out i.e. it doesn’t even detect the card.

Device tree source for the usdhc2’s voltage regular:

reg_usdhc2_vmmc: regulator-usdhc2 {
 compatible = "regulator-fixed";
 pinctrl-names = "default";
 pinctrl-0 = <&pinctrl_reg_usdhc2_vmmc>;
 regulator-name = "VSD_3V3";
 regulator-min-microvolt = <3300000>;
 regulator-max-microvolt = <3300000>;
 gpio = <&gpio2 19 GPIO_ACTIVE_HIGH>;
 off-on-delay-us = <12000>;
 enable-active-high;
};

&usdhc2 {
assigned-clock-rates = <200000000>;
assigned-clocks = <&clk IMX8MN_CLK_USDHC2>;
pinctrl-names = "default", "state_100mhz", "state_200mhz";
pinctrl-0 = <&pinctrl_usdhc2>, <&pinctrl_usdhc2_gpio>;
pinctrl-1 = <&pinctrl_usdhc2_100mhz>, <&pinctrl_usdhc2_gpio>;
pinctrl-2 = <&pinctrl_usdhc2_200mhz>, <&pinctrl_usdhc2_gpio>;
cd-gpios = <&gpio1 15 GPIO_ACTIVE_LOW>;
bus-width = <4>;
vmmc-supply = <&reg_usdhc2_vmmc>;
status = "okay";
};

pinctrl_reg_usdhc2_vmmc: regusdhc2vmmcgrp {
fsl,pins = <MX8MN_IOMUXC_SD2_RESET_B_GPIO2_IO19 0x41>;
};​

Device tree binary for the usdhc2 voltage regulator :

regusdhc2vmmcgrp {
                    fsl,pins = <0xec 0x354 0x00 0x05 0x00 0x41>;
                    u-boot,dm-spl;
                    phandle = <0x42>;
                };

regulator-usdhc2 {
        compatible = "regulator-fixed";
        pinctrl-names = "default";
        pinctrl-0 = <0x42>;
        regulator-name = "VSD_3V3";
        regulator-min-microvolt = <0x325aa0>;
        regulator-max-microvolt = <0x325aa0>;
        gpio = <0x26 0x13 0x00>;
        enable-active-high;
        u-boot,off-on-delay-us = <0x4e20>;
        phandle = <0x2d>;
    };

mmc@30b50000 {
                compatible = "fsl,imx8mn-usdhc\0fsl,imx8mm-usdhc\0fsl,imx7d-usdhc";
                reg = <0x30b50000 0x10000>;
                interrupts = <0x00 0x17 0x04>;
                clocks = <0x02 0x56 0x02 0x4d 0x02 0xaf>;
                clock-names = "ipg\0ahb\0per";
                fsl,tuning-start-tap = <0x14>;
                fsl,tuning-step = <0x02>;
                bus-width = <0x04>;
                status = "okay";
                assigned-clocks = <0x02 0x68>;
                assigned-clock-rates = <0xbebc200>;
                pinctrl-names = "default\0state_100mhz\0state_200mhz";
                pinctrl-0 = <0x29 0x2a>;
                pinctrl-1 = <0x2b 0x2a>;
                pinctrl-2 = <0x2c 0x2a>;
                cd-gpios = <0x23 0x0f 0x01>;
                vmmc-supply = <0x2d>;
                u-boot,dm-spl;
                sd-uhs-sdr104;
                sd-uhs-ddr50;
            };