eMMC 8GB to 4GB - crash on linux (yocto) boot

[timl]

Hello all,

I work on a custom iMX6 board derived from sabre_SD design and I use u-boot and yocto Dora with the 3.0.35_4.1.0 kernel. I have two hardware versions of this board. The first version is provided with a 8GB eMMC 5.0 and works fine, my problem is with the second version. It is provided with a 4GB eMMC 4.51 (and this is the only difference), I can use u-boot and save environment on the eMMC but I can't boot linux. I use an external microSD card to host u-boot ,the uImage file and the rootfs, uboot and uImage are loaded correctly but on the 4GB version the boot fails after udev starts. The error is always different : udev or mount.sh segmentation fault, core dump, time out on mmc, pagination error ... on my lvds screen the psplash loading bar freezes at different positions, sometimes psplash doesn't load and it freezes on the for penguins console screen. But, if you are patient enough, if you reboot the board again and again, you can boot the board (about one time on twenty) on the fsl-image-gui sato desktop and you can mount and use the eMMC.

Do you have an idea of where could be the problem ?

I tried on different boards, I tried to check with different 4GB eMMC (Sandisk, kingston v4.5 and 4.51), I tried to slow down the bus ... Nothing works at this time and I found nothing about this on google and freescale community search.

[timl · Accepted solution]

Hi all,

I finally fix the issue.

I had to modify setup in file kernel-3.0.35-4.1.0/arch/arm/plat-mxc/include/mach/iomux-mx6q.h and in the board .h file in kernel-3.0.35-4.1.0/arch/arm/mach-mx6/board_myboard.h

I had to add new define of MX6Q_USDHC_PAD_CTRL : PAD_CTL_PUS and PAD_CTL_DSE (or modify the existing one but I prefere use a new one because i don't know if it is used somewhere else in the code)

Change this :

// iomux-mx6q.h
#define MX6Q_USDHC_PAD_CTRL    (PAD_CTL_PKE | PAD_CTL_PUE |    \
        PAD_CTL_PUS_47K_UP | PAD_CTL_SPEED_LOW |    \
        PAD_CTL_DSE_80ohm | PAD_CTL_SRE_FAST | PAD_CTL_HYS)
// ...

#define  MX6Q_PAD_SD4_DAT0__USDHC4_DAT0_50MHZ    \
        (_MX6Q_PAD_SD4_DAT0__USDHC4_DAT0 | MUX_PAD_CTRL(MX6Q_USDHC_PAD_CTRL))
// same for  USDHC4_DAT1 DAT2 DAT3 DAT4 DAT5 DAT6 DAT7 CMD CLK

// board.h
/* USDHC4 */
MX6Q_PAD_SD4_CLK__USDHC4_CLK_50MHZ,
MX6Q_PAD_SD4_CMD__USDHC4_CMD_50MHZ,
MX6Q_PAD_SD4_DAT0__USDHC4_DAT0_50MHZ,
MX6Q_PAD_SD4_DAT1__USDHC4_DAT1_50MHZ,
MX6Q_PAD_SD4_DAT2__USDHC4_DAT2_50MHZ,
MX6Q_PAD_SD4_DAT3__USDHC4_DAT3_50MHZ,
MX6Q_PAD_SD4_DAT4__USDHC4_DAT4_50MHZ,
MX6Q_PAD_SD4_DAT5__USDHC4_DAT5_50MHZ,
MX6Q_PAD_SD4_DAT6__USDHC4_DAT6_50MHZ,
MX6Q_PAD_SD4_DAT7__USDHC4_DAT7_50MHZ,

with this :

// iomux-mx6q.h
#define MX6Q_USDHC_PAD_CTRL_NEW    (PAD_CTL_PKE | PAD_CTL_PUE |    \
        PAD_CTL_PUS_22K_UP | PAD_CTL_SPEED_LOW |    \
        PAD_CTL_DSE_120ohm | PAD_CTL_SRE_FAST | PAD_CTL_HYS)
// ...

#define  MX6Q_PAD_SD4_DAT0__USDHC4_DAT0_NEW    \
        (_MX6Q_PAD_SD4_DAT0__USDHC4_DAT0 | MUX_PAD_CTRL(MX6Q_USDHC_PAD_CTRL_NEW))
// same for  USDHC4_DAT1 DAT2 DAT3 DAT4 DAT5 DAT6 DAT7 CMD CLK

//board.h
/* USDHC4 */
MX6Q_PAD_SD4_CLK__USDHC4_CLK_NEW,
MX6Q_PAD_SD4_CMD__USDHC4_CMD_NEW,
MX6Q_PAD_SD4_DAT0__USDHC4_DAT0_NEW,
MX6Q_PAD_SD4_DAT1__USDHC4_DAT1_NEW,
MX6Q_PAD_SD4_DAT2__USDHC4_DAT2_NEW,
MX6Q_PAD_SD4_DAT3__USDHC4_DAT3_NEW,
MX6Q_PAD_SD4_DAT4__USDHC4_DAT4_NEW,
MX6Q_PAD_SD4_DAT5__USDHC4_DAT5_NEW,
MX6Q_PAD_SD4_DAT6__USDHC4_DAT6_NEW,
MX6Q_PAD_SD4_DAT7__USDHC4_DAT7_NEW,

This works better with a little add in the make menuconfig : System Type > Freescale MXC Implementations > [*] MX6 clk settings for smooth UI transtion from bootloader to kernel

Thanks you for your answers.

[igorpadykov]

Hi timl

from a hardware perspective i.MX6 technically do no support eMMC > 4.4 standard

boot, although it can boot from such cards,  boot ROM will fall back to the eMMC4.4

standard when a eMMC > 4.4 capable device is detected.

Actually described issue may be caused by DDR errors as well.

One can try to update DDR calibration settings with tool below

https://community.freescale.com/docs/DOC-96412

also it may be recommended to check signals with oscilloscope

and run basic read/write tests with SDK

i.MX 6Series Platform SDK

Best regards

igor

[timl]

Hi Igor,

thanks you for your answer.
I had the same idea but RAM seems to be O.K. : the same RAM configuration (hardware and software) works fine on the 8GB version. I checked the 4GB and the 8GB version with the DDR-stresser (25 loops) in your first link and both pass it successfully. I can use uboot mtest and save uboot environment variable on both 4 and 8 GB eMMC so I think read/write is OK.

I suspect there might be some variable in a kernel or fs configuration file that describe the eMMC size but i have found nothing so far.

[igorpadykov]

Hi timl

you said in first post that "Nothing works at this time",

do you mean that Uboot is loaded OK ?

After boot failure, one connect the i.MX6 OTG on the board to the PC, if a new HID

device can be found,  the ROM code enter serial download mode which mean

some error occurred in the booting process. One can also check with MFG Tool,

if it sees i.MX6 device.

Then attach the board with any JTAG debugger, such as RV-ICE or

Lauterbach and dump 0x907400 (ROM uses 0x907000 as starting address),

you should see your IVT header here if the eMMC access is ok.

~igor

[timl]

Hi Igor,

you said in first post that "Nothing works at this time",

do you mean that Uboot is loaded OK ?

Yes. I have two boards with the same hardware only the eMMC are different, I run u-boot and linux from a micro-SD card, I use the same card on both board :

• one with 8GB eMMC 5.0 => u-boot, uImage and rootfs are loaded OK the desktop appear on the LVDS screen
• one with 4GB eMMC 4.51 => u-boot is loaded OK (and I can save and load environment from eMMC), uImage is loaded, but linux crash at boot (after udev starts) with a random error (as core dump, time out or freeze before psplash ...)

What I mean when I said "Nothing works at this time" is

I tried on different boards, I tried to check with different 4GB eMMC (Sandisk, kingston v4.5 and 4.51), I tried to slow down the bus

those try didn't change anything at my problem.

We have no JTAG connector on this board but the OTG test looks normal, the HID device is only detected on start with no u-boot and never appears again after u-boot begin. (tested with lsusb in ubuntu and MFGTool in Win7)

You will find in the attached documents the dmesg of : boot with 8GB (always), boot with 4GB (not very often but sometimes it happens !), crash with 4GB (2 errors copied)

[timl]

Hi Igor and Rick,

I've tested something on my own : to boot with "set boot_fdt no" it is a little better on the 4GB it boots about 1/5 1/4 times

I just tried your solutions,

@Rick >  the patch on mmc driver did not change anything.

@Igor > I tried to add those parameters, it is worst the 4GB freeze or crash with one core only. It tried with and without Rick's patch on mmc driver.

The 8GB boot with no problem with both solutions.

[igorpadykov]

Hi timl

if it worst with one core only this may point that eMMC is affected by board noise,

since with one core system boots much faster. You may wish to experiment with eMMC

drive strength settings, as shown below in patches

0001-mmc-core-Set-non-default-Drive-Strength-via-platform.patch

sdio_patch_from_3.5.7.zip

will imx6 JB code supports SDIO 3.0 cards

Also one can try to decrease SD driver strength from i.MX6 side,

using fields SPEED,SRE and DSE of registers IOMUXC_SW_PAD_CTL_PAD_SDx_DATAx

for appropriate SD pads.

~igor

[timl]

Hi Igor,

I have tuned my settings for SPEED SRE and DSE in the i.MX6 side.

The good news is that it has fixed one stability issue I had with my external SD card reader thanks you

The bad news is that it has not fixed the eMMC problem.

There is something new I tested, I can boot without problem if I remove the eMMC.

As I read very often on the crash prints an error

Unable to handle kernel paging request at virtual address XXXXXXXX

and as the crash occurs only on the 4GB board and because u-boot works well (I assume it is a direct access to the eMMC)

I think the problem might comes from virtual memory allocation (mmu ?) or kernel configuration (I found nothing on the make menuconfig up to now) ?

[igorpadykov]

Hi timl

"paging request" errors may point to DDR errors (caused by not very well

optimized DDR settings or big board noise). One can try to decrease DDR operating

frequency.

For eMMC I would suggest to add filtering capacitors (say 20-50uF or better as RC)

directly on eMMC power pads.

I am inclined to think that board may have big noise (probably due to layout),

one can try to run minimal image, removing GPU/VPU and decrease operating frequency.

~igor

[igorpadykov]

Hi timl

probably one can also try kernel boot parameters

maxcpus=0 ("nosmp"), enable_wait_mode=off

and remove config option CONFIG_LOCAL_TIMERS

i.mx6q slow boot - systemd smp

~igor

[rickchu]

Would you like to try this patch?

git tag: jb4.2.2_1.1.0-ga  kernel_imx/drivers/mmc/host/sdhci-esdhc-imx.c

static int esdhc_pltfm_init(struct sdhci_host *host, struct sdhci_pltfm_data *pdata)

{

  struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);

  struct esdhc_platform_data *boarddata = host->mmc->parent->platform_data;

....

    /*

    * on mx6dl TO 1.1, ADMA can work when ahb bus frequency is low,

    * like 24Mhz. MX53 does have working ADMA.

    */

    if (mx6dl_revision() >= IMX_CHIP_REVISION_1_1 || cpu_is_mx53())

        host->quirks &= ~SDHCI_QUIRK_BROKEN_ADMA;

    if (cpu_is_mx6())

        host->quirks2 |= SDHCI_QUIRK_BROKEN_AUTO_CMD23;

    /*

    * Issue Describe:

    * Sometimes files data will lost at the end of parts with ACMD23 (

    * enable multiple block read with pre-defined block count).

    */

    if (host->mmc->index == ESDHC_PORT_EMMC) {

        if (cpu_is_mx6()) {

            host->quirks |= SDHCI_QUIRK_MULTIBLOCK_READ_ACMD12;

            host->quirks2 &= ~SDHCI_QUIRK_BROKEN_AUTO_CMD23;

        }

    }

    /* write_protect can't be routed to controller, use gpio */

    sdhci_esdhc_ops.get_ro = esdhc_pltfm_get_ro;

.....