Custom i.MX6SX board freeze after enabling L310 cache controller

[robyf]

We have developed a new PCB starting from mx6sxsabresd design. It currently passes 12h of calibration test without problem,

so the board design should be ok.

The board boot straight with u-boot-imx and it's able to at least boot the first kernel stage until it stops with some printed garbage, the log is below.

It has a custom DTS slight changed from imx6sx-17x17-arm2.dts.

Any idea how to debug it?

Cheers,

Roberto Fichera.

U-Boot 2014.04-imx_v2014.04_3.14.28_1.0.0_ga+g88123ea (Jul 08 2015 - 17:57:57)

CPU:   Freescale i.MX6SX rev1.2 at 792 MHz

CPU:   Temperature 41 C, calibration data: 0x5ba52269

Reset cause: POR

Board: Domus iMX6SX (ID:e301dab51902b1d4)

I2C:   ready

DRAM:  512 MiB

MAX7322 Not found

MMC:   FSL_SDHC: 0, FSL_SDHC: 1

*** Warning - bad CRC, using default environment

In:    serial

Out:   serial

Err:   serial

Found PFUZE100! deviceid 0x11, revid 0x21

mmc0 is current device

Net:   Phy not found

PHY reset timed out

FEC0

Warning: failed to set MAC address

Normal Boot

Hit any key to stop autoboot:  0

=> bdinfo

arch_number = 0x00000000

boot_params = 0x80000100

DRAM bank   = 0x00000000

-> start    = 0x80000000

-> size     = 0x20000000

eth0name    = FEC0

ethaddr     = (not set)

current eth = FEC0

ip_addr     = <NULL>

baudrate    = 115200 bps

TLB addr    = 0x9FFF0000

relocaddr   = 0x9FF66000

reloc off   = 0x18766000

irq_sp      = 0x9DF63EE0

sp start    = 0x9DF63ED0

=> setenv mmcroot /dev/mmcblk3p2 rootwait rw earlyprintk

=> saveenv

Saving Environment to MMC...

Writing to MMC(0)... done

=> boot

mmc0 is current device

mmc0 is current device

reading boot.scr

** Unable to read file boot.scr **

reading zImage

5934672 bytes read in 278 ms (20.4 MiB/s)

Booting from mmc ...

reading domus-reva.dtb

48167 bytes read in 19 ms (2.4 MiB/s)

Kernel image @ 0x80800000 [ 0x000000 - 0x5a8e50 ]

## Flattened Device Tree blob at 83000000

    Booting using the fdt blob at 0x83000000

    Using Device Tree in place at 83000000, end 8300ec26

switch to ldo_bypass mode!

Starting kernel ...

Uncompressing Linux... done, booting the kernel.

Booting Linux on physical CPU 0x0

Linux version 3.14.28-1.0.0_ga+g91cf351 (roberto@dragon.tekno-soft.it) (gcc version 4.9.2 (GCC) ) #1 SMP PREEMPT Fri Jul 3 12:43:06 CEST 2015

CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c53c7d

CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache

Machine model: Voneus i.MX6 SoloX Domus ARM2 Board

bootconsole [earlycon0] enabled

cma: CMA: reserved 320 MiB at 8c000000

Memory policy: Data cache writealloc

PERCPU: Embedded 8 pages/cpu @8bb3a000 s8896 r8192 d15680 u32768

Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 130048

Kernel command line: console=ttymxc0,115200 root=/dev/mmcblk3p2 rootwait rw earlyprintk

PID hash table entries: 2048 (order: 1, 8192 bytes)

Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)

Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)

Memory: 176704K/524288K available (7213K kernel code, 394K rwdata, 6196K rodata, 328K init, 428K bss, 347584K reserved, 0K highmem)

Virtual kernel memory layout:

     vector  : 0xffff0000 - 0xffff1000   (   4 kB)

     fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)

     vmalloc : 0xa0800000 - 0xff000000   (1512 MB)

     lowmem  : 0x80000000 - 0xa0000000   ( 512 MB)

     pkmap   : 0x7fe00000 - 0x80000000   (   2 MB)

     modules : 0x7f000000 - 0x7fe00000   (  14 MB)

       .text : 0x80008000 - 0x80d20678   (13410 kB)

       .init : 0x80d21000 - 0x80d732c0   ( 329 kB)

       .data : 0x80d74000 - 0x80dd6ac0   ( 395 kB)

        .bss : 0x80dd6acc - 0x80e41d9c   ( 429 kB)

SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1

Preemptible hierarchical RCU implementation.

         RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=1.

RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1

NR_IRQS:16 nr_irqs:16 16

L310 cache controller enabled

l2x0: 16 ways, CACHE_ID 0x410000c8, AUX_CTRL 0x32030000, Cache size: 256 kB

*** GARBAGE START HERE ***

[robertlehmann]

Hello,

I have fix the problem for me. What helps me was the kernel commandline parameter initcall_debug. Then I follow the error to the function imx6sx_qos_init in the file arch/arm/mach-imx/mach-imx6sx.c. The error was caused by the statemt writel_relaxed(0, src_base+0x1400); and below. I removed that statements and then then my kernel boots.

Regards

   Robert

[igorpadykov]

Hi Roberto

if board based on mx6sxsabresd, why you did not use imx6sx-sdb-reva-ldo.dts ?

Note it is preferably to start with ldo enable option.

Also had you changed ddr settings obtained from

calibration results in uboot/../mx6sx_17x17_arm2/plugin.S

(imx6sx_17x17_ddr3_evk_ddr_setting)

DDR size should be configured in uboot/include/configs/mx6sx_arm2.h,

if you chooses imx6sx-sdb configs then mx6sxsabresd.h.

Additionally one can try to decrease cma size (imx_v7_defconfig

CONFIG_CMA_SIZE_MBYTES=xxx) and use ip=none in kernel boot parameter,

decrease "mem" parameter (refer to attached Release Notes Table 11. Common

kernel boot parameters). Check Table 20. Known issues and workarounds for i.MX 6SoloX

"CSI and LCD cannot be used simultaneously"

Best regards

igor

-----------------------------------------------------------------------------------------------------------------------

Note: If this post answers your question, please click the Correct Answer button. Thank you!

-----------------------------------------------------------------------------------------------------------------------

[robyf]

Hi Igor,

I did a custom uboot board. We are using 2xDDR3 MT41K128M16JT-125 items, attached you will find the imximage.cfg which I'm not really sure is fine.

Regarding the DTSs I've attached it so you can see the changes, mostly everything is disabled.

In the meantime I will give a try to CONFIG_CMA_SIZE_MBYTES setting.

[igorpadykov]

Hi Roberto

I would suggest to try mx6sxsabresd, imx6sx-sdb-reva-ldo.dts .

~igor

[robyf]

Hi Igor,

already tried the mx6sxsabresd, but doesn't work. Our board is very minimal compared to the mx6sxsabresd and currently we have such pin mapping:

GROUP	SLIC Signal	Schematic Signal	Board Signal	PAD IOMUX Register	IOMUX Settings	CPU Signal Becomes
PCM	PCLK	IMX_PCLK	SD3_DATA1	IOMUXC_SW_MUX_CTL_PAD_SD3_DATA1	ALT3	audmux.AUD6_TXC
PCM	FSYNC	FSYNC	SD3_DATA2	IOMUXC_SW_MUX_CTL_PAD_SD3_DATA2	ALT3	audmux.AUD6_TXFS
PCM	DTX	SRXD	SD3_DATA0	IOMUXC_SW_MUX_CTL_PAD_SD3_DATA0	ALT3	audmux.AUD6_RXD
PCM	DRX	DRX	SD3_DATA3	IOMUXC_SW_MUX_CTL_PAD_SD3_DATA3	ALT3	audmux.AUD6_TXD
SPI	SCLK	QSPI1B_SS1_B	QSPI1B_SS1_B	IOMUXC_SW_MUX_CTL_PAD_QSPI1B_SS1_B	ALT3	ecspi5.SCLK
SPI	CS#	QSPI1B_DQS	QSPI1B_DQS	IOMUXC_SW_MUX_CTL_PAD_QSPI1B_DQS	ALT3/ALT5	ecspi5.SS0/GPIO4_IO28
SPI	SDO	QSPI1A_SS1_B	QSPI1A_SS1_B	IOMUXC_SW_MUX_CTL_PAD_QSPI1A_SS1_B	ALT3	ecspi5.MISO
SPI	SDI	QSPI1A_DQS	QSPI1A_DQS	IOMUXC_SW_MUX_CTL_PAD_QSPI1A_DQS	ALT3	ecspi5.MOSI
Control	RST#	GPIO1_IO11	GPIO1_IO11	IOMUXC_SW_MUX_CTL_PAD_GPIO1_IO11	ALT5	GPIO1_IO11
Control	INT#	GPIO1_IO12	GPIO1_IO12	IOMUXC_SW_MUX_CTL_PAD_GPIO1_IO12	ALT5	GPIO1_IO12

I've also tried to boot the kernel with CONFIG_CMA_SIZE_MBYTES=32 but doesn't change anything.

[igorpadykov]

Hi Roberto

I would suggest to try *ldo.dts .

For customization support one also can apply to Professional Services

Professional Engineering Services|Freescale

~igor

[robyf]

Hi Igor,

I've tried the *ldo.dts you suggested, but doesn't work.

[igorpadykov]

Hi Roberto

then one can try to run minimal image.

~igor

[robyf]

Hi Igor,

sure I will do! Can you suggest one I can dd straight to my microSD?

Cheers,

Roberto Fichera.

[igorpadykov]

IMX6_L3.14.28_MFG_TOOL  : IMX6_L3.14.28_MFG_TOOL

L3.14.28_1.0.0_iMX6SX_BUNDLE : i.MX 6SoloX Linux Binary Demo Files

also one can try uart_from_osc as kernel boot parameter

~igor

[robyf]

Hi Igor,

I've used fsl-image-gui-x11-imx6sxsabresd.sdcard but still doesn't boot the kernel! See below:

U-Boot 2014.04 (Mar 21 2015 - 01:28:45)

CPU:   Freescale i.MX6SX rev1.2 at 792 MHz

CPU:   Temperature 44 C, calibration data: 0x5ba52269

Reset cause: POR

Board: MX6SX SABRE SDB

I2C:   ready

DRAM:  1 GiB

MMC:   FSL_SDHC: 0, FSL_SDHC: 1, FSL_SDHC: 2

Display: Hannstar-XGA (1024x768)

Video: 1024x768x18

gis input --- No input

In:    serial

Out:   serial

Err:   serial

Found PFUZE200! deviceid 0x1, revid 0x21  (?!?!)

mmc0 is current device

Net:   Phy not found

PHY reset timed out

FEC0

Warning: failed to set MAC address

Normal Boot

Hit any key to stop autoboot:  0

=> setenv mmcroot /dev/mmcblk1p2 rootwait rw uart_from_osc

=> boot

mmc0 is current device

reading boot.scr

** Unable to read file boot.scr **

reading zImage

5930352 bytes read in 277 ms (20.4 MiB/s)

Booting from mmc ...

reading imx6sx-sdb-reva-ldo.dtb

47064 bytes read in 21 ms (2.1 MiB/s)

Kernel image @ 0x80800000 [ 0x000000 - 0x5a7d70 ]

## Flattened Device Tree blob at 83000000

   Booting using the fdt blob at 0x83000000

   Using Device Tree in place at 83000000, end 8300e7d7

Starting kernel ...

.... freeze ....

[igorpadykov]

Hi Roberto

do you have ethernet on board, if not

had you removed FEC driver from linux build ?

~igor

[robyf]

Hi Igor,

yes there is one! The board doesn't have any thing than just one ethernet, one eMMC, one microSD and one SLIC connected wired as reported above.

[igorpadykov]

then I think that it is not working, from original U-Boot log:

"

Net:   Phy not found

PHY reset timed out

FEC0

Warning: failed to set MAC address"

you shoud set mac address and make it to work so these errors will not appear.

Alternatively one can exclude it temporarily from linux build.

~igor

[robyf]

Hi Igor,

the error come from u-boot-imx. But I've also notice another thing regarding PFUZE!

My uboot wrongly reports

Found PFUZE100! deviceid 0x11, revid 0x21

But the fsl-image-gui-x11-imx6sxsabresd

Found PFUZE200! deviceid 0x1, revid 0x21

[marcocavallini]

Roberto,

I suggest you to set up your own u-boot adapted to your hardware as Igor suggested you above, in case remove all the hw devices not present or not yet configured.

then try to boot your own kernel using the DT suggested again by Igor.

The root filesystem image is irrilevant at this point of the development until you reach the init. but core-image-minimal would be smaller and easier to manage.

Just my 2 cents.

Cordiali Saluti / Kindest Regards / Mit freundlichen Grüßen

--

Marco Cavallini | KOAN sas | Bergamo - Italia

embedded and real-time software engineering

Phone:+39-035-255.235 - Fax:+39-178-22.39.748

[robyf]

Hi Marco,

I've my custom uboot board derived from the mx6sxsabresd because the design started from it. Uboot has been adapted to remove the pinmux conflict against the SLIC wiring. In the DTS I've disabled mostly all the peripherals which are not needed at all.

[igorpadykov]

Hi Roberto

imx6sx-17x17-arm2 board uses pfuse100 while mx6sxsabresd - pfuse200.

In general I would suggest to pay attention to ethernet bring-up and

create new thread for that.

Best regards

igor

[robyf]

Hi Igor,

Regarding pfuse100 vs pfuse200, well it is supposed to be a pfuse200, at least this is what the schematic shown.

I'll cross check with our HW engineer regarding that.

Back to the ethernet bring-up problem, I will open in case a new thread after cross-checking some details with our HW engineer, anyway the item is a LAN8720A.

Cheers,

Roberto Fichera.

[robyf]

Hi Igor,

just had some time to look again to this and I've fixed the MAC problem as well as the PFUSE which now is detected correctly.

U-Boot 2014.04-imx_v2014.04_3.14.28_1.0.0_ga+g88123ea (Aug 28 2015 - 15:38:25)

CPU:   Freescale i.MX6SX rev1.2 at 792 MHz

CPU:   Temperature 49 C, calibration data: 0x5c553569

Reset cause: POR

Board: Domus iMX6SX (ID:e301dab51823b1d4)

I2C:   ready

DRAM:  512 MiB

MMC:   FSL_SDHC: 0, FSL_SDHC: 1

*** Warning - bad CRC, using default environment

In:    serial

Out:   serial

Err:   serial

Found PFUZE200! deviceid 0x1, revid 0x21

mmc0 is current device

Net:   FEC0

Warning: failed to set MAC address

Normal Boot

Hit any key to stop autoboot:  0

=> bdinfo

arch_number = 0x00000000

boot_params = 0x80000100

DRAM bank   = 0x00000000

-> start    = 0x80000000

-> size     = 0x20000000

eth0name    = FEC0

ethaddr     = (not set)

current eth = FEC0

ip_addr     = <NULL>

baudrate    = 115200 bps

TLB addr    = 0x9FFF0000

relocaddr   = 0x9FF66000

reloc off   = 0x18766000

irq_sp      = 0x9DF63EE0

sp start    = 0x9DF63ED0

=> setenv mmcroot /dev/mmcblk1p2 rootwait rw earlyprintk mem=384M

=> save

Saving Environment to MMC...

Writing to MMC(0)... done

=> boot

mmc0 is current device

reading boot.scr

** Unable to read file boot.scr **

reading zImage

5934680 bytes read in 277 ms (20.4 MiB/s)

Booting from mmc ...

reading domus-reva-ldo.dtb

47642 bytes read in 20 ms (2.3 MiB/s)

Kernel image @ 0x80800000 [ 0x000000 - 0x5a8e58 ]

## Flattened Device Tree blob at 83000000

   Booting using the fdt blob at 0x83000000

   Using Device Tree in place at 83000000, end 8300ea19

Starting kernel ...

Uncompressing Linux... done, booting the kernel.

Booting Linux on physical CPU 0x0

Linux version 3.14.28-1.0.0_ga+g91cf351 (roberto@dragon.tekno-soft.it) (gcc version 4.9.2 (GCC) ) #1 SMP PREEMPT Fri Aug 28 14:58:41 CEST 2015

CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c53c7d

CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache

Machine model: Voneus i.MX6 SoloX Domus Board

bootconsole [earlycon0] enabled

cma: CMA: reserved 32 MiB at 96000000

Memory policy: Data cache writealloc

PERCPU: Embedded 8 pages/cpu @95cca000 s8896 r8192 d15680 u32768

Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 97536

Kernel command line: console=ttymxc0,115200 root=/dev/mmcblk1p2 rootwait rw earlyprintk mem=384M

PID hash table entries: 2048 (order: 1, 8192 bytes)

Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)

Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)

Memory: 342144K/393216K available (7213K kernel code, 394K rwdata, 6196K rodata, 328K init, 428K bss, 51072K reserved, 0K highmem)

Virtual kernel memory layout:

    vector  : 0xffff0000 - 0xffff1000   (   4 kB)

    fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)

    vmalloc : 0x98800000 - 0xff000000   (1640 MB)

    lowmem  : 0x80000000 - 0x98000000   ( 384 MB)

    pkmap   : 0x7fe00000 - 0x80000000   (   2 MB)

    modules : 0x7f000000 - 0x7fe00000   (  14 MB)

      .text : 0x80008000 - 0x80d20678   (13410 kB)

      .init : 0x80d21000 - 0x80d732c0   ( 329 kB)

      .data : 0x80d74000 - 0x80dd6ac0   ( 395 kB)

       .bss : 0x80dd6acc - 0x80e41d9c   ( 429 kB)

SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1

Preemptible hierarchical RCU implementation.

        RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=1.

RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1

NR_IRQS:16 nr_irqs:16 16

L310 cache controller enabled

l2x0: 16 ways, CACHE_ID 0x410000c8, AUX_CTRL 0x32030000, Cache size: 256 kB

...then lots of garbage!!!

I've also executed some mtest under uboot:

DDR3 Device Information
Memory type:	DDR3
Manufacturer:	Micron
Memory part number:	MT41K128M16JT-125
Density per chip select (Gb):	4
Number of Chip Selects per channel used	1
Total DRAM density (Gb)	4
Number of ROW Addresses	14
Number of COLUMN Addresses	10
Number of BANKS	8
Bus Width (input 16, 32, or 64 bits)	32
Clock Cycle Freq (MHz)	400
Clock Cycle Time (ns)	2.5

Booting this uboot, a straight mtest freeze immediately without an iteration

Normal Boot

Hit any key to stop autoboot:  0

=> mtest

Testing 80000000 ... a0000000:

Pattern 00000000  Writing... <freeze>

Trying to reduce the tested range like below:

Normal Boot

Hit any key to stop autoboot:  0

=> mtest 0x80000000 0x9e000000

Testing 80000000 ... 9e000000:

Pattern 00000000  Writing...  Reading...

Mem error @ 0x9DF63D80: found 9FF849E0, expected 077D8F60

Mem error @ 0x9DF63D84: found 9FF8499C, expected 077D8F61

Mem error @ 0x9DF63D88: found 00000000, expected 077D8F62

Mem error @ 0x9DF63D8C: found 9FF7C3E0, expected 077D8F63

Mem error @ 0x9DF63D90: found 9DF63D8C, expected 077D8F64

Mem error @ 0x9DF63D94: found 9FF748B4, expected 077D8F65

Mem error @ 0x9DF63D9C: found 077D8F65, expected 077D8F67

Pattern FFFFFFFF  Writing...

reports some errors than freeze, but works with this range:

Hit any key to stop autoboot:  0

=> mtest 0x80000000 0x9d000000

Testing 80000000 ... 9d000000:

Pattern 00000000  Writing...  Reading...Iteration:     500

I assume this might depend by the PCB layout below?

TLB addr    = 0x9FFF0000

relocaddr   = 0x9FF66000

irq_sp      = 0x9DF63EE0

sp start    = 0x9DF63ED0

Any thought where to look at?

Thanks in advance,

Roberto Fichera.