Custom i.MX6SX board freeze after enabling L310 cache controller

[robyf]

We have developed a new PCB starting from mx6sxsabresd design. It currently passes 12h of calibration test without problem,

so the board design should be ok.

The board boot straight with u-boot-imx and it's able to at least boot the first kernel stage until it stops with some printed garbage, the log is below.

It has a custom DTS slight changed from imx6sx-17x17-arm2.dts.

Any idea how to debug it?

Cheers,

Roberto Fichera.

U-Boot 2014.04-imx_v2014.04_3.14.28_1.0.0_ga+g88123ea (Jul 08 2015 - 17:57:57)

CPU:   Freescale i.MX6SX rev1.2 at 792 MHz

CPU:   Temperature 41 C, calibration data: 0x5ba52269

Reset cause: POR

Board: Domus iMX6SX (ID:e301dab51902b1d4)

I2C:   ready

DRAM:  512 MiB

MAX7322 Not found

MMC:   FSL_SDHC: 0, FSL_SDHC: 1

*** Warning - bad CRC, using default environment

In:    serial

Out:   serial

Err:   serial

Found PFUZE100! deviceid 0x11, revid 0x21

mmc0 is current device

Net:   Phy not found

PHY reset timed out

FEC0

Warning: failed to set MAC address

Normal Boot

Hit any key to stop autoboot:  0

=> bdinfo

arch_number = 0x00000000

boot_params = 0x80000100

DRAM bank   = 0x00000000

-> start    = 0x80000000

-> size     = 0x20000000

eth0name    = FEC0

ethaddr     = (not set)

current eth = FEC0

ip_addr     = <NULL>

baudrate    = 115200 bps

TLB addr    = 0x9FFF0000

relocaddr   = 0x9FF66000

reloc off   = 0x18766000

irq_sp      = 0x9DF63EE0

sp start    = 0x9DF63ED0

=> setenv mmcroot /dev/mmcblk3p2 rootwait rw earlyprintk

=> saveenv

Saving Environment to MMC...

Writing to MMC(0)... done

=> boot

mmc0 is current device

mmc0 is current device

reading boot.scr

** Unable to read file boot.scr **

reading zImage

5934672 bytes read in 278 ms (20.4 MiB/s)

Booting from mmc ...

reading domus-reva.dtb

48167 bytes read in 19 ms (2.4 MiB/s)

Kernel image @ 0x80800000 [ 0x000000 - 0x5a8e50 ]

## Flattened Device Tree blob at 83000000

    Booting using the fdt blob at 0x83000000

    Using Device Tree in place at 83000000, end 8300ec26

switch to ldo_bypass mode!

Starting kernel ...

Uncompressing Linux... done, booting the kernel.

Booting Linux on physical CPU 0x0

Linux version 3.14.28-1.0.0_ga+g91cf351 (roberto@dragon.tekno-soft.it) (gcc version 4.9.2 (GCC) ) #1 SMP PREEMPT Fri Jul 3 12:43:06 CEST 2015

CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c53c7d

CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache

Machine model: Voneus i.MX6 SoloX Domus ARM2 Board

bootconsole [earlycon0] enabled

cma: CMA: reserved 320 MiB at 8c000000

Memory policy: Data cache writealloc

PERCPU: Embedded 8 pages/cpu @8bb3a000 s8896 r8192 d15680 u32768

Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 130048

Kernel command line: console=ttymxc0,115200 root=/dev/mmcblk3p2 rootwait rw earlyprintk

PID hash table entries: 2048 (order: 1, 8192 bytes)

Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)

Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)

Memory: 176704K/524288K available (7213K kernel code, 394K rwdata, 6196K rodata, 328K init, 428K bss, 347584K reserved, 0K highmem)

Virtual kernel memory layout:

     vector  : 0xffff0000 - 0xffff1000   (   4 kB)

     fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)

     vmalloc : 0xa0800000 - 0xff000000   (1512 MB)

     lowmem  : 0x80000000 - 0xa0000000   ( 512 MB)

     pkmap   : 0x7fe00000 - 0x80000000   (   2 MB)

     modules : 0x7f000000 - 0x7fe00000   (  14 MB)

       .text : 0x80008000 - 0x80d20678   (13410 kB)

       .init : 0x80d21000 - 0x80d732c0   ( 329 kB)

       .data : 0x80d74000 - 0x80dd6ac0   ( 395 kB)

        .bss : 0x80dd6acc - 0x80e41d9c   ( 429 kB)

SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1

Preemptible hierarchical RCU implementation.

         RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=1.

RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1

NR_IRQS:16 nr_irqs:16 16

L310 cache controller enabled

l2x0: 16 ways, CACHE_ID 0x410000c8, AUX_CTRL 0x32030000, Cache size: 256 kB

*** GARBAGE START HERE ***

[igorpadykov]

HI Roberto

suggest to run

i.MX6 DDR Stress Test Tool V1.0.3

find new calibration koefficients, then put them

in uboot/..mx6sxsabresd/imximage.cfg and rebuild uboot

Best regards

igor

[robyf]

Hi Igor,

I'm going to use the DDR Stress tester DDR_Stress_Tester_V1.0.3_UART1_for_SDboot&JTAG.zip and currently it puts some garbage at startup. This triggered me the idea that something is wrong with UART1 setup:

Normal Boot

Hit any key to stop autoboot:  0

=> fatload mmc 0:1 0x907000 ddr-stress-test-mx6sx.bin

reading ddr-stress-test-mx6sx.bin

70420 bytes read in 21 ms (3.2 MiB/s)

=> go 0x907000

## Starting application at 0x00907000 ...

▒<▒▒▒<▒▒▒<▒▒▒<▒▒▒<▒<8x▒▒▒<▒<8p▒▒▒<▒<8x▒▒▒<▒<8x▒▒▒<▒<8x▒▒▒<▒<8x▒▒▒<▒<8x▒▒▒<▒<8x▒▒▒<▒<8x▒▒▒<▒<▒▒▒<p▒<8x▒▒▒▒<p▒<8▒<▒▒<?▒<?▒<▒▒<▒<▒<▒<▒▒<▒▒▒<▒▒<8▒<▒<?x▒▒<▒▒▒▒<▒▒<▒<▒▒<▒▒▒<<▒<▒▒▒▒▒<▒▒▒<▒▒<▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒<▒▒▒▒▒▒▒▒▒<▒▒<▒▒<?▒<?▒<▒<▒▒<▒▒▒<▒<▒<▒▒<▒<?▒<▒▒<▒▒<▒▒<▒▒<?▒<▒▒<▒▒<▒<8x▒▒<?

So far so good I've added uart_from_osc to kernel command line and finally I got a freeze later in pinctrl setup. This is ok, I can check the DTS to see who is causing the problem, however this is showing some problem to the UART1 setup within either DCD or kernel side.

Any idea how to fix this at least to run the DDR Stress test from uboot?

Thanks in advance,

Roberto Fichera.

=> boot

mmc0 is current device

reading boot.scr

** Unable to read file boot.scr **

reading zImage

5934672 bytes read in 277 ms (20.4 MiB/s)

Booting from mmc ...

reading domus-reva-ldo.dtb

40696 bytes read in 19 ms (2 MiB/s)

Kernel image @ 0x80800000 [ 0x000000 - 0x5a8e50 ]

## Flattened Device Tree blob at 83000000

   Booting using the fdt blob at 0x83000000

   Using Device Tree in place at 83000000, end 8300cef7

Starting kernel ...

Uncompressing Linux... done, booting the kernel.

Booting Linux on physical CPU 0x0

Linux version 3.14.28-1.0.0_ga+g91cf351 (roberto@dragon.tekno-soft.it) (gcc version 4.9.2 (GCC) ) #1 SMP PREEMPT Fri Aug 28   18:32:02 CEST 2015

CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c53c7d

CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache

Machine model: Voneus i.MX6 SoloX Domus Board

bootconsole [earlycon0] enabled

cma: CMA: reserved 32 MiB at 9e000000

Memory policy: Data cache writealloc

PERCPU: Embedded 8 pages/cpu @9dbcd000 s8896 r8192 d15680 u32768

Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 130048

Kernel command line: console=ttymxc0,115200 root=/dev/mmcblk1p2 rootwait rw uart_from_osc earlyprintk

PID hash table entries: 2048 (order: 1, 8192 bytes)

Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)

Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)

Memory: 472212K/524288K available (7213K kernel code, 394K rwdata, 6196K rodata, 328K init, 428K bss, 52076K reserved, 0K h  ighmem)

Virtual kernel memory layout:

    vector  : 0xffff0000 - 0xffff1000   (   4 kB)

    fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)

    vmalloc : 0xa0800000 - 0xff000000   (1512 MB)

    lowmem  : 0x80000000 - 0xa0000000   ( 512 MB)

    pkmap   : 0x7fe00000 - 0x80000000   (   2 MB)

    modules : 0x7f000000 - 0x7fe00000   (  14 MB)

      .text : 0x80008000 - 0x80d20678   (13410 kB)

      .init : 0x80d21000 - 0x80d732c0   ( 329 kB)

      .data : 0x80d74000 - 0x80dd6ac0   ( 395 kB)

       .bss : 0x80dd6acc - 0x80e41d9c   ( 429 kB)

SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1

Preemptible hierarchical RCU implementation.

        RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=1.

RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1

NR_IRQS:16 nr_irqs:16 16

L310 cache controller enabled

l2x0: 16 ways, CACHE_ID 0x410000c8, AUX_CTRL 0x32030000, Cache size: 256 kB

Switching to timer-based delay loop

sched_clock: 32 bits at 3000kHz, resolution 333ns, wraps every 1431655765682ns

clocksource_of_init: no matching clocksources found

Console: colour dummy device 80x30

Calibrating delay loop (skipped), value calculated using timer frequency.. 6.00 BogoMIPS (lpj=30000)

pid_max: default: 32768 minimum: 301

Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)

Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)

CPU: Testing write buffer coherency: ok

CPU0: thread -1, cpu 0, socket 0, mpidr 80000000

Setting up static identity map for 0x806dd3f8 - 0x806dd450

Brought up 1 CPUs

SMP: Total of 1 processors activated.

CPU: All CPU(s) started in SVC mode.

devtmpfs: initialized

VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 4

pinctrl core: initialized pinctrl subsystem

regulator-dummy: no parameters

NET: Registered protocol family 16

DMA: preallocated 256 KiB pool for atomic coherent allocations

cpuidle: using governor ladder

cpuidle: using governor menu

Use WDOG1 as reset source

syscon 20c8000.anatop: regmap [mem 0x020c8000-0x020c8fff] registered

vdd1p1: 800 <--> 1375 mV at 1100 mV

vdd3p0: 2800 <--> 3150 mV at 3000 mV

vdd2p5: 2100 <--> 2875 mV at 2500 mV

cpu: 725 <--> 1450 mV at 1150 mV

vddpcie-phy: 725 <--> 1450 mV

vddsoc: 725 <--> 1450 mV at 1175 mV

syscon 20e4000.iomuxc-gpr: regmap [mem 0x020e4000-0x020e7fff] registered

syscon 21bc000.ocotp-ctrl: regmap [mem 0x021bc000-0x021bffff] registered

syscon 21ac000.romcp: regmap [mem 0x021ac000-0x021affff] registered

[igorpadykov]

HI Roberto

one can try to build uboot with uart clock generated with

CCM_CSCDR1, uart_clk_sel =1, this is equivalent to usage

uart_from_osc in kernel command line.

Best regards

igor

[robyf]

Hi Igor,

I've collected again the DDR3 calibration values, still the same problem.

Regarding uart_clk_sel=1 under u-boot, it seems that setting it to 1 doesn't have any effect to ddr-stress-test-mx6sx.bin execution, still getting garbage after running it.

[robyf]

Hi Igor,

I've fixed basically all problems and now the PCB boot pretty straight! The problem was caused by not having disabled the gpu entry in my custom DTS.

There still the uboot version of DDR3 calibration which still printing garbage due to serial console setup I guess.

[robertlehmann]

Hello Roberto,

I have a similiar problem that my kernel will not boot until

syscon 21ac000.romcp: regmap [mem 0x021ac000-0x021affff] registered

Could you please explain what you have changed that your board boot?

Thanks a lot.

Robert

[robyf]

Hi Roberto,

Was quite easy, I've disabled the GPU from the kernel setup.

Cheers,

Roberto Fichera.

[robertlehmann]

Hi Roberto,

did you disable it or did you delete all entries from your dts which are related to Graphic units? The delete of the GPU entries doesn't work for me.

     Robert

[robyf]

Hi Robert,

you should disable the GPU from your kernel config too.

Cheers,

Roberto Fichera.

[robertlehmann]

Hello Roberto,

I disabled all the graphic support from the kernel, but it doesn't work for me. Maybe there is a problem with the ldo_bypass settings. I'll check it

  Robert

[robertlehmann]

Hello Roberto,

one more question. Did you although remove the gpu support from the imx6sx.dtsi file?

  Robert

[robyf]

Hi Robert,

Is not necessary, just disable it in your custom DTS and you are done.

[florian_germain]

Hi Roberto,

I use imx6sabresd machine config on a custom board using imx6sx 17x17 no PCIe, no GPU, no LVDS and no VADC and i get your original problem.

My board crash at kernel loading:

Booting Linux on physical CPU 0x0

Linux version 3.14.28-1.0.0_ga-mfgtool+g91cf351 (florian@ubuntu) (gcc version 4.9.1 (GCC) ) #1 SMP PREEMPT Mon Nov 30 11:53:35 CET 2015

CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c5387d

CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache

Machine model: Freescale i.MX6 SoloX SDB Board

bootconsole [earlycon0] enabled

cma: CMA: reserved 160 MiB at 96000000

Memory policy: Data cache writealloc

PERCPU: Embedded 8 pages/cpu @95b8e000 s8896 r8192 d15680 u32768

Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 130048

Kernel command line: console=ttymxc2,115200 earlyprintk=serial

PID hash table entries: 2048 (order: 1, 8192 bytes)

Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)

Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)

Memory: 328104K/524288K available (5634K kernel code, 252K rwdata, 1680K rodata, 268K init, 386K bss, 196184K reserved, 0K highmem)

Virtual kernel memory layout:

    vector  : 0xffff0000 - 0xffff1000  (  4 kB)

    fixmap  : 0xfff00000 - 0xfffe0000  ( 896 kB)

    vmalloc : 0xa0800000 - 0xff000000  (1512 MB)

    lowmem  : 0x80000000 - 0xa0000000  ( 512 MB)

    pkmap  : 0x7fe00000 - 0x80000000  (  2 MB)

    modules : 0x7f000000 - 0x7fe00000  (  14 MB)

      .text : 0x80008000 - 0x8072cd14  (7316 kB)

      .init : 0x8072d000 - 0x807702c0  ( 269 kB)

      .data : 0x80772000 - 0x807b10e0  ( 253 kB)

      .bss : 0x807b10ec - 0x808119f0  ( 387 kB)

SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1

Preemptible hierarchical RCU implementation.

        RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=1.

RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1

NR_IRQS:16 nr_irqs:16 16

L310 cache controller enabled

l2x0: 16 ways, CACHE_ID 0x410000c8, AUX_CTRL 0x32030000, Cache size: 256 kB

And crash...

You say that i need to disable GPU in my kernel config. I think this is a good way to solve my crash. So i removed CONFIG_MXC_GPU_VIV.

Whar are other options to remove in kernel config?

I join to this post my defconfig file.

Thanks.

[florian_germain]

Adding 'uart_from_osc' in the kernel command line, i get also the kernel freeze at line:

"syscon 21ac000.romcp: regmap [mem 0x021ac000-0x021affff] registered"

[robyf]

Hi Florian,

I'm out of my office all today, tomorrow I can have a look to my config and I can also post it. In the meantime can you try to set CMA memory to 64MB or disable it? It might cause some problems.

[florian_germain]

Hi Roberto,

CMA memory was already reduced, but I tried with CMA memory to 64MB and this don't change anything.

It would be interesting for me to check differences between you config and mine.

[robyf]

Hi Florian,

I've attached my current config for kernel v3.14.28

[florian_germain]

Ok, it works now.

The problem was like for Roberto, that the gpu was not totaly remove from dts..

Thank you very much for your help.

[igorpadykov]

HI Roberto

one can try to connect jtag in DDR3 calibration tool, read uart frequency dividers, then

set the same in uboot, so when uboot loads DDR3 calibration tool, uart frequency remains the same.

Best regards

igor

[robyf]

Hi Igor.

ok! I'll check it.

Cheers,

Roberto Fichera.