Problems booting Linux on custom P4040 / P4080 -based board

chmod000 · ‎01-31-2012

Hello,

I'm currently in the process of bringing up a custom board containing a P4040 processor. I have U-boot running and can communicate with it via terminal and ethernet (via on-board Marvell PHYs). All of my software is based on the latest Freescale BSP. I'm currently having a hard time getting Linux to completely boot successfully. Near the end of the terminal output (just before the system halts) the output looks like garbled data (as shown below.) Interestingly, this garbled output is also witnessed when running this same code on the P4080DS. The difference, however, is that after a series of output like this, the Linux login prompt appears. On my system, it does not and the system just hangs leaving me wondering what the last attempted initialization was. Any suggestion on how to debug this problem is greatly appreciated!

Thanks,

Alex

__________________________________

=> bootm e8020000 e9300000 e8800000
WARNING: adjusting available memory to 30000000
## Booting kernel from Legacy Image at e8020000 ...
   Image Name:   System Builder/3.0.6/p4080ds
   Image Type:   PowerPC Linux Kernel Image (gzip compressed)
   Data Size:    3642776 Bytes = 3.5 MiB
   Load Address: 00000000
   Entry Point: 00000000
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at e9300000 ...
   Image Name:   devel-image-p4080ds-201201310122
   Image Type:   PowerPC Linux RAMDisk Image (gzip compressed)
   Data Size:    39019987 Bytes = 37.2 MiB
   Load Address: 00000000
   Entry Point: 00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at e8800000
   Booting using the fdt blob at 0xe8800000
   Uncompressing Kernel Image ... OK
   Loading Ramdisk to 2dac9000, end 2ffff5d3 ... OK
   Loading Device Tree to 03fe3000, end 03fff15d ... OK
WARNING: could not find compatible node fsl-usb2-dr: FDT_ERR_NOTFOUND.
Using P4080 DS machine description
Memory CAM mapping: 256/256/256 Mb, residual: 1280Mb
Linux version 3.0.6 (alexander@redhat1) (gcc version 4.5.2 (Sourcery G++ Lite 2011.03-38) ) #1 SMP Fri Jan 27 13:01:45 PST 2012
Found initrd at 0xedac9000:0xeffff5d3
No /soc@ffe000000/qman@318000 property 'fsl,qman-fqd', using memblock_alloc(0000000000200000)
No /soc@ffe000000/qman@318000 property 'fsl,qman-pfdr', using memblock_alloc(0000000001000000)
Qman ver:0a01,01,01
No /soc@ffe000000/bman@31a000 property 'fsl,bman-fbpr', using memblock_alloc(0000000001000000)
Bman ver:0a02,01,00
pme: No /soc@ffe000000/pme@316000 property 'fsl,pme-pdsr', using memblock_alloc(0x0000000001000000)
pme: No /soc@ffe000000/pme@316000 property 'fsl,pme-sre', using memblock_alloc(0x0000000000a00000)
USDPAA region at 60000000:10000000
CPU maps initialized for 1 thread per core
bootconsole [udbg0] enabled
setup_arch: bootmem
P4080 DS board from Freescale Semiconductor
arch: exit
Zone PFN ranges:
DMA      0x00000000 -> 0x00030000
Normal   empty
HighMem 0x00030000 -> 0x00080000
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
    0: 0x00000000 -> 0x00080000
MMU: Allocated 1088 bytes of context maps for 255 contexts
PERCPU: Embedded 10 pages/cpu @c17b1000 s18624 r8192 d14144 u40960
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 520192
Kernel command line:
PID hash table entries: 4096 (order: 2, 16384 bytes)
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Memory: 1709960k/2097152k available (7164k kernel code, 387192k reserved, 216k data, 438k bss, 284k init)
Kernel virtual memory layout:
* 0xfff5f000..0xfffff000 : fixmap
* 0xffc00000..0xffe00000 : highmem PTEs
* 0xffbfc000..0xffc00000 : early ioremap
* 0xf1000000..0xffbfc000 : vmalloc & ioremap
Hierarchical RCU implementation.
        RCU debugfs-based tracing is enabled.
NR_IRQS:512 nr_irqs:512 16
mpic: Setting up MPIC " OpenPIC " version 1.2 at ffe040000, max 4 CPUs
mpic: ISU size: 256, shift: 8, mask: ff
mpic: Initializing for 256 sources
clocksource: timebase mult[6aaaaab] shift[22] registered
Console: colour dummy device 80x25
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 512
mpic: requesting IPIs...
e500 family performance monitor hardware support registered
Brought up 4 CPUs
xor: measuring software checksum speed
   8regs     : 1240.000 MB/sec
   8regs_prefetch: 1213.000 MB/sec
   32regs    : 1274.000 MB/sec
   32regs_prefetch: 1135.000 MB/sec
xor: using function: 32regs (1274.000 MB/sec)
NET: Registered protocol family 16

PCI: Probing PCI hardware
bio: create slab <bio-0> at 0
raid6: int32x1    250 MB/s
raid6: int32x2    394 MB/s
raid6: int32x4    519 MB/s
raid6: int32x8    385 MB/s
raid6: using algorithm int32x4 (519 MB/s)
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
Bman err interrupt handler present
Bman portal (shared) initialised, cpu 1
Bman portal initialised, cpu 0
Bman portal (slave) initialised, cpu 2
Bman portal (slave) initialised, cpu 3
Bman: reserved bpid 0, seeded 256 items
Bman: reserved bpid 7
Bman: reserved bpid 8
Bman: reserved bpid 9
Bman portals initialised
Qman err interrupt handler present
Qman portal (shared) initialised, cpu 1
Qman portal initialised, cpu 0
Qman portal (slave) initialised, cpu 2
Qman portal (slave) initialised, cpu 3
Qman portals initialised
Switching to clocksource timebase
Switched to NOHz mode on CPU #2
Switched to NOHz mode on CPU #0
Switched to NOHz mode on CPU #3
Switched to NOHz mode on CPU #1
NET: Registered protocol family 2
IP route cache hash table entries: 32768 (order: 5, 131072 bytes)
TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
TCP: Hash tables configured (established 131072 bind 65536)
TCP reno registered
UDP hash table entries: 512 (order: 2, 16384 bytes)
UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
Trying to unpack rootfs image as initramfs...
rootfs image is not initramfs (no cpio magic); looks like an initrd
Freeing initrd memory: 38108k freed
Setting Freescale static PAMU/IOMMU configuration
Freescale PowerQUICC MII Bus: probed
mdio_bus mdio@ffe4e1120: /soc@ffe000000/fman@400000/mdio@e1120/p4080ds-mdio0 has invalid PHY address
mdio_bus mdio@ffe4e1120: /soc@ffe000000/fman@400000/mdio@e1120/p4080ds-mdio1 has invalid PHY address
mdio_bus mdio@ffe4e1120: /soc@ffe000000/fman@400000/mdio@e1120/p4080ds-mdio2 has invalid PHY address
mdio_bus mdio@ffe4e1120: /soc@ffe000000/fman@400000/mdio@e1120/p4080ds-mdio3 has invalid PHY address
Freescale P4080DS MDIO Bus: probed
Freescale P4080DS MDIO Bus: probed
audit: initializing netlink socket (disabled)
type=2000 audit(0.956:1): initialized
highmem bounce pool size: 64 pages
HugeTLB registered 4 MB page size, pre-allocated 0 pages
HugeTLB registered 16 MB page size, pre-allocated 0 pages
HugeTLB registered 64 MB page size, pre-allocated 0 pages
HugeTLB registered 256 MB page size, pre-allocated 0 pages
HugeTLB registered 1 GB page size, pre-allocated 0 pages
NTFS driver 2.1.30 [Flags: R/O].
JFFS2 version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
msgmni has been set to 1366
async_tx: api initialized (async)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
f��`�~�fx��ff�xfxf��x�fx��f��~��~��f��x��x��~��f~~�`�f~ff�`f�f��fx�~f`~��f��`xf�f~��xxf��f~��f��x�f�f~f��~x�~��怘�f~��f��~�`��`x��f��~��x��ff�fx��fx��~f~xf~�憞xffxf�x�f��f�xff��`fxf�x�~��~�~x�fx��f~f�f��~f�~�x`��f`��xf`~��f�f��f��~x�fxff�f��f��f��~��~��f��f��f�f��f`x��xx��x��f~��~�f��fxf~ff��fff��fff��`�ff��x~��`��f~�xf~�~x�f��fff~�xf�f~��xfx�x��fx�~�x��f`��x��f��怘�fx�xf�f��fx��f�xx�fff��x�`x��x��``fx�f��怘�x��ff~`f��~��f��fx��f�f~~��f��f��fx�ffx��~��~x�`ff�f��~��~�f�fffx��f��

scottwood · ‎08-15-2013

Add "console=ttyS0,115200" (assuming 115200 is the baud rate you're using) to your kernel command line.

pegasus711_knig · ‎11-02-2012

Hi People.

Im also trying to bring the P4080DS up with 2.6.34. The board was coming up nicely previously. Now as per my need, I needed to make the hardware backwards compatible so that older software can run on it. Well..specifically I needed my Data cache line size to be 32KB. So had to set the DCZ32 bit inside the L1CSR0 register.

Now Im doing this inside cpu_setup_fsl_booke.S right when the dcache is being set up. I check for this bit being set outside, and it shows that it is..until its time for the init process to start..whch the kernel does using a kernel_execve which is a quick architecture specific wrapper across code that, here in powerpc puts the sys call number for execve which is 11, into r0 and does a system call instruction..something it going wrong here or so I think because the prompt never appears and its hard to know what exactly is failing since the /sbin/init it refers to, is perhaps a precompiled binary lying on the NFS mounted rootfs (by the way how do i know if its using the one referred to there or perhaps it is looking for some other /sbin/init maybe??)

Im keen to hear hints or anything to at least help me look into the right direction..

Regards,

TomE · ‎01-31-2012

The last sensible thing it printed was:

> Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled

Then it looked like something changed the console baud rate.

> Any suggestion on how to debug this problem is greatly appreciated!

1 - Find the command or script or configuration that is changing the baud rate.

2 - Or attach an oscilloscope to the serial line and measure the bit width when it has changed. Derive the baud rate from that and then set the terminal to that to find out what it is printing when it fails.

3 - Or there's only so many baud rates, so simply try all of them and see which one starts printng sensibly where it would otherwise be printing "garbled".

4 - Once you know the rate it changes to, you could change U-Boot to use the same baud rate until you can find what part of the startup is changing it to the value you don't want.

Someone with more experience with this specific Linux setup should be able to tell you where this is being set and what to change to fix it.

Tom

chmod000 · ‎02-02-2012

Thanks for the response Tom!

It turns out that I had left one item in my device tree that referred to hardware that exists on the p4080ds but does not exist on my board. This is the reason why the boot process was halting and I never got to the login prompt.

The garbled data happens as soon as the call is made to platform_driver_register for the serial8250_isa_driver (in 8250.c). I'm not 100% certain why it happens though I should note that on one occasion it did not happen and I saw legible output. (This was confusing.)

When the system then turns off the boot console udbg0 just before the login prompt, I typically regain proper output.

It is likely that the baud rate is getting changed.

TomE · ‎02-03-2012

> on one occasion it did not happen and I saw legible output. (This was confusing.)

If that is happening then maybe the baud rate isn't changing, but the code that registers the driver might be resetting the chip WHILE it is still transmitting.

This might result in a "half a byte" being transmitted when this happens. The following data would now be out of sync. Adding an appropriate delay of about 1 character into whatver code is resetting the chip (platform_driver_register) would probably fix the problem. Any code resetting the chip should probably wait until it is empty before resetting it. There may be somewhere in a script file you can add a "sleep 0.1" or something to bodge this if you want to.

Tom

chmod000 · ‎02-06-2012

It does confuse me that I saw more output on one occasion. I inserted a delay to see if it was the half-byte condition but did not see a change in output.

I probed the output signals during the garbled output and witnessed pulses that implied a roughly 56000 Baud rate. I found a function that configures the uart divisor and managed to print the 'baud' variable to find that it was printing something like 57200. After inserting a little hack (for testing purposes only) that sets this baud variable to 115200, I discovered that the output during boot is no longer garbled and I am able to see the rest of the initialization output!

Baud rate change after all.

Thank you so much for your help.

--alex

simon_duboue · ‎02-09-2012

Hello Alex and Tom,

I am trying to do the same than Alex: my Linux is booting but I have some questions about some line in the code of Alex:

>mdio_bus mdio@ffe4e1120: /soc@ffe000000/fman@400000/mdio@e1120/p4080ds-mdio0 has invalid PHY address
>mdio_bus mdio@ffe4e1120: /soc@ffe000000/fman@400000/mdio@e1120/p4080ds-mdio1 has invalid PHY address
>mdio_bus mdio@ffe4e1120: /soc@ffe000000/fman@400000/mdio@e1120/p4080ds-mdio2 has invalid PHY address
>mdio_bus mdio@ffe4e1120: /soc@ffe000000/fman@400000/mdio@e1120/p4080ds-mdio3 has invalid PHY address

I have the same and I do not really know if it is normal...

I do not know if it is linked with the fact that I use the P4080DS in which the mdio bus is muxed.

Could you give me some suggestion?

Thank you:

Simon

chmod000 · ‎02-09-2012

Hi Simon,

As far as I can tell...

I'm not sure you can say that this is 'normal' but it is expected on the P4080DS based on the code found in of_mdio.c

The code was written in a strange way and it's just telling you that in the device tree the mdio child nodes called "p4080ds-mdio<0:3>" all have invalid PHY addresses. The loop goes on to register all child nodes (grandchildren nodes?) of each of those entries and will then discover the actual phy definitions in ethernet-phy@1x.

When it finds the actual PHYs, it will report:

mdio_bus p4080ds-mdio2@2: registered phy ethernet-phy at address 0

So basically, the code is properly discovering PHY definitions and registering their addresses but it's mistakenly telling you that a tree node that does not represent a phy has an invalid phy address.

You can ignore this warning or change the code to only report this when the 'reg' field actually exists.

ivanmercier · ‎08-14-2013

Hi Alexander,

I'm having a similar problem with my board,can you advise me please?

I'm trying to make my 1G Ethernet working on a qoriq p3041 custom board using a phy 1G Marvell 88E1118.

It works on uboot on my board and works on uboot and Linux on p3041ds eval board(which use a phy VItesse VSC8244XHG).

It appears that the problem comes from module fsl_pq_mdio

root@p3041ds:~# insmod /media/mmcblk0p1/fsl_pq_mdio.ko
libphy: Freescale PowerQUICC MII Bus: probed
libphy: mdio_mux: probed
Oops: Machine check, sig: 7 [#1]
SMP NR_CPUS=4 P3041 DS
Modules linked in: fsl_pq_mdio
NIP: c0016e94 LR: c0344848 CTR: 00000000
REGS: ee68ff10 TRAP: 0204 Not tainted (3.8.13-rt9-DEBUG-WP6-04558-gf4c757e-dirty)
MSR: 00029002 <CE,EE,ME> CR: 48002022 XER: 00000000
TASK = ea687570[29] 'kworker/u:1' THREAD: ea6c8000 CPU: 2
GPR00: fffffff4 ea6c9c70 ea687570 f1172009 f0541215 ea007b90 ffdf0000 0000fffd
GPR08: f0541215 ea007b90 00000000 00000000 22002024 00000000 c0056210 ea09fdec
GPR16: 00000000 00000000 00000000 eadaa9e0 c06bc988 c06bc964 c06bc928 c0783f90
GPR24: c06bc948 c06bc90c e78c3648 c066db20 00000008 e799add0 eadaa9d0 f1172009
NIP [c0016e94] ioread8+0x4/0x20
LR [c0344848] mdio_mux_mmioreg_switch_fn.part.1+0x38/0xc0
Call Trace:
[ea6c9c70] [c0344838] mdio_mux_mmioreg_switch_fn.part.1+0x28/0xc0 (unreliable)
[ea6c9c88] [c03444f4] mdio_mux_read+0x44/0xa0
[ea6c9ca8] [c0343880] mdiobus_read+0x50/0x80
[ea6c9cc0] [c034298c] get_phy_device+0xbc/0x100
[ea6c9d08] [c043f2b8] of_mdiobus_register.part.2+0xd8/0x210
[ea6c9d58] [c0344368] mdio_mux_init+0x198/0x240
[ea6c9d98] [c0344738] mdio_mux_mmioreg_probe+0x1b8/0x290
[ea6c9de8] [c02e2120] platform_drv_probe+0x20/0x30
[ea6c9df0] [c02e0290] really_probe+0x90/0x390
[ea6c9e10] [c02de270] bus_for_each_drv+0x70/0xb0
[ea6c9e38] [c02e0698] device_attach+0xc8/0xf0
[ea6c9e50] [c02df504] bus_probe_device+0xa4/0xc0
[ea6c9e68] [c02dfb80] deferred_probe_work_func+0xa0/0xf0
[ea6c9e88] [c004b6b0] process_one_work+0x170/0x4f0
[ea6c9eb8] [c004be54] worker_thread+0x154/0x360
[ea6c9ef0] [c00562a8] kthread+0x98/0xa0
[ea6c9f40] [c00102a0] ret_from_kernel_thread+0x64/0x6c
Instruction dump:
60000000 60000000 60000000 2f840005 38600002 4d9e0020 38600000 4e800020
00000000 00000000 00000000 7c0004ac <88630000> 0c030000 4c00012c 4c00018c
---[ end trace 001bd1cff2c97bfc ]---

When I'm trying to use the network interface with this module, I got:
root@p3041ds:~#ifconfig fm1-gb3 10.0.0.36
fsl_dpa ethernet.14 fm1-gb3: Could not connect to PHY /localbus@ffe124000/board-control@3,0/mdio-mux-emi1/rgmii-mdio@8/ethernet-phy@0
fsl_dpa ethernet.14 fm1-gb3: init_phy() = -19
SIOCSIFFLAGS: No such device

To me, this is a problem of phy address declared in device tree.

Am I right?
Does anybody know how to configure the dts to adapt it for a different phy?

attached, you can find my Linux boot log, my Linux config and the device tree source which is the original one.

Thanks a lot in advance.

Best Regards,

Ivan Mercier

simon_duboue · ‎03-06-2012

Well,

I finally manage to understand what you clearly explain to me chmod.

Could I know why are you interested in p4080?

What is the next step for you after booting Linux?