Ethernet issues on T4240QDS

[vigneshwarensan]

I am working on a T4240QDs board. When I boot into the system, the eth0 is not generated.

I checked the dmesg dump and found out the following error messages

[	1.751442] fsl_mac: mac.c:417:mac_load() fsl_mac: FSL FMan MAC API based driver ()
[	1.758122] fsl_mac ffe4e0000.ethernet: of_get_mac_address(/soc@ffe000000/fman@400000/ethernet@e0000) failed
[	1.766663] fsl_mac: probe of ffe4e0000.ethernet failed with error -22
[	1.771981] fsl_mac ffe4e8000.ethernet: of_get_mac_address(/soc@ffe000000/fman@400000/ethernet@e8000) failed
[	1.780518] fsl_mac: probe of ffe4e8000.ethernet failed with error -22
[	1.785914] fsl_mac ffe5e8000.ethernet: of_get_mac_address(/soc@ffe000000/fman@500000/ethernet@e8000) failed
[	1.794452] fsl_mac: probe of ffe5e8000.ethernet failed with error -22
[	1.799911] fsl_dpa: dpaa_eth.c:4304:dpa_load() fsl_dpa: FSL DPAA Ethernet driver ()
[	1.806813] fsl_dpa ethernet.16: dev_get_drvdata(ffe4e8000.ethernet) failed
[	1.812483] fsl_dpa: probe of ethernet.16 failed with error -22
[	1.817219] fsl_dpa ethernet.17: dev_get_drvdata(ffe5e8000.ethernet) failed
[	1.822888] fsl_dpa: probe of ethernet.17 failed with error -22

I booted into the U-boot prompt and tried to set the IP address

I set a suitable IP address(10.116.65.38) netmask(255.255.255.0) gatewayip(10.116.65.1)

The error that I got was:

*** ERROR: `eth1addr' not set

FM1@DTSEC5 Waiting for PHY auto negotiation to complete......... TIMEOUT !

FM1@DTSEC5: No link.

ping failed; host 10.116.65.38 is not alive

I checked the board infor structure:

=> bdinfo 

memstart    = 0x00000000

memsize     = 0x200000000

flashstart  = 0xE0000000

flashsize   = 0x08000000

flashoffset = 0x00000000

sramstart   = 0x00000000

sramsize    = 0x00000000

immr_base   = 0xFE000000

bootflags   = 0x00000000

intfreq     = 1666.667 MHz

busfreq     = 666.667 MHz

addressing  = 36-bit

ethaddr     = (not set)

IP addr     = 10.116.65.32

baudrate    = 115200 bps

relocaddr   = 0x7FF30000

Core voltage= 1043 mV

XVDD        = 1.50 V

SYSCLK      = 66.67 MHz

DDRCLK      = 66.67 MHz

RCW source  = I2C extended addressing

HRESET_REQ  = Reset system

Clearly ethaddr was not set.

I am trying to figure out a way to recover the MAC address and I am not able to. Where and how do I retrieve my MAC address.

Thanks in advance.

I have attached my environment variables:

=> print

baudrate=115200

bdev=sda3

bootargs=root=/dev/mmcblk0p1 rw rootdelay=5 console=ttyS0,115200

bootcmd=setenv bootargs root=/dev/mmcblk0p1 rw rootdelay=5 console=$consoledev,$baudrate;mmcinfo;ext2load mmc 0:1 $loadaddr $bootfile;ext2load mmc 0:1 $fdtaddr $fdtfile; bootm $loadaddr - $fdtaddr

bootdelay=10

bootfile=uImage

c=ffe

consoledev=ttyS0

ethact=FM1@DTSEC5

ethprime=FM1@DTSEC1

fdtaddr=0x17200000

fdtfile=uImage-t4240qds.dtb

filesize=0x3df080

fman_ucode=0xeff40000

gatewayip=10.116.65.1

hwconfig=fsl_ddr:ctlr_intlv=3way_4KB,bank_intlv=auto;usb1:dr_mode=host,phy_type=utmi

ipaddr=10.116.65.32

loadaddr=0x10000000

netdev=eth0

netmask=255.255.255.0

nfsboot=setenv bootargs root=/dev/nfs rw nfsroot=$serverip:$rootpath ip=$ipaddr:$serverip:$gatewayip:$netmask:$hostname:$netdev:off console=$consoledev,$baudrate $othbootargs;tftp $loadaddr $bootfile;tftp $fdtaddr $fdtfile;bootm $loadaddr - $fdtaddr

ramboot=setenv bootargs root=/dev/ram rw console=$consoledev,$baudrate $othbootargs;tftp $ramdiskaddr $ramdiskfile;tftp $loadaddr $bootfile;tftp $fdtaddr $fdtfile;bootm $loadaddr $ramdiskaddr $fdtaddr

ramdiskaddr=0x02000000

ramdiskfile=t4240qds/ramdisk.uboot

rootpath=/opt/nfsroot

serverip=10.116.65.38

stderr=serial

stdin=serial

stdout=serial

tftpflash=tftpboot $loadaddr $uboot && protect off $ubootaddr +$filesize && erase $ubootaddr +$filesize && cp.b $loadaddr $ubootaddr $filesize && protect on $ubootaddr +$filesize && cmp.b $loadaddr $ubootaddr $filesize

uboot="u-boot.bin"

ubootaddr=0xeff80000

[yipingwang]

Is FMAN microcode uploaded in u-boot?

Would you please also share your u-boot setup log?

[vigneshwarensan]

Thank you for the reply!! My U-boot log is:

U-Boot 2013.01 (Jun 12 2013 - 16:13:31), Build: jenkins-york-fsl-u-boot-119

CPU0:  T4240E, Version: 1.0, (0x82480010)

Core:  E6500, Version: 1.0, (0x80400010)

Clock Configuration:

       CPU0:1666.667 MHz, CPU1:1666.667 MHz, CPU2:1666.667 MHz, CPU3:1666.667 MHz, 

       CPU4:1666.667 MHz, CPU5:1666.667 MHz, CPU6:1666.667 MHz, CPU7:1666.667 MHz, 

       CPU8:1666.667 MHz, CPU9:1666.667 MHz, CPU10:1666.667 MHz, CPU11:1666.667 MHz, 

       CCB:666.667 MHz,

       DDR:800  MHz (1600 MT/s data rate) (Asynchronous), IFC:166.667 MHz

       FMAN1: 444.444 MHz

       FMAN2: 333.333 MHz

       QMAN:  333.333 MHz

       PME:   333.333 MHz

L1:    D-cache 32 kB enabled

       I-cache 32 kB enabled

Reset Configuration Word (RCW):

       00000000: 14180019 0c101914 00000000 00000000

       00000010: 04383063 30548c00 1c020000 1d000000

       00000020: 00000000 ee0000ee 00000000 000307fc

       00000030: 00000000 00000000 00000000 00000020

Board: T4240QDS, Sys ID: 0x1e, Sys Ver: 0x24, vBank: 0

FPGA: v3 (T4240QDS_2012_1113_1114), build 438 on Tue Nov 13 17:14:23 2012

SERDES Reference Clocks: SERDES1=125MHz SERDES2=125MHz SERDES3=100MHz SERDES4=100MHz 

I2C:   ready

SPI:   ready

DRAM:  Initializing....using SPD

Detected UDIMM 9JSF25672AZ-2G1K1 

Detected UDIMM 9JSF25672AZ-2G1K1 

Detected UDIMM 18KSF51272AZ-1G6K1

Not all DIMMs are identical. Memory controller interleaving disabled.

6 GiB left unmapped

    DDR: 8 GiB (DDR3, 64-bit, CL=11, ECC on)

Flash: 128 MiB

L2:    2048 KB enabled

enable l2 for cluster 1 fec60000

enable l2 for cluster 2 feca0000

Corenet Platform Cache: 1536 KB enabled

Using SERDES1 Protocol: 1 (0x1)

Using SERDES2 Protocol: 28 (0x1c)

Using SERDES3 Protocol: 6 (0x6)

Using SERDES4 Protocol: 12 (0xc)

SRIO1: enabled

SRIO2: disabled

NAND:  512 MiB

MMC:  FSL_SDHC: 0

EEPROM: CRC mismatch (752b828e != db2d5536)

PCIe1: Root Complex, no link, regs @ 0xfe240000

PCIe1: Bus 00 - 00

PCIe2: disabled

PCIe3: Root Complex, no link, regs @ 0xfe260000

PCIe3: Bus 01 - 01

PCIe4: disabled

In:    serial

Out:   serial

Err:   serial

Net:   Fman1: Uploading microcode version 106.4.6

Fman2: Uploading microcode version 106.4.6

FM1@DTSEC5, FM2@DTSEC5

[yipingwang]

You need to configure MAC addresses in u-boot environment, otherwise Kernel will report errors which you encountered, please refer to the following configuration from T4240 demo board.

eth10addr=00:e0:0c:00:ae:0a

eth11addr=00:e0:0c:00:ae:0b

eth12addr=00:e0:0c:00:ae:0c

eth13addr=00:e0:0c:00:ae:0d

eth14addr=00:e0:0c:00:ae:0e

eth15addr=00:e0:0c:00:ae:0f

eth1addr=00:e0:0c:00:ae:01

eth2addr=00:e0:0c:00:ae:02

eth3addr=00:e0:0c:00:ae:03

eth4addr=00:e0:0c:00:ae:04

eth5addr=00:e0:0c:00:ae:05

eth6addr=00:e0:0c:00:ae:06

eth7addr=00:e0:0c:00:ae:07

eth8addr=00:e0:0c:00:ae:08

eth9addr=00:e0:0c:00:ae:09

ethact=e1000#0

ethaddr=00:e0:0c:00:ae:00

[vigneshwarensan]

Thank you! It worked perfectly.

And further when I boot the board and set the Ip address using:

ifconfig fm2-mac5 inet 10.116.65.8 netmask 255.255.255.0

and set the default gateway with:

route add default gw 10.116.65.1

I am not able to ping a machine with IP 10.116.65.38 in the same network.

Thanks in advance.

EDIT: I had raised the question of why the board had 16 variables, I checked the T4240 manual and figured that it has 16, 1Gbps Ethernet MACs. Thanks.

[yipingwang]

Please check whether you could ping gateway successfully,

The dtb file also defines 16 Ethernet MACs, probably they are not enabled simultaneously, this is decided by RCW.

[vigneshwarensan]

I am not able to ping the default gateway. We suspect it's IP tunneling, should we consider that to be the issue?

[yipingwang]

Please check whether you could use FM2@DTSEC5 in u-boot.

=> setenv ethact FM2@DTSEC5

=>setenv ipaddr 10.116.65.8

=>ping 10.116.65.38

In Linux, please try to use "ethtool" to check the ethernet port status.

Please check your network environment with your PC.

[vigneshwarensan]

Thank you!! I have tried out various methods to mitigate this error, and still the ping utility within the kernel is not able to find my host machine.

Within u-boot when I ping my host machine, the prompt ends with:

Using FM2@DTSEC5 device

host 10.116.65.38 is alive

In addition I have been able to download a kernel from my host machine and boot it, using tftp.

and ethact IS set to:

=> printenv ethact

ethact=FM2@DTSEC5

However when I set an IP to fm2-mac5 interface the network doesnt seem to be reachable.

Here is the ouput of my ifconfig

root@t4240qds:~# ifconfig fm2-mac5

fm2-mac5  Link encap:Ethernet  HWaddr 00:04:9f:02:e5:a7 

          inet addr:10.116.65.8  Bcast:10.116.65.255  Mask:255.255.255.0

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:0 errors:0 dropped:0 overruns:0 frame:0

          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

          Memory:fe5e8000-fe5e8fff

And  my IP table, before and after adding the default IP:

root@t4240qds:~# route

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

10.116.65.0     *               255.255.255.0   U     0      0        0 fm2-mac5

root@t4240qds:~# route add default gw 10.116.65.1

root@t4240qds:~# route

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

default         10.116.65.1     0.0.0.0         UG    0      0        0 fm2-mac5

10.116.65.0     *               255.255.255.0   U     0      0        0 fm2-mac5

And the ping output:

root@t4240qds:~# ping 10.116.65.1

PING 10.116.65.1 (10.116.65.1): 56 data bytes

64 bytes from 10.116.65.8: Destination Host Unreachable

Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst Data

4  5  00 0054 0000   2 0000  40  01 a3b8 10.116.65.8  10.116.65.1

64 bytes from 10.116.65.8: Destination Host Unreachable

We have an additional T4240 board and with the same configuration this seems to be the issue in both the boards. Is there anything I am missing? Or is any of these methods wrong?

(As far as the netowrk is concerned, I attached the ethernet to another machine and was able to ping my host, with the same IP configuration)

I checked the Quick Start Guide, and in the bot log, once udevd was started, udeved proceeded to rename interface eth3 till eth7.. No such log was found in the my bootlog..

[yipingwang]

Did you use Kernel and dtb images from the the pre-built image ISO?

Would you please attach your Kernel boot log?

[vigneshwarensan]

I am using a kernel built from the qoriq sdk framework. I also tried this on the kernel that is there in the NOR flash image that shipped with the system. Nevertheless the problem persisted in both the kernels.

I am attaching the bootlog of both these kernels

This log is the kernel from the yocto build:

T4240QDS Yocto kernel Bootlog - Pastebin.com

This log is from the kernel in the NOR flash, that arrived with the system

U-Boot 2013.01 (Jun 12 2013 - 16:13:31), Build: jenkins-york-fsl-u-boot-119 C - Pastebin.com

Thanks

[yipingwang]

No problem in the Kernel log, what is the status of ethtool.

#ifcofig fm2-mac5 10.116.65.8

#ping 10.116.65.38

#ethtool fm2-mac5

[vigneshwarensan]

root@t4240qds:~# ifconfig fm2-mac5 10.116.65.8 netmask 255.255.255.0    

root@t4240qds:~# ifconfig fm2-mac5 up

root@t4240qds:~# route add default gw 10.116.65.1

root@t4240qds:~# route

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

default         10.116.65.1     0.0.0.0         UG    0      0        0 fm2-mac5

10.116.65.0     *               255.255.255.0   U     0      0        0 fm2-mac5

root@t4240qds:~# ping 10.116.65.38

PING 10.116.65.38 (10.116.65.38): 56 data bytes

64 bytes from 10.116.65.8: Destination Host Unreachable

Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst Data

4  5  00 0054 0000   2 0000  40  01 a393 10.116.65.8  10.116.65.38

64 bytes from 10.116.65.8: Destination Host Unreachable

root@t4240qds:~# ping 10.116.65.1

PING 10.116.65.1 (10.116.65.1): 56 data bytes

64 bytes from 10.116.65.8: Destination Host Unreachable

Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst Data

4  5  00 0054 0000   2 0000  40  01 a3b8 10.116.65.8  10.116.65.1

Ethtool interface:

root@t4240qds:~# ethtool fm2-mac5

Settings for fm2-mac5:

  Supported ports: [ MII ]

  Supported link modes:   10baseT/Half 10baseT/Full

                         100baseT/Half 100baseT/Full

                         1000baseT/Full

  Supported pause frame use: No

  Supports auto-negotiation: Yes

  Advertised link modes:  10baseT/Half 10baseT/Full

                         100baseT/Half 100baseT/Full

                         1000baseT/Full

  Advertised pause frame use: No

  Advertised auto-negotiation: Yes

  Speed: 1000Mb/s

  Duplex: Full

  Port: MII

  PHYAD: 1

  Transceiver: external

  Auto-negotiation: on

  Current message level: 0xffffffff (-1)

        drv probe link timer ifdown ifup rx_err tx_err tx_queued intr tx_done rx_status pktdata hw wol 0xffff8000

  Link detected: yes

root@t4240qds:~#

[yipingwang]

Please try whether this u-boot parameter would be helpful

hwconfig=fsl_ddr:ctlr_intlv=3way_4KB,bank_intlv=auto;fsl_fm1_xaui_phy:xfi;fsl_fm2_xaui_phy:xfi;usb1:dr_mode=host,phy_type=utmi

[vigneshwarensan]

Thank you.

I verified the Documentation and the dts file. The two 1Gbps interfaces on the board are of PHY type: RGMII

The above setting didnt make a difference and didn't work.

Can I modify it to:

fsl_ddr:ctlr_intlv=3way_4KB,bank_intlv=auto;fsl_fm1_xaui_phy:rgmii;fsl_fm2_xaui_phy:rgmii;usb1:dr_mode=host,phy_type=utmi

and test it out?

I have also attached the DTS file and the USDPAA dts file, I use. Just to make sure everything is in place.

DTS file

http://git.freescale.com/git/cgit.cgi/ppc/sdk/linux.git/tree/arch/powerpc/boot/dts/t4240qds.dts

USDPAA DTS file

sdk/linux.git - Freescale PowerPC Linux Tree

Thanks

[vigneshwarensan]

Hi,

From the u-boot prompt I am able to ping my host machine, but my host machine is not able to ping my board. And further the RX bytes, once the kernel is loaded is always 0 and don't seem to be accepting any packets. I enabled the CONFIG_FSL_DPAA_ETH_DEBUGFS variable within the kernel and I found no hardware errors. I did check the MAC's set in the uboot environment and everything seems fine. Is there any issues in the RCW that I need to change?

[yipingwang]

I think you should have SDK image ISO QorIQ-SDK-V1.4-PPCE6500-IMAGE-20130625-yocto.iso.

Please try to use Kernel image, dtb file and filesystem in t4240qds folder like the following.

uImage-t4240qds.bin

uImage-t4240qds.dtb

fsl-image-core-t4240qds.ext2.gz.u-boot

[vigneshwarensan]

I tried pinging the IP while performing a tftp and I was able to ping the board.

I also tried booting the machine with the pre-built images that was in the SDK. The uImage-t4240qds.bin, uImage-t4240qds.dtb and ram filesystem, fsl-image-core-t4240qds.ext2.gz.u-boot. The network still didnt respond.

I analysed the dts file

sdk/linux.git - Freescale PowerPC Linux Tree

I found that 4 MDIO slots were disabled t4240mdio2:mdio@20, mdio@40. mdio@60, mdio@80 and also the fman_oh2, fman_ph3, ... , fman1_oh6 options were also set to the status disabled.

I went ahead and removed the disable statement. The resulting kernel had a series of mdio errors and the network still didn't work out.

mdio_bus ffe5fc000: MDIO read error

mdio_bus ffe5fc000: MDIO read error

mdio_bus ffe5fc000: MDIO read error

mdio_bus ffe5fc000: MDIO read error

mdio_bus ffe5fc000: MDIO read error

mdio_bus ffe5fc000: MDIO read error

mdio_bus ffe5fc000: MDIO read error

Is the dts faulty?

[hwrobel]

if you can ping the board under tftp, then the HW is ok. You didn’t specify over which board connector the ping went.

I assume you have used the TOP Ethernet RJ45 connector, because that would be fairly common.

The top connector is fm1-mac5 under the standard SDK image, not fm2-mac5.

[vigneshwarensan · Accepted solution]

Hello,

This is the solution to the issue that I have raised. And this issue is currently resolved.

The FMAN u code (version 106.4.6) is the issue. I updated the FMAN ucode from the version that arrived in the board (106.4.6) to 106.4.9

Version 106.4.9+ would work on the board and I was able ping my network.

The SDK documentation had the method to perform this:

The NOR flash on the DPAA board has two flash banks

Bank switching can be done in U-Boot using the following statements:
• Switch to bank 0: reset
• Switch to alternate bank: qix altbank

Programming a new ucode

0xebf40000 is the location of ucode in the alternate bank.

=>tftp 1000000 <ucode_file_name>.bin

=>erase ebf40000 ebf5ffff

=>cp.b 1000000 ebf40000 $filesize

=>qix altbank

[hwrobel]

You cannot ping U-Boot unless U-Boot is at the same trying to do a network operation.

If you want to check if U-Boot can be ping’ed, start, e.g., a dummy tftp that will take a while to time out with an error. While tftp is hoping for a response, you can ping U-Boot. If you can ping the external world though, then the other way will work, too.

Note that success in U-Boot means that all physical elements are ok. If, e.g., Linux fails then, you have an issue in the configuration of the device tree or driver.