Diagnose Mfgtool Bad Data CRC

[simonr99]

I'm setting up Mfgtool re-flashing of my Hummingboard i.MX6Q system. I've built a mfgtool U-boot, kernel and rootfs using Yocto.

When I boot over OTG USB using Mfgtool my target sometimes successfully boots into the linux kernel. Most attempts fail with:

## Loading init Ramdisk from Legacy Image at 12c00000 ...    
Image Name:   fsl-image-mfgtool-initramfs-soli    
Image Type:   ARM Linux RAMDisk Image (gzip compressed)    
Data Size:    88443107 Bytes = 84.3 MiB    
Load Address: 00000000    
Entry Point:  00000000    
Verifying Checksum ... Bad Data CRC 
Ramdisk image is corrupt or invalid

Because of the intermittent nature of the failure I suspected DDR calibration problems. So I've run the DDR Stress Tester with the settings used by my Mfgtool U-boot. The stress test passed without needing calibration data changes.

If I modify my ucl2.xml to load the initramfs file into memory twice in different locations, and then run cmp in U-boot, the data is different. Run cmp again and I get the same difference. So the error occurred during the write to RAM, not the read.

If I do a cp in u-boot to copy a large block of data and then a cmp to compare the memory areas, they are the same.

Can I then conclude that my errors occur during USB data transfer? 

Can anyone suggest how to diagnose further, or what could be the problem?

Thanks

Simon

[igorpadykov]

Hi Simon

one can try to decrease image size or check imx_v7_mfg_defconfig
CONFIG_BLK_DEV_RAM_SIZE mentioned in
http://stackoverflow.com/questions/36387516/size-constraints-of-initramfs-on-arm

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[simonr99]

Igor
Thanks for your suggestion, but CONFIG_BLK_DEV_RAM_SIZE affects the Kernel. My problem is seen in U-boot.
For example, if I get the mfgtool to load the rootfs twice into RAM:

<CMD state="BootStrap" type="load" file="firmware/fsl-image-mfgtool-initramfs.cpio.gz.u-boot" address="0x12C00000"
loadSection="OTH" setSection="OTH" HasFlashHeader="FALSE" ifdev="MX6Q">Loading Initramfs1.</CMD>
<CMD state="BootStrap" type="load" file="firmware/fsl-image-mfgtool-initramfs.cpio.gz.u-boot" address="0x20000000"
loadSection="OTH" setSection="OTH" HasFlashHeader="FALSE" ifdev="MX6Q">Loading Initramfs2.</CMD>

and then compare the two data blocks:

cmp.b 0x12C00000 0x20000000 5000000

U-Boot > cmp.b 0x12C00000 0x20000000 5000000
byte at 0x13963680 (0x21) != byte at 0x20d63680 (0xbe)
Total of 14038656 byte(s) were the same

The location of the difference will change every time I load the data over USB, and sometimes the data matches!

[igorpadykov]

Hi Simon

could you try to load some small (<1MB) binary file with patterns (like all zeros, all "1", e.t.c)

instead fsl-image-mfgtool-initramfs.cpio.gz and check it. If difference persist this may point to

ddr errors or board supply issues.

Best regards
igor

[simonr99]

I tried loading files of different sizes and investigating the error rate. An 8Mbyte file will nearly always load correctly. An 80Mbyte nearly always gets corrupted. 

I then realised that my mfgtool-initramfs file is far larger than average (80MBytes - the examples are < 8MBytes) and so I fixed this in my Yocto layers (caused by specifying packages globally in local.conf).

My mfgtool-initramfs is now 7MBytes, so usually loads without errors. This is a work-around rather than a fix - there is still an error rate.