MPC8308 :: fw_setenv crashes and corrupts U-Boot environment

[Nich]

Hi all,

I have a problem with fw_setenv utility on MPC8308.

Problem has been noticed in software update procedure where we need to change some U-Boot environment variables from bash script. It appeared that fw_setenv sometimes fails when it is called in script with other file operations with flash.

Following script reproduce this problem. It copies file from one NAND partition to another and then sets some value to dummy U-Boot variable.

#!/bin/sh

echo "Starting fw_setenv stress test..."

rm -rf /mnt/data/testdir

mkdir /mnt/data/testdir 

x=1

while [ $x -le 1000 ]

do

    cp /usr/bin/synce-ptp-init.sh /mnt/data/testdir/file$x

    fw_setenv dummy "dummy-$x"

    echo "fw_setenv executed $x times"

    x=$(( $x + 1 ))

done

echo "Test succeded"

Typical output looks like that

...

fw_setenv executed 453 times

fw_setenv executed 454 times

fw_setenv executed 455 times

fw_setenv executed 456 times

Oops: Machine check, sig: 7 [#1]

MPC830x RDB

Modules linked in:

NIP: c01c6990 LR: c01c7b2c CTR: c001c6c8

REGS: c7063cf0 TRAP: 0200 Not tainted (2.6.29.6-rt23)

MSR: 00049030 <EE,ME,IR,DR> CR: 42000442 XER: 20000000

TASK = c78628c0[7056] 'fw_setenv' THREAD: c7062000

GPR00: 00000000 c7063da0 c78628c0 c79cc7a4 00060706 00000024 d1960361 00000001

GPR08: 00000001 c9080000 24002022 c0020000 001f648c 1001d6ac 10018010 10010000

GPR16: 00000000 00000000 c7063dc8 c0310000 c7063da8 c001ba28 00000000 00006f6f

GPR24: c7941818 c7941834 c79417e0 00000000 00000001 00060706 00010813 c79cc7a4

Call Trace:

[c7063da0] [c01c7b50] 0xc01c7b50 (unreliable)

[c7063e10] [c01c7e64] 0xc01c7e64

[c7063e80] [c01c0a50] 0xc01c0a50

[c7063e90] [c01c2b74] 0xc01c2b74

[c7063ef0] [c0075f28] 0xc0075f28

[c7063f10] [c0076440] 0xc0076440

[c7063f40] [c001141c] 0xc001141c

--- Exception: c01 at 0xff606ec

  LR = 0x10001644

Instruction dump:

41860024 2f800004 419e005c 419a0040 4186001c 7c031a78 7c630034 5463d97e

4e800020 8123000c 7c09222e 8123000c <7c69222e> 7c031a78 7c630034 5463d97e

---[ end trace 7560118e3d077417 ]---

Erase at 0x00ce0000 failed immediately: errno -5

Erase at 0x00ce0000 failed immediately: errno -5

JFFS2: marking eraseblock at 00ce0000

as bad<4>nand_bbt: Error while writing bad block table -5

Similar problem was described here. Unfortunately there was no solution except a workaround that doesn't work for us.

I have already tried to build fw_setenv from latest U-Boot release. Doesn't help.

Looks like something is wrong with MTD driver... I feel that we need to update kernel... But I would like to find some simple solution.

[Nich · Accepted solution]

We have finally found the root of problem.

According to "MPC8308 Chip Errata" document there is a known bug with NAND flash. It is listed as "eLBC-A001: Simultaneous FCM and GPCM or UPM operation may erroneously trigger bus monitor timeout".

Document is also providing information how to implement workaround for this issue. We have found a patch for recent kernels and made similar changes in our kernel. My test script now works fine and our firmware update procedure too.

[Nich]

I haven't found solution yet. Ideally I think I need to try recent kernel, but unfortunately there is no support for MPC8308, so it looks necessary to apply some patches there. And I think it could take a lot of time. So at the moment I decided to try workaround. I have enabled redundant environment in U-Boot and so now it is not lost after crash. And device can continue to work after reboot.

[ChrisLaRocque]

Hello Nikolay

I solved this problem last fall with an mpc8308rdb. Here is what I did to solve it.

I'm using buildroot 2013.02 and booting from the NOR flash. Here are my mtd partitions.

mtdparts=physmap-flash.0:0x60000@0x0000(U-Boot),0x10000@0x60000(Env1),0x10000@0x70000(Env2),0x260000@0xA0000(Linux),0x4F0000@0x300000(Ramdisk),-(dtb_and_vsc)

mtd0 is the uboot partition and the environment and backup are at mtd1 and mtd2. They are copies of each other and are maintained automatically by by the uboot env utils.

The file /etc/fw_env.config contains the locations and sizes for the environment partitions.

# Configuration file for fw_(printenv/saveenv) utility.

# Up to two entries are valid, in this case the redundant

# environment sector is assumed present.

# Notice, that the "Number of sectors" is ignored on NOR and SPI-dataflash.

# Futhermore, if the Flash sector size is ommitted, this value is assumed to

# be the same as the Environment size, which is valid for NOR and SPI-dataflash

# NOR example

# MTD device name    Device offset    Env. size    Flash sector size    Number of sectors

/dev/mtd1               0x0000          0x2000          0x10000

/dev/mtd2        0x0000        0x2000        0x10000

The shell script below modifies the ip address but you can use it to learn how to do what you need.

#!/bin/bash

# ************************************************

# Created: 4/18/2013

# By Chris LaRocque

# Measurement Specialties

#

# -------------------------------------------------

# Accepts either of two patterns and modifies

# the u-boot environmental parameter;

#

# setramargs

#

# May also modify the parameters

#

# ipaddr

# netmask

#

# These modifications are carried out using the u-boot tools

#

#

# fw_printenv

# fw_setenv

#

# located in the /usr/sbin directory

  if [ "dynamic" == "$1" ]; then

    # read the parameter dynamic_setramargs using fw_printenv and write it back out using fw_setenv

    #

    # but that paramter will return in the form

    #

    # dynamic_setramargs=setenv bootargs root=/dev/ram ramdisk_size=$ramdisksize rw ip=:::::$netdev:dhcp console=$consoledev,$baudrate $mtd_parts

    #

    # We need to set the parameter setramargs to the value of dynamic_setramargs

    #

    # To do this we need to use single quotes to frame the value of dynamic_setramargs in the function call to

    # fw_setenv.

    #

    # sed, the stream editor does this nicely using regular expressions.

    #

    # enclosing the shell commands in $() executes them and writes the output to the indicated variable.

    #

    # So we read this as:

    #

    # Execute the application fw_printenv passing it the parameter dynamic_setramargs

    #

    # pipe the output to the command sed

    #

    # sed parses that value:

    #

    # First remove the string "dynamic_setramargs=" and replace it with the string "setramargs '"

    #

    # The output is them piped to sed a second time

    #

    # Now place a trailing "'" on the string

    #

    # The output is piped to sed a third time

    #

    # Now the command "fw_setenv " is placed at the beginning of the string.

    str=$(fw_printenv dynamic_setramargs | sed "s/\(dynamic_setramargs=\)\.*/setramargs '\2/" | sed "s/$/'/" | sed "s/^.*/fw_setenv &/")

    echo "$str"

  elif [ "static" == "$1" ] && [ -n "$2" ] && [ -n "$3" ]; then

  # The static ip is a little more involved.

      # OK the correct key is present

      # Get the ip= key and netmask= key

      ip_key=$(echo "$2" | sed "s/\(ip=\).*/\1/")

      netmask_key=$(echo "$3" | sed "s/\(netmask=\).*/\1/")

      if [ "ip=" == "$ip_key" ] && [ "netmask=" == "$netmask_key" ]; then

      # the parameter keys are present

      # get the ipaddress and netmask

      # check that they are formatted correctly and in range.

      ipadd=$(echo "$2" | sed "s/\(ip=\)\.*/\2/" | egrep '\b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b')

      subnetmask=$(echo "$3" | sed "s/\(netmask=\)\.*/\2/" | egrep '\b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b')

      if [ -n "$ipadd" ] && [ -n "$subnetmask" ]; then

          # the ip address and subnet mask are formatted correctly

          # hope they're correct.

          str=$(fw_printenv static_setramargs | sed "s/\(static_setramargs=\)\.*/setramargs '\2/" | sed "s/$/'/" | sed "s/^.*/fw_setenv &/")

          echo "$str"

          str=$(echo "$ipaddr" | sed "s/^.*/fw_setenv ipaddr &/"))

          echo "$str"

          str=$(echo "$subnetmask" | sed "s/^.*/fw_setenv netmask &/"))

          echo "$str"

      fi

      fi

  else

    # print the usage info

    str= %0" usage: cmd [param1] [param2]\n"\

      "command list\n"\

      "dynamic - sets the unit for dhcp\n"\

      "static ip=www.xxx.yyy.zzz mask=www.xxx.yyy.zzz - sets the unit to ip address and subnet mask\n"

    echo "$str"

  fi

Regards.

Chris LaRocque

[Nich]

Hi Chris,

Thanks for your comment. Unfortunately I haven't understood how this scripts solves the issue. Do you want to say that fw_setenv doesn't crash when you call it from this script?

You wrote "I'm using buildroot 2013.02". Is it U-Boot release?

Could you clarify this things please?

[ChrisLaRocque]

Hello Nikolai

Yes, I will clarify.

1> I am working with an mpc8308rdb, a development board available from freescale.

2> Instead of using ltib, I use a similar tool called buildroot <google is your friend> to create: u-boot version 2012.02, u-boot tools, linux kernel 3.6.11, the root file system, and all of the other components which are necessary to boot the board / processor and create my project.

3> The problem you are experiencing is likely caused by errors in the the "/etc/fw_env.config" file in your file system.

So, what I was saying was:

Here is a copy of my fw_env.config file. It defines the location of the env mtd partitions on the nor flash chip of my board for fw_printenv and fw_setenv.

# NOR example

# MTD device name    Device offset    Env. size    Flash sector size    Number of sectors

/dev/mtd1               0x0000          0x2000          0x10000

/dev/mtd2        0x0000        0x2000        0x10000

Here is the mtdparts parameter which is passed to the kernel on the command line. It defines the partitions to the kernel so that they can be found from user space by the fw_printenv and fw_setenv

mtdparts=physmap-flash.0:0x60000@0x0000(U-Boot),0x10000@0x60000(Env1),0x10000@0x70000(Env2),0x260000@0xA0000(Linux),0x4F0000@0x300000(Ramdisk),-(dtb_and_vsc)

Once you have understood how these two items work together then the script can be used as a template to do anything you want with the environmental variables.

Your reported problem "corruption of the environmental partitions" is one which I experienced. I was able to read the variables using fw_printenv but when I tried to modify them the incorrect definition of the partition location in the fw_env.config file caused fw_print to write to the wrong area in flash.

Regards

C

[Nich]

Ok. Now I understand. You had different problem. My problem is that fw_setenv crashes sometimes. Not always. And mostly it happens when I run it from script.

Anyway thank you for your comments.