cannot erase 256MB NOR flash beyond first 139MB

[tbowen]

Background:

Recently, we upgraded the NOR flash on our board from a 64MB Spansion NOR flash to a 256MB Micron NOR flash part.  We encountered the same problem, documented here:

How can I get more vmalloc/ioremap space?

According to the kernel log (Linux-2.6.32.13), our kernel memory map and NOR-probe error looks like:

Placing 64MB software IO TLB between c0c03000 - c4c03000

software IO TLB at phys 0xc03000 - 0x4c03000

Zone PFN ranges:

  DMA      0x00000000 -> 0x00030000

  Normal   0x00030000 -> 0x00030000

  HighMem  0x00030000 -> 0x00080000

Movable zone start PFN for each node

early_node_map[1] active PFN ranges

    0: 0x00000000 -> 0x00080000

MMU: Allocated 1088 bytes of context maps for 255 contexts

PERCPU: Embedded 7 pages/cpu @c5c59000 s7688 r8192 d12792 u65536

pcpu-alloc: s7688 r8192 d12792 u65536 alloc=16*4096

pcpu-alloc: [0] 0 [0] 1

Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 520192

...

PID hash table entries: 4096 (order: 2, 16384 bytes)

Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)

Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)

Memory: 2006124k/2097152k available (6544k kernel code, 90156k reserved, 260k data, 155k bss, 240k init)

Kernel virtual memory layout:

  * 0xfffe0000..0xfffff000  : fixmap

  * 0xff800000..0xffc00000  : highmem PTEs

  * 0xff7d8000..0xff800000  : early ioremap

  * 0xf1000000..0xff7d8000  : vmalloc & ioremap

SLUB: Genslabs=13, HWalign=32, Order=0-3, MinObjects=0, CPUs=2, Nodes=1

...

vmap allocation for size 268439552 failed: use vmalloc=<size> to increase size.

of-flash e0000000.nor: Failed to ioremap() flash region

of-flash: probe of e0000000.nor failed with error -12

Like the above solution, we shifted the page-offset / task-size boundary in the kernel config:

#

# Advanced setup

#

CONFIG_ADVANCED_OPTIONS=y

CONFIG_LOWMEM_SIZE_BOOL=y

CONFIG_LOWMEM_SIZE=0x30000000

# CONFIG_LOWMEM_CAM_NUM_BOOL is not set

CONFIG_LOWMEM_CAM_NUM=3

# CONFIG_RELOCATABLE is not set

CONFIG_PAGE_OFFSET_BOOL=y

CONFIG_PAGE_OFFSET=0xa0000000

CONFIG_KERNEL_START_BOOL=y

CONFIG_KERNEL_START=0xa0000000

# CONFIG_PHYSICAL_START_BOOL is not set

CONFIG_PHYSICAL_START=0x00000000

CONFIG_PHYSICAL_ALIGN=0x04000000

CONFIG_TASK_SIZE_BOOL=y

CONFIG_TASK_SIZE=0xa0000000

CONFIG_NET=y

Which allows us to move past the above VMALLOC error.  The new memory map and MTD partition according to the kernel log is:  (We updated to SDK-v1.7 and kernel 3.12.19 and mtd-utils-1.5.1+, just to eliminate kernel errors.)

[    0.000000] MPC85xx RDB board from Freescale Semiconductor

arch: exit

[    0.000000] Zone ranges:

[    0.000000]   DMA      [mem 0x00000000-0x2fffffff]

[    0.000000]   Normal   empty

[    0.000000]   HighMem  [mem 0x30000000-0x7fffffff]

[    0.000000] Movable zone start for each node

[    0.000000] Early memory node ranges

[    0.000000]   node   0: [mem 0x00000000-0x7fffffff]

[    0.000000] MMU: Allocated 1088 bytes of context maps for 255 contexts

[    0.000000] PERCPU: Embedded 7 pages/cpu @b1c0f000 s7104 r8192 d13376 u32768

[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 522752

...

[    0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)

[    0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)

[    0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)

[    0.000000] Sorting __ex_table...

[    0.000000] Memory: 2005324K/2097152K available (5248K kernel code, 288K rwdata, 1484K rodata, 256K init, 1182K bss, 91828K reserved, 1310716K highmem)

[    0.000000] Kernel virtual memory layout:

[    0.000000]   * 0xfff5f000..0xfffff000  : fixmap

[    0.000000]   * 0xffc00000..0xffe00000  : highmem PTEs

[    0.000000]   * 0xffbfc000..0xffc00000  : early ioremap

[    0.000000]   * 0xe1000000..0xffbfc000  : vmalloc & ioremap

[    0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=2, Nodes=1

...

[    0.546971] e0000000.nor: Found 1 x16 devices at 0x0 in 16-bit bank. Manufacturer ID 0x000089 Chip ID 0x00227e

[    0.556990] Amd/Fujitsu Extended Query Table at 0x0040

[    0.562164]   Amd/Fujitsu Extended Query version 1.3.

[    0.567214] number of CFI chips: 1

[    0.570871] 8 ofpart partitions found on MTD device e0000000.nor

[    0.576884] Creating 8 MTD partitions on "e0000000.nor":

[    0.582197] 0x000000000000-0x000002700000 : "NOR (RO) SquashFS Root File System"

[    0.590451] ftl_cs: FTL header not found.

[    0.594962] 0x000002700000-0x000002800000 : "NOR (RO) 1MB Dead Space"

[    0.602230] ftl_cs: FTL header not found.

[    0.606707] 0x000002800000-0x000003300000 : "NOR (RW) UBI-FS Persistent Storage"

[    0.614922] ftl_cs: FTL header not found.

[    0.619416] 0x00000f300000-0x00000fe00000 : "NOR (RO) Raw Linux Kernel Image"

[    0.627367] ftl_cs: FTL header not found.

[    0.631876] 0x00000fe00000-0x00000ff40000 : "NOR (RO) 1.25 MB Dead Space"

[    0.639489] ftl_cs: FTL header not found.

[    0.644017] 0x00000ff40000-0x00000ff60000 : "NOR (RO) Raw Device Tree Data"

[    0.651387] ftl_cs: FTL header not found.

[    0.655895] 0x00000ff60000-0x00000ff80000 : "NOR (RO) Raw u-boot Environment Variables"

[    0.664275] ftl_cs: FTL header not found.

[    0.668778] 0x00000ff80000-0x000010000000 : "NOR (RO) Raw u-boot Bootloader Image"

[    0.676915] ftl_cs: FTL header not found.

Problem:

We can access all 256 MB via u-boot; however, in Linux user space, we cannot erase/write anything beyond 139MB of the 25MB NOR.  (The 139MB boundary was identified by spot-checking, like below.)  See:

$ mtd_debug erase /dev/mtd1 0 0x20000

Erased 131072 bytes from address 0x00000000 in flash

$ mtd_debug erase /dev/mtd4 0 0x20000

MEMERASE: Input/output error

Any suggestions?

Thanks!

Trevor

[vidyasagartata]

Let me provide the chip part no and which sdk is being used for the development.

[lunminliang]

Hi,

Was the size of Nor Flash in DTS file changed from 64M to 256M? Could refer to below 128M Nor Flash in T1040RDB.

ifc: localbus@ffe124000 {

                reg = <0xf 0xfe124000 0 0x2000>;

ranges = <0 0 0xf 0xe8000000 0x08000000

2 0 0xf 0xff800000 0x00010000

3 0 0xf 0xffdf0000 0x00008000>;

nor@0,0 {

#address-cells = <1>;

#size-cells = <1>;

compatible = "cfi-flash";

reg = <0x0 0x0 0x8000000>;

bank-width = <2>;

device-width = <1>;

                };

...

}

Have a great day,
Lunmin

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[tbowen]

@lunminliang: Yes, we already updated the device tree. ... I attached the complete device-tree source, but I'm also including here the most relevant portion.  If you or anyone sees any errors or suspicious things, please let me know. ... Thanks! ... Trevor

   localbus@ffe05000 {

        #address-cells = <2>;

        #size-cells = <1>;

        compatible = "fsl,p2020-elbc", "fsl,elbc", "simple-bus";

        reg = <0 0xffe05000 0 0x1000>;

        interrupts = <19 2>;

        interrupt-parent = <&mpic>;

        /* NOR Flash */

        ranges = <0x0 0x0 0x0 0xE0000000 0x10000000>;

        nor@0,0 {

            #address-cells = <1>;

            #size-cells = <1>;

            compatible = "cfi-flash";

            reg = <0x0 0x0 0x10000000>;

            /* 128Mx16x1 - 2 banks of 128M at 16-bit wide = 256M total */

            bank-width = <2>;

            device-width = <1>;

            partition@0 {

                /* 39 MB for SquashFS Root File System - Image #1 */

                reg = <0x00000000 0x02700000>;

                label = "NOR (RO) SquashFS Root File System";

                /* read-only; */

            };

            partition@02700000 {

                /* 01 MB for SquashFS Root File System - Image #2 */

                reg = <0x02700000 0x00100000>;

                label = "NOR (RO) 1MB Dead Space";

                /* read-only; */

            };

            partition@02800000 {

                /* 11 MB for JFFS2 persistent storage */

                reg = <0x02800000 0x00B00000>;

                label = "NOR (RW) UBI-FS Persistent Storage";

            };

            /* BIG GAP */

            partition@0F300000 {

                /* 11.00 MB for Linux Kernel #1 */

                reg = <0x0F300000 0x00B00000>;

                label = "NOR (RO) Raw Linux Kernel Image";

                /* read-only; */

            };

            partition@0FE00000 {

                /* 1.25 MB for Linux Kernel #2 */

                reg = <0x0FE00000 0x00140000>;

                label = "NOR (RO) 1.25 MB Dead Space";

                /* read-only; */

            };

            partition@0FF40000 {

                /* This location must not be altered  */

                /* 128 KB for Device Tree Blob */

                reg = <0x0FF40000 0x00020000>;

                label = "NOR (RO) Raw Device Tree Data";

                /* read-only; */

            };

            partition@0FF60000 {

                /* This location must not be altered  */

                /* 128 KB for u-boot environment variables */

                reg = <0x0FF60000 0x00020000>;

                label = "NOR (RO) Raw u-boot Environment Variables";

                /* read-only; */

            };

            partition@0FF80000 {

                /* This location must not be altered  */

                /* 512 KB for u-boot environment */

                reg = <0x0FF80000 0x00080000>;

                label = "NOR (RO) Raw u-boot Bootloader Image";

                /* read-only; */

            };

        };

    };

[lunminliang]

Hi,

The totally size is right:

nor@0,0 {

            #address-cells = <1>;

            #size-cells = <1>;

            compatible = "cfi-flash";

            reg = <0x0 0x0 0x10000000>;

            /* 128Mx16x1 - 2 banks of 128M at 16-bit wide = 256M total */

            bank-width = <2>;

            device-width = <1>;

But the dts settings of 256M Nor Flash are not as assumed, they were for 64M flash.

            partition@02800000 {

                /* 11 MB for JFFS2 persistent storage */

                reg = <0x02800000 0x00B00000>; ---------------------->increase the size here.

                label = "NOR (RW) UBI-FS Persistent Storage";

            };

Have a great day,
Lunmin

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[tbowen]

Yes, I am deliberately not using all of that partition at the moment to save UBI-FS formatting time.  However, I don't think that relates to the failure, because I'm trying to simply erase the first sector of partition 5, and that is for partition 4. ... Thanks!

[lunminliang]

Hi,

Could you please correct the settings and have a try if the failure is still there. Thanks.

Have a great day,

Lunmin

[tbowen]

Hi Lunmin,

That was not the problem.  We discovered the problem by hacking the flash driver in drivers/mtd/chips/cfi_cmdset_0002.c.  Apparently, this particular part actually contains 2 chips, and the erase-commands need to be sent to the appropriate chip.

Do you know the "correct" way to tell the kernel that this 1 NOR flash is really 2 chips and it needs to send erase commands to the specific chip address?  We are doing the following in drivers/mtd/chips/cfi_cmdset_0002.c:

static int __xipram do_erase_chip(struct map_info *map, struct flchip *chip)

{

...

    if (adr>0x07ffffff) {

    cfi_send_gen_cmd(0xAA, cfi->addr_unlock1, 0x8000000 + chip->start, map, cfi, cfi->device_type, NULL);

    cfi_send_gen_cmd(0x55, cfi->addr_unlock2, 0x8000000 + chip->start, map, cfi, cfi->device_type, NULL);

    cfi_send_gen_cmd(0x80, cfi->addr_unlock1, 0x8000000 + chip->start, map, cfi, cfi->device_type, NULL);

    cfi_send_gen_cmd(0xAA, cfi->addr_unlock1, 0x8000000 + chip->start, map, cfi, cfi->device_type, NULL);

    cfi_send_gen_cmd(0x55, cfi->addr_unlock2, 0x8000000 + chip->start, map, cfi, cfi->device_type, NULL);

    cfi_send_gen_cmd(0x10, cfi->addr_unlock1, 0x8000000 + chip->start, map, cfi, cfi->device_type, NULL);

    } else {

    cfi_send_gen_cmd(0xAA, cfi->addr_unlock1, chip->start, map, cfi, cfi->device_type, NULL);

    cfi_send_gen_cmd(0x55, cfi->addr_unlock2, chip->start, map, cfi, cfi->device_type, NULL);

    cfi_send_gen_cmd(0x80, cfi->addr_unlock1, chip->start, map, cfi, cfi->device_type, NULL);

    cfi_send_gen_cmd(0xAA, cfi->addr_unlock1, chip->start, map, cfi, cfi->device_type, NULL);

    cfi_send_gen_cmd(0x55, cfi->addr_unlock2, chip->start, map, cfi, cfi->device_type, NULL);

    cfi_send_gen_cmd(0x10, cfi->addr_unlock1, chip->start, map, cfi, cfi->device_type, NULL);

    }

Thanks!

Trevor

[tbowen · Accepted solution]

I think I found it!  See:

http://lxr.free-electrons.com/source/Documentation/devicetree/bindings/mtd/mtd-physmap.txt

Specifically:

http://lxr.free-electrons.com/source/Documentation/devicetree/bindings/mtd/mtd-physmap.txt#L9

and

http://lxr.free-electrons.com/source/Documentation/devicetree/bindings/mtd/mtd-physmap.txt#L62

Our device-tree now looks like the following, which works fine:

        localbus@ffe05000 {

                #address-cells = <2>;

                #size-cells = <1>;

                compatible = "fsl,p2020-elbc", "fsl,elbc", "simple-bus";

                reg = <0 0xffe05000 0 0x1000>;

                interrupts = <19 2>;

                interrupt-parent = <&mpic>;

                /* NOR Flash */

                ranges = <0x0 0x0 0x0 0xE0000000 0x10000000>;

                nor@0,0 {

                        #address-cells = <1>;

                        #size-cells = <1>;

                        compatible = "cfi-flash";

                        /* 2 chips of 128MB = 256M total */

                        reg = <0x0 0x00000000 0x08000000

                               0x0 0x08000000 0x08000000>;

                        bank-width = <2>;

                        device-width = <1>;

            partition@0 {

                /* 39 MB for SquashFS Root File System - Image #1 */

                reg = <0x00000000 0x02700000>;

                label = "NOR (RO) SquashFS Root File System";

                /* read-only; */

            };

            partition@02700000 {

                /* 01 MB for SquashFS Root File System - Image #2 */

                reg = <0x02700000 0x00100000>;

                label = "NOR (RO) 1MB Dead Space";

                /* read-only; */

            };

            partition@02800000 {

                /* 11 MB for JFFS2 persistent storage */

                reg = <0x02800000 0x00B00000>;

                label = "NOR (RW) UBI-FS Persistent Storage";

            };

            partition@0F300000 {

                /* 11.00 MB for Linux Kernel #1 */

                reg = <0x0F300000 0x00B00000>;

                label = "NOR (RO) Raw Linux Kernel Image";

                /* read-only; */

            };

            partition@0FE00000 {

                /* 1.25 MB for Linux Kernel #2 */

                reg = <0x0FE00000 0x00140000>;

                label = "NOR (RO) 1.25 MB Dead Space";

                /* read-only; */

            };

            partition@0FF40000 {

                /* This location must not be altered  */

                /* 128 KB for Device Tree Blob */

                reg = <0x0FF40000 0x00020000>;

                label = "NOR (RO) Raw Device Tree Data";

                /* read-only; */

            };

            partition@0FF60000 {

                /* This location must not be altered  */

                /* 128 KB for u-boot environment variables */

                reg = <0x0FF60000 0x00020000>;

                label = "NOR (RO) Raw u-boot Environment Variables";

                /* read-only; */

            };

            partition@0FF80000 {

                /* This location must not be altered  */

                /* 512 KB for u-boot environment */

                reg = <0x0FF80000 0x00080000>;

                label = "NOR (RO) Raw u-boot Bootloader Image";

                /* read-only; */

            };

        };

        };

[lunminliang]

Hi,

O, so there are two chips actually? Good to know the problem solved, thank you to share this on our community!

Have a great day,

Lunmin

[tbowen]

lunminliang - Well, it is deceiving.  There is only 1 package, but there are 2 chips inside the 1 package!  Looking at the above links, this is apparently not uncommon, but unless you are born knowing such things, it is not obvious that multiple register ranges are needed in the device tree to support a single package. ... Oh, well, at least we know now.  :smileyhappy:  Thanks for the help!

[lunminliang]

Hi,

Have you tried configuring these to be "N":

DeviceDrivers ->
Memory Technology Devices (MTD) ->
<>FTL (Flash Translation Layer) support
<> NFTL(NAND Flash Translation Layer) support
<>INFTL(Inverse NAND Flash Translation Layer) support

Have a great day,
Lunmin

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[tbowen]

lunminliang: Yes, those were already disabled. ... Thanks!