A/B Boot Failure: Partition B Fails to Boot But Partition A Works

[Samt07]

Hi,

I have a custom Android 13 build for the iMX8MM. The device can boot from Partition A successfully, but when I try to switch to Partition B (either after flashing the super.img or using OTA), the device attempts to boot from Partition B, but it eventually falls back to Partition A.

I have tried the following methods to switch to Partition B:

1. fastboot --set-active=b
2. Used OTA to trigger the A/B switch. 

Despite these, Partition B never boots, and the system switches back to Partition A.

Additionally, I cannot see the _b partitions listed in /dev/block/mapper/ after booting into slot A. Here’s the output of ls /dev/block/mapper/

by-uuid com.android.ipsec com.android.tethering system_ext-verity
com.android.adbd com.android.media com.android.uwb system_ext_a
com.android.adservices com.android.media.swcodec com.android.wifi userdata
com.android.art com.android.mediaprovider product-verity vendor-verity
com.android.cellbroadcast com.android.neuralnetworks product_a vendor_a
com.android.conscrypt com.android.permission system-verity vendor_dlkm-verity
com.android.extservices com.android.resolv system_a vendor_dlkm_a

During boot, I see these attached log entries it booted from partition A after 7 failure in partition B: 

Why is Partition B unbootable even though I have flashed the super.img?
How can I ensure that the B slot is properly populated and bootable after flashing or OTA?

Any guidance or suggestions would be greatly appreciated!

[Harvey021]

Hi,

Can you please send the log of OTA not starting from B slot?

Regards

Harvey

[Samt07]

Hi,

Thanks for responding. I’ve collected the following logs related to the OTA process and the slot B boot attempt:

1. Slot metadata — Collected using `lpdump` and `fastboot`.
2. update_engine logs — Captured during the OTA update process.
3. Boot logs — Captured during the boot attempt on slot B.
   
   

Looking forward to your insights!

[Harvey021]

## Starting auxiliary core stack = 0x20020000, pc = 0x1FFE035D...
Verifying slot _b ...
Writing A/B metadata to disk.
verify OK, boot 'boot_b'
Kernel load addr 0x40480000 size 48419 KiB
kernel @ 40480000 (50200576)
ramdisk @ 44680000 (22499132)
fdt @ 435e0400 (50105)
Moving Image from 0x40480000 to 0x40600000, end=435e0000
## Flattened Device Tree blob at 435e0400
Booting using the fdt blob at 0x435e0400
Working FDT set to 435e0400
Using Device Tree in place at 00000000435e0400, end 00000000435ef7b8
Working FDT set to 435e0400

Starting kernel ...

From here, it seems that there is something wrong with the kernel and it cannot be started. Maybe there is something wrong with the image downloaded by OTA upgrade. If you do not use the OTA upgrade method, directly burn the image to be upgraded to slot A and try it.

And have you tried to refer to the section <7 Over-The-Air (OTA) Update> of ANDROID_USERS_GUIDE ?

Regards

Harvey

[Samt07]

Hi @Harvey021,

Thank you for your suggestions. I have already tried directly flashing the images to both slot A and slot B. Slot A boots successfully, but slot B still fails to boot even after flashing the images directly.

Additionally, I followed the OTA process as outlined in the ANDROID_USERS_GUIDE, ensuring that the steps were correctly implemented. Despite this, slot B remains unbootable, and the device reverts to slot A.

I am attaching the failure log from when I attempted to set the active slot to B using fastboot --set-active=b. This log may provide more insights into what might be going wrong during the boot process for slot B.

Could there be an issue with the kernel or device tree specific to slot B?

[Harvey021]

Hi @Samt07 

With reference to internal team, here below are the reply.

Before virtual A/B is enabled on Android, all partitions will have both "A" and "B" physical partitions and both "A"/"B" slot can boot separately by setting the target boot slot (for example: fastboot --set-active=b, fastboot --set-active=a). 

After virtual A/B is enabled on Android, some physical partitions(system, vendor, etc.) would be merged to the "super" partition. The "super.img" in the build directory would only contains the "A" slot partition images (system_a, vendor_a, etc.)  and that's why the customer can't boot the "B" slot by setting "fastboot --set-active=b" anymore. Hence, this behavior is expected.

According to the Android A/B philosophy: https://source.android.com/docs/core/ota/virtual_ab. If the device boots from slot "A", its next OTA will updates to slot "B". And if the slot "B" doesn't boot successfully, it will fall back to slot "A" to make sure the device can still work. From the customer's boot logs, I can see the OTA process succeed (means the images are downloaded to slot "B") but the bootloader failed to boot slot "B" kernel, which means their kernel image could be wrong.

I would sugguest they flash the same image which they have packed to the OTA package to the slot "A" and see if they can boot succeed. From the logs they have provided, the kernel image in the slot "A" is not same with the one in their OTA package (the one downloaded to "B" slot) because the kernel size is different:

Verifying slot _b ...
Writing A/B metadata to disk.
 verify OK, boot 'boot_b'
Kernel load addr 0x40480000 size 48419 KiB

Verifying slot _a ...
 verify OK, boot 'boot_a'
Kernel load addr 0x40480000 size 21376 KiB

Regards

Harvey

[Samt07]

Hi @Harvey021,

Thank you for the detailed explanation. I understand the issue with the OTA packaging, but I have a question regarding the super image and dynamic partitions.

You mentioned that the super.img contains only the slot A partitions (e.g., system_a, vendor_a). Since the super image is designed to work with dynamic partitions, how is it supposed to correctly set up and manage both slot A and slot B?

Specifically:

1. If super.img only contains slot A partitions, how does slot B get populated or prepared for OTA updates?
2. Is there a specific step I might be missing during the flashing process to ensure that slot B is properly set up and ready to boot after an OTA update?

how to ensure the super image correctly supports both slots under the virtual A/B setup.

Thank you again for your support.

[Harvey021]

hi

Based on current design, if device boots from A slot, its next OTA will download images to the B slot (not physical B slot but just snapshots) then it triggers reboot from B slot. If B slot boots ok, it merges the data and it will mark the B slot as current slot, slot A will become unbootable. Even you try to swith to A slot manually, you still can't boot A slot because A slot images are already gone. In next OTA, the images would be downloaded to slot A and repeat the same process.
 
 You can check Google articles about virtual A/B and dynamic partitions for more technical details:
  https://source.android.com/docs/core/ota/virtual_ab 
  https://source.android.com/docs/core/ota/dynamic_partitions 

Are you trying to make both A/B slot bootable even you are using virtual A/B and doing OTA? The answer would be "NO" because the previous slot images would be gone after successful OTA. And it doesn't make much sense to keep the "old" system bootable.
 
However, as what I have mentioned, the default super.img only contains slot A partition images, it also contains B slot partitions but with empty image. You can reboot to fastbootd and flash slot B images to make the slot B bootable. For example, follow below steps:
1. flash images with uuu tool: sudo ./uuu_imx_android_flash.sh -f imx8mm -e -u trusty-dual
2. boot from A slot and "reboot fastboot" to enter into fastbootd mode.
3. flash slot B partitions images to super:
    fastboot flash system_b system.img
    fastboot flash system_ext_b system_ext.img
    fastboot flash vendor_b vendor.img
    fastboot flash product_b product.img
4. set B slot as active "fastboot -ab" and reboot.
After aboving setting, the device should be able to boot from both A/B slots. But this is a tricky mothod and it won't work anymore after an OTA update(The previous partitions would be gone).
 
May I know the reason why you want both A/B slots bootable? We should always running the latest system instead of switching back to the "old" system arbitrarily because there will be risks of rollback attack.

Regards

Harvey

[Samt07]

Hi @Harvey021 ,

To clarify, my goal is not necessarily to keep both A/B slots permanently bootable in the traditional sense. Instead, I was initially concerned because slot B was failing to boot after an OTA update, and I was exploring ways to troubleshoot.

Your steps on manually flashing the slot B partitions via fastbootd are helpful for testing purposes, and I will give them a try.

Best regards,

[Maruti]

Hi @Samt07 

Have you found a solution to this issue?

I'm facing a similar problem — the OTA update completes successfully, but after rebooting, the system tries to boot from the boot_b slot and enters a continuous reboot loop during kernel boot.

Would really appreciate it if you could share any insights or fixes you've found.

Best regards,
Maruti Naik

[Harvey021]

hi @Samt07 

Let me know if problem,

Regards

Harvey

[Samt07]

Here is the relevant output of fastboot getvar all:

waste> fastboot getvar all
(bootloader) version:0.4
(bootloader) version-bootloader:U-Boot 2023.04-71149-gdea3de20ecc
(bootloader) version-baseband:N/A
(bootloader) product:i.MX 8M Mini
(bootloader) secure:yes
(bootloader) max-download-size:0x19000000
(bootloader) erase-block-size:0x80000
(bootloader) logical-block-size:0x200
(bootloader) unlocked:yes
(bootloader) off-mode-charge:1
(bootloader) battery-voltage:0mV
(bootloader) variant:dart
(bootloader) battery-soc-ok:yes
(bootloader) is-userspace:no
(bootloader) tee_enabled:no
(bootloader) soc_rev:N/A
(bootloader) partition-type:gpt:raw
(bootloader) partition-type::
(bootloader) partition-type:mcu_os:raw
(bootloader) partition-type:all:device
(bootloader) partition-type:bootloader0:raw
(bootloader) partition-type:bootloader_a:raw
(bootloader) partition-type:bootloader_b:raw
(bootloader) partition-type:dtbo_a:raw
(bootloader) partition-type:dtbo_b:raw
(bootloader) partition-type:boot_a:raw
(bootloader) partition-type:boot_b:raw
(bootloader) partition-type:init_boot_a:raw
(bootloader) partition-type:init_boot_b:raw
(bootloader) partition-type:vendor_boot_a:raw
(bootloader) partition-type:vendor_boot_b:raw
(bootloader) partition-type:misc:raw
(bootloader) partition-type:metadata:f2fs
(bootloader) partition-type:presistdata:raw
(bootloader) partition-type:super:raw
(bootloader) partition-type:userdata:f2fs
(bootloader) partition-type:fbmisc:raw
(bootloader) partition-type:vbmeta_a:raw
(bootloader) partition-type:vbmeta_b:raw
(bootloader) partition-size:gpt:0x100000
(bootloader) partition-size::0x0
(bootloader) partition-size:mcu_os:0x40000
(bootloader) partition-size:all:0x3b4800000
(bootloader) partition-size:bootloader0:0x1f80000
(bootloader) partition-size:bootloader_a:0x1000000
(bootloader) partition-size:bootloader_b:0x1000000
(bootloader) partition-size:dtbo_a:0x400000
(bootloader) partition-size:dtbo_b:0x400000
(bootloader) partition-size:boot_a:0x4000000
(bootloader) partition-size:boot_b:0x4000000
(bootloader) partition-size:init_boot_a:0x800000
(bootloader) partition-size:init_boot_b:0x800000
(bootloader) partition-size:vendor_boot_a:0x4000000
(bootloader) partition-size:vendor_boot_b:0x4000000
(bootloader) partition-size:misc:0x400000
(bootloader) partition-size:metadata:0x4000000
(bootloader) partition-size:presistdata:0x100000
(bootloader) partition-size:super:0x100000000
(bootloader) partition-size:userdata:0x295900000
(bootloader) partition-size:fbmisc:0x100000
(bootloader) partition-size:vbmeta_a:0x100000
(bootloader) partition-size:vbmeta_b:0x100000
(bootloader) has-slot:gpt:no
(bootloader) has-slot::no
(bootloader) has-slot:mcu_os:no
(bootloader) has-slot:all:no
(bootloader) has-slot:bootloader0:no
(bootloader) has-slot:bootloader:yes
(bootloader) has-slot:dtbo:yes
(bootloader) has-slot:boot:yes
(bootloader) has-slot:init_boot:yes
(bootloader) has-slot:init:no
(bootloader) has-slot:vendor_boot:yes
(bootloader) has-slot:vendor:no
(bootloader) has-slot:misc:no
(bootloader) has-slot:metadata:no
(bootloader) has-slot:presistdata:no
(bootloader) has-slot:super:no
(bootloader) has-slot:userdata:no
(bootloader) has-slot:fbmisc:no
(bootloader) has-slot:vbmeta:yes
(bootloader) current-slot:b
(bootloader) slot-count:2
(bootloader) slot-successful:a:yes
(bootloader) slot-successful:b:no
(bootloader) slot-unbootable:a:no
(bootloader) slot-unbootable:b:no
(bootloader) slot-retry-count:a:1
(bootloader) slot-retry-count:b:5
(bootloader) snapshot-update-status:snapshotted
all: Done!
Finished. Total time: 0.088s