LLCE_CAN interface did not show up with bsp38

[XD]

Hi,

I modified the conf/local.conf file and added llce_can in DISTRO_FEATURES:append. I have all four .bin files in the pre-defined firmware folder, and the LLCE firmware version is 1.0.8. The SD card image used is fsl-image-auto.

When the board boots up, I can see that the LLCE kernel modules are loaded. However, running ifconfig -a doesn't show any llce_can interfaces. I've dumped the device tree from the running system, and all llce_can interfaces have their status set to OK.

Can anyone help me identify what might be wrong? I've included more details below.

root@s32g399ardb3:~# lsmod
Module                  Size  Used by
pfeng                 565248  0
llce_can               24576  0
llce_mailbox           32768  1
llce_core              16384  0

grep the dmesg, only a few lines are llce related

root@s32g399ardb3:~# dmesg | grep -i llc
[    6.777298] Modules linked in: llce_mailbox(+) llce_core
[    6.807139] lr : llce_mb_last_tx_done+0xb8/0x170 [llce_mailbox]
[    6.894104]  execute_hif_cmd+0x70/0xe0 [llce_mailbox]
[    6.899138]  get_fw_version+0x84/0xb0 [llce_mailbox]
[    6.904087]  llce_mb_probe+0x4c8/0x854 [llce_mailbox]
[    6.969889]  llce_mb_driver_init+0x24/0x1000 [llce_mailbox]
root@s32g399ardb3:~# 

these are related to some kernel exception

    Starting Load Kernel Modules...
         Starti[    6.756042] systemd[1]: Starting Generate network units from Kernel command line...
ng Generate network �…ts from Kernel command li[    6.769759] Internal error: synchronous external abort: 96000210 [#1] PREEMPT SMP
n[    6.777298] Modules linked in: llce_mailbox(+) llce_core
e[    6.782664] CPU: 7 PID: 156 Comm: systemd-modules Not tainted 5.15.119-rt65+ge18f05316cd9 #1
   6.791169] Hardware name: NXP S32G399A-RDB3 (DT)
[[    6.795944] pstate: 800000c5 (Nzcv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
0[    6.802973] pc : __memcpy_fromio+0x48/0xa0
m[    6.807139] lr : llce_mb_last_tx_done+0xb8/0x170 [llce_mailbox]
.[    6.813130] sp : ffffffc00a0bb600
.[    6.816514] x29: ffffffc00a0bb600 x28: ffffff885f8fdc50 x27: 0000000000000000
.[    6.823718] x26: ffffffc00a0bb6b0 x25: ffffffc00ba80000 x24: ffffff88009621c8
[    6.830923] x23: 0000000000000000 x22: 0000000000000000 x21: ffffff88017b4000

[    6.838127] x20: ffffff8801638080 x19: 0000000000000001 x18: 0000000000000001
[    6.845239] x17: 0000000000000000 x16: ffffff8800962128 x15: 0000000000000000
[    6.852354] x14: ffffffffffffffff x13: 0000000000000020 x12: 0101010101010101
[    6.859471] x11: 0000000000000040 x10: ffffff88009ae128 x9 : 0000000000000000
[    6.866588] x8 : ffffffc00a0bb848 x7 : 0000000000000000 x6 : 0000000000000198
[    6.873706] x5 : 0000000000000000 x4 : ffffffc00bab9f70 x3 : ffffffc00a0bb848
[    6.880823] x2 : 0000000000000198 x1 : ffffffc00bab9de0 x0 : ffffffc00a0bb840
[    6.887942] Call trace:

 Thanks,

XD

[Daniel-Aguirre · Accepted solution]

Hi,

We understand that you are using LLCE-FW v1.0.8 with BSP38.0. If so, we see under the BSP38.0 release notes that the compatible LLCE-FW version is v1.0.7 [Page 2, Linux BSP 38.0 Release Notes for S32G3, 10/2023]:

We can recommend using LLCE-FW v1.0.7 to confirm the problem is not related to compatibility.

Please, let us know.

[XD]

Hi Daniel,

Thank you for your quick response.

I replaced all four .bin files in /lib/firmware on the SD card with the 1.0.7 LLCE firmware. Unfortunately, the llce_can interfaces are still missing. I noticed a failure when loading the llce_core kernel module during bootup. I've posted a screenshot below and included the md5sum of the .bin files for cross-checking.

Apr 28 17:42:26 s32g399ardb3 systemd-modules-load[156]: Inserted module 'llce_core'
Apr 28 17:42:26 s32g399ardb3 systemd[1]: systemd-modules-load.service: Main process exited, code=killed, status=11/SEGV
Apr 28 17:42:26 s32g399ardb3 systemd[1]: systemd-modules-load.service: Failed with result 'signal'.
Apr 28 17:42:26 s32g399ardb3 systemd[1]: Failed to start Load Kernel Modules.
Notice: journal has been rotated since unit was started, output may be incomplete.
root@s32g399ardb3:~#
root@s32g399ardb3:~# lsmod
Module                  Size  Used by
pfeng                 565248  0
llce_can               24576  0
llce_mailbox           32768  1
llce_core              16384  0

root@s32g399ardb3:~# md5sum /lib/firmware/*.bin
7031a5834b95e73c77f4da72bab44dcb  /lib/firmware/dte.bin
96a3ac83a650418f331b560d5c8c7d1a  /lib/firmware/frpe.bin
fb2e43b53f9470c9fb4348d15b81409b  /lib/firmware/ppe_rx.bin
ee4bfaaf2a172e7a842fb272aaa85004  /lib/firmware/ppe_tx.bin

Thanks,

XD

[Daniel-Aguirre]

Hi,

Thanks for your feedback. Did you re-build your image with LLCE-FW v1.0.7? Or you only replaced the binaries?

If you build your image for v1.0.8, we understand that it might have an undefined behavior if you just replaced the LLCE-FW binaries. 

Also, we could recommend doing the Manual compilation steps for LLCE (provided under the BSP38.0 User Manual) to understand if the problem might be related to yocto itself.

Please, let us know.

[XD]

Hi Daniel,

I did not rebuild the LLCE 1.0.7 firmware; it was prebuilt and downloaded, then copied to the /lib/firmware folder on the SD card.

Based on your suggestion, I tried using manual steps. Interestingly, after I removed llce_can from the DISTRO_FEATURES:append in the local.conf file (and previously set the LLCE CAN as a module in the kernel configuration), I rebuilt the image. Upon booting the system, all LLCE CAN and logger interfaces were present. I checked the /lib/firmware folder and found only the PFE firmware; no LLCE firmware binary files were present. I even tried using -c cleanall to rebuild the image, but the result was the same—every interface was up and running.

I am confused about how the LLCE CAN interface is operational without copying the kernel module and firmware files. Here is the change I made in the local.conf file. Additionally, the LLCE firmware binary files are still physically present in the NXP_FIRMWARE_LOCAL_DIR.

DISTRO_FEATURES:append = " ipc-shm pfe "
NXP_FIRMWARE_LOCAL_DIR = "/home/xhu/workspace/fsl-auto-yocto-bsp/nxp_firmware_files"

Thanks,

XD

[Daniel-Aguirre]

Hi,

Thanks for your feedback. We understand that your image was built with LLCE-FW v1.0.8 and not rebuilt with LLCE-FW v1.0.7, is this correct?

As for the behavior, seems to be that the binaries were missing inside the Linux image, hence LLCE was not being initialized. Once the binaries were available, then it could start.

Please, let us know.

[XD]

Hi Daniel,

Sorry for the confusion.

Current Setup:

I have the 1.0.7 LLCE firmware(.bin files) on my local build machine, which the NXP_FIRMWARE_LOCAL_DIR in the local.conf file points to. However, 'llce-can' is not included in DISTRO_FEATURES:append.

I also enabled NXP LLCE CAN support in the kernel configuration using menuconfig, setting it to 'M' (module), as I couldn't make it a built-in module. After rebuilding fsl-image-auto and booting the system, I found that the llce-can kernel module loads automatically, and all llce-can interfaces are up and running.

I did not manually copy the llce-can kernel module .ko files or the LLCE CAN .bin files to the board.

Concerns:
I did not specify 'llce-can' in the local.conf file.
I did not manually copy any .ko files to the board.
I did not specify how the OS retrieves the LLCE firmware .bin files.

Questions:
How does the kernel load the llce-can kernel modules automatically?
How does the OS obtain the LLCE firmware .bin files without explicit specification?

I need to understand this behavior to ensure the interfaces don't disappear unexpectedly in the future.

Thanks,

XD

[Daniel-Aguirre]

Hi,

Thanks for your feedback. Below will be some comments on regards of your questions:

How does the kernel load the llce-can kernel modules automatically?

[DA]: For the following statement:

"I also enabled NXP LLCE CAN support in the kernel configuration...

...

After rebuilding fsl-image-auto..."

Did you clean your build? Could be that remnants of the previous build were still available, hence it still took the LLCE-FW available.

How does the OS obtain the LLCE firmware .bin files without explicit specification?

[DA]: If the LLCE module was enabled, and the binaries were taken by the build as mentioned under the previous question, then the OS used the binaries available to activate LLCE.

Please, let us know.

[XD]

Hi Daniel,

I used bitbake -c cleanall fsl-image-auto before rebuilding the image. Is this sufficient to clean up everything from the previous build? Do you have any other suggestions?

Thanks,

XD

[Daniel-Aguirre]

Hi,

Thanks for your feedback. Aside from the clean, we can recommend also removing the LLCE-FW binaries from the folder, to understand if this prevents the automatic loading of the components.

Please, let us know.

[XD]

Hi Daniel,

Thank you for your support. I followed your advice and removed all LLCE .bin files. I also used both -c cleansstate and -c cleanall to clean up the build environment, and then ran bitbake -f fsl-image-auto to regenerate the entire image. However, the LLCE interfaces are still active and running.

I’m considering deleting the entire tmp folder and performing a complete rebuild. If you have any other suggestions, please let me know.

Thanks,

XD

[Daniel-Aguirre]

Hi,

Thanks for your feedback. Aside from what we have mentioned before, we don't have additional suggestions. We do apologize.

Please, let us know.

[XD]

Hi Daniel,

Thank you for your support. I'm okay with the current state now and will create a new topic if I encounter any new issues.

Thanks,

XD