/dev/kmsg buffer overrun, some messages lost

[mahendrasondaga]

Hello there...

Hope all are doing well 

I'm dealing with the Yocto Hardknott version with the Linux kernel version 5.10

Along with the default device driver packages, i have added external driver packages for the Micrel KSZ8795, 4 port LAN switch support {Link}
I have also applied the patches for the Kconfig and Makefile to integrate the drivers with the mainline kernel 

For me, The driver working well and i got the all 4 LAN port working!

However, I'm facing the issue with the booting time!

As during the booting i got the following message continuous
-----------------------------------------------------------------------------------------------------------------

[  448.652491] systemd-journald[325]: /dev/kmsg buffer overrun, some messages lost.

[  449.112732] systemd-journald[325]: /dev/kmsg buffer overrun, some messages lost.

[  449.625573] systemd-journald[325]: /dev/kmsg buffer overrun, some messages lost.

[  449.641618] systemd-journald[325]: /dev/kmsg buffer overrun, some messages lost.

[  451.156241] systemd-journald[325]: /dev/kmsg buffer overrun, some messages lost.

[  451.182373] systemd-journald[325]: /dev/kmsg buffer overrun, some messages lost.

[  451.662724] systemd-journald[325]: /dev/kmsg buffer overrun, some messages lost.

[  451.732552] systemd-journald[325]: /dev/kmsg buffer overrun, some messages lost.

[  453.705702] systemd-journald[325]: /dev/kmsg buffer overrun, some messages lost.

--------------------------------------------------------------------------------------------------------------------------

From my side, i have updated the manuconfig as follows 

CONFIG_LOG_BUF_SHIFT=19

CONFIG_LOG_CPU_MAX_BUF_SHIFT=14

CONFIG_PRINTK_SAFE_LOG_BUF_SHIFT=14

CONFIG_CONSOLE_LOGLEVEL_DEFAULT=3

CONFIG_CONSOLE_LOGLEVEL_QUIET=2

CONFIG_MESSAGE_LOGLEVEL_DEFAULT=2


apart from that, in bootargs i have updated, log_buf_len=65536

still, it's not working for me!

Apart from that, because of the buffer overrun, 

Journal services taking too long time to start 

here is the logs from the kernel

"A start job is running for the Journal service"

And over all it's increasing the boot time!

Can anyone help me here with the best solution 

Thanks 
Mahendra 

[mahendrasondaga]

Hi.. @Sanket_Parekh 
Thanks for the reply...

Yes, I'll try with printk.devkmsg=off

To port the KSZ8795 drivers, I'm manually copying the device drivers source, and headers files into the /drivers/net//ethernet/micrel directory into the kernel source 

As the default kernel source doesn't contain those drivers (As the DSA drivers for this LAN switch seem unstable for the Linux kernel version 5.10)

here are the drivers which Microchip provides for the kernel version 5.10

https://github.com/Microchip-Ethernet/EVB-KSZ9477/tree/master/KSZ/linux-drivers/ksz8795/linux-5.10

When it comes to makefile and Kconfig, then I'm applying the patch for the same for those drivers 

If you can see the drivers provided by the microchip then, they have provided the patches for the cadence and Freescale FEC controller drivers

I'm applying the patches for both of these drivers 

I can't grep the logs for the KSZ8795 because I'm dealing with the minimal bootlogs with the LEVEL=3

Because without this, it causes infinite boot time!

As it continuously displays the "A start job is running for Journal Service"

Can you please check?

Welcome all suggestions and comments 

Thanks 

Mahendra 

[Sanket_Parekh]

Hi @mahendrasondaga,

I hope you are doing well.

Yes, Please try with printk.devkmsg=off

Does this occur if custom ethernet drivers are not ported?

Please try disabling drivers related to Ethernet to rule out issues with the KSZ9477 driver.

It is not recommended to make any changes to the default fec_main.c driver.

Thanks & Regards,
Sanket Parekh

[mahendrasondaga]

Hi.. @Sanket_Parekh 

Thanks for Quick feedback!

Can you pl. suggest me the steps to do printk.devkmsg=off ?

is it through the menuconfig ?

It's not occurring if the custom drivers not ported!

I have to use the custom driver only because the default DSA drivers available in /microchip directory dose not seems stable for this version of the kernel

KSZ9477 drivers are already disabled

Okay, will remove the patch associated with the fec_main.c

Thanks

Mahendra

[Sanket_Parekh]

Hi @mahendrasondaga 

I hope you are doing well.

One can change kernel command line options in the board header file in u-boot.
or

directly from u-boot commands.

for example:
uboot-imx/include/configs/mx6ul_14x14_evk.h
mmcargs=setenv bootargs printk.devkmsg=off  console=${console},${baudrate} " \
        BOOTARGS_CMA_SIZE \

        "root=${mmcroot}\0"

Thanks & Regards,

Sanket Parekh

[Sanket_Parekh]

Hi @mahendrasondaga 

I hope you are doing well.

Please provide me with full boot logs for further debugging.

Please also share the steps used to port KSZ8795,

Meanwhile. One could try using printk.devkmsg=off kernel parameter.

Please refer to printk_devkmsg for more information.

Thanks & Regards,

Sanket Parekh