Hi Yiping,
Seems that is a bug of SDK? We didn't enable it in the kernel, please refer below picture shows

Also, I checked the CAAM driver folder Kconfig file, seems it is fine as we absolutely not select iMX series CPU, but no idea why it was enabling
-------------
config CRYPTO_DEV_FSL_CAAM_IMX
def_bool SOC_IMX6 || SOC_IMX7D
depends on CRYPTO_DEV_FSL_CAAM
config CRYPTO_DEV_FSL_CAAM_LE
def_bool (CRYPTO_DEV_FSL_CAAM_IMX || SOC_LS1021A) && !ARCH_LAYERSCAPE
depends on CRYPTO_DEV_FSL_CAAM
-----------
I will disable it and compare config file which you attached one and do a test now, will keep you update
Thanks