Hello Aldo,
thank you so much for your help, this was indeed the issue!
For anyone wondering, nvm_daemon is a systemd service that is part of imx-secure-enclave, the userspace library for the ELE.
The service seems to not be automatically started by default.
From what I could tell from Chapter3 of the AN14105 Application Note ("Understanding SECO Secure Storage and Non-Volatile Memory Management"), the reason for this is, that the ELE itself does not have any non-volatile memory and instead relies on the NVM manager to store the wrapped keys to non-volatile memory.
In our case, this NVM manager is nvm_daemon.
That Application Note applies to SECO, but I'm going out on a limb an assuming that similar if not identical concepts apply to the ELE.