Hello,
This is an expected behavior since reset state will change IOMUX register to configure the pin as CCM_CLKOX function and is changed until device tree is loaded.
As you mention, U-boot/kernel does not causes this.
If you change the IOMUX option before e.g. in SLP, you could reduce the time with the clock output at reset.
Best regards.