Hi Marceli,
To be honest, I don't support analog product, such as PCAL6524 product. I have no idea about the issue.
I would recommend to submit an online support, our analog product engineer can support this issue.
Thanks for understanding.
How to submit a new question for NXP Support
https://community.nxp.com/docs/DOC-329745
best regards,
Mike