Hi Hafiz Haja,
There is no problem with your dessign, since the PCA9517 is multi-master capable, the DC off set at B side wont be a problem, just make sure to be at or lower than 0.4V for arbitration or clock stretching to be transmitted to A side.
Regards,
Aldo