Yes, there is a mux on the way to those i2c register banks, so first I selected the mux address using 7-bit addressing, then I wrote to the control register of the mux to select the first line (it is a 1:4 mux), and then I selected which register bank I want to write in (on each line there are two register banks)....then since these register banks are having indirect addressing, I selected the pointer at location 0, meaning I want to read from address zero of register bank, and last but not least, I started to read from it.
I think the sequence is correct that I used, since I am getting correct data back.
I think that the problem was with hardware/PCB. I had a new hardware coming in and not experiencing this issue anymore with the new hardware as of yet. The reason that I2C was going down after a while or not responding could be due to that. I keep this updated if I see a similar behavior again on I2C.
Thanks
Mehdi