MIPI CSI2 frame synchronization with i.MX6Q

[jwatts]

We are currently receiving progressive frames from a MachX02 via MIPI CSI2 on the i.MX6Q, but have so far been unable to solve one final issue. Similar to what is described in section 5 of the IPU TVIN app-note, our stream sometimes starts out-of-sync:

 

It is important to note that the stream "sticks" with this offset until we completely stop and restart the entire capture stream. It seems that the only time we get correctly synchronized frames is when the MIPI receiver begins receiving before our FPGA begin sending data. If our V4L2 sub-dev driver takes too long to return from ioctl_s_parm() then we end up with some amount of offset as above. Similarly, if something interrupts the stream, such as experimentally restarting the MIPI data stream from the FPGA without restarting the MIPI receiver, we end up with out-of-sync frames.

Our FPGA logic is producing frame-start/frame-end packets, but not line-start/line-end. When frames are not synchronized, we occasionally receive NFB4EOF errors but not on every frame. Most recently we tried to implement the patch provided in the TVIN app-note, even extending it to stop and restart our FPGA's output. Previously we also implemented frame-numbers in the frame-start packets. This appeared to improve the issue, 1 out of 10 starts fail down from 9 out of 10, but did not completely resolve the issue.

Does any one have any comments on why the IPU's FSU is not locking on to the next frame-start/vsync-edge?

[maximkuk]

Hello! Are you solved this issue?

[jwatts]

We ran out of time to solve the issue: We managed to find a kind of hokey-pokey dance that appears to work 99% of the time by sending 2 (or perhaps 3?) zero-length frames to wake up and synchronize the MIPI receiver. This doesn't help if a line gets corrupted/lost later in the stream, and the end customer has still been experiencing failures (~1% of the time) to start capture at a very low frequency which is frustrating to diagnose.

The only way to absolutely solve this problem, according to section 5.1.3 of the linked app-note is to insert or find a unique line of data in the active video stream, and restart capture if it is not where it belongs. Unfortunately we ran out of development time before we were able to implement this. In theory it should work in our case, as we control the FPGA design that is transmitting to the IMX6, but in other cases such as directly connected image sensors there may not be a way to uniquely identify frame alignment.

[maximkuk]

Hello! Thanks for yor reply.

[jimmychan]

please try the patches here

https://community.nxp.com/docs/DOC-102233 

[jwatts]

Those patches effectively ensure that all inputs are routed to CSI_MEM0, which will not work for my usecase: I currently have parallel capture on IPU0 CSI0 and MIPI capture on IPU0 CSI1 (virtual channel = 1). My current _ipu_smfc_init() and _ipu_csi_set_mipi_di() look like:

void _ipu_smfc_init(struct ipu_soc *ipu, ipu_channel_t channel, uint32_t mipi_id, uint32_t csi)
{
     uint32_t temp;
     uint32_t id = channel; // Temporary hack: Presuming that CSI_MEM0 thru CSI_MEM3 are 0 thru 3 and that MIPI VC will be mapped to matching CSI_MEMn

     temp = ipu_smfc_read(ipu, SMFC_MAP);

     switch (channel) {
     case CSI_MEM0:
          temp &= ~SMFC_MAP_CH0_MASK;
          temp |= ((csi << 2) | id) << SMFC_MAP_CH0_SHIFT;
          break;
     case CSI_MEM1:
          temp &= ~SMFC_MAP_CH1_MASK;
          temp |= ((csi << 2) | id) << SMFC_MAP_CH1_SHIFT;
          break;
     case CSI_MEM2:
          temp &= ~SMFC_MAP_CH2_MASK;
          temp |= ((csi << 2) | id) << SMFC_MAP_CH2_SHIFT;
          break;
     case CSI_MEM3:
          temp &= ~SMFC_MAP_CH3_MASK;
          temp |= ((csi << 2) | id) << SMFC_MAP_CH3_SHIFT;
          break;
     default:
          return;
     }

     ipu_smfc_write(ipu, temp, SMFC_MAP);
}

int _ipu_csi_set_mipi_di(struct ipu_soc *ipu, uint32_t num, uint32_t di_val, uint32_t csi)
{
     uint32_t temp;
     int retval = 0;

     if (di_val > 0xFFL) {
          retval = -EINVAL;
          goto err;
     }

     temp = ipu_csi_read(ipu, csi, CSI_MIPI_DI);

     switch (num) {
     case IPU_CSI_MIPI_DI0:
          temp &= ~CSI_MIPI_DI0_MASK;
          temp |= (di_val << CSI_MIPI_DI0_SHIFT);
          ipu_csi_write(ipu, csi, temp, CSI_MIPI_DI);
          break;
     case IPU_CSI_MIPI_DI1:
          temp &= ~CSI_MIPI_DI1_MASK;
          temp |= (di_val << CSI_MIPI_DI1_SHIFT);
          ipu_csi_write(ipu, csi, temp, CSI_MIPI_DI);
          break;
     case IPU_CSI_MIPI_DI2:
          temp &= ~CSI_MIPI_DI2_MASK;
          temp |= (di_val << CSI_MIPI_DI2_SHIFT);
          ipu_csi_write(ipu, csi, temp, CSI_MIPI_DI);
          break;
     case IPU_CSI_MIPI_DI3:
          temp &= ~CSI_MIPI_DI3_MASK;
          temp |= (di_val << CSI_MIPI_DI3_SHIFT);
          ipu_csi_write(ipu, csi, temp, CSI_MIPI_DI);
          break;
     default:
          retval = -EINVAL;
     }

err:
     return retval;
}

Experimentally trying the patches produced the same frame-sync issue as described above.