IMX8 multiple camera virtual channels on CSI interface not producing mulipe /dev/video entries

[TerryBarnaby1]

We have a TP2855 4 channel video decoder which we are using on a custom IMX8MP board. We also have an TP2855 EVK board connected to an imx8mp-evk board for testing.

We are using the NXP IMX Linux with the linux-imx 5.10 lf-5.10.y kernel.

We have a tp2855 I2C driver from Techpoint which appears to have been originally created by Freescale Semiconductor, Inc. We have modified this to work in the system and we see a good fullHD video stream from a connected camera.

However when we install the tp2855 driver, the system only creates a /dev/video3 node when we believe it should have created 4 x /dev/video* nodes. The tp2855 driver looks like it uses the media interface to create the pads/nodes:

v4l2_i2c_subdev_init(sd, client, &tp2855_subdev_ops);
sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;

sd->entity.function = MEDIA_ENT_F_CAM_SENSOR;
tp2855_data->pads[MIPI_CSI2_SENS_VC0_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
tp2855_data->pads[MIPI_CSI2_SENS_VC1_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
tp2855_data->pads[MIPI_CSI2_SENS_VC2_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
tp2855_data->pads[MIPI_CSI2_SENS_VC3_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
retval = media_entity_pads_init(&sd->entity, MIPI_CSI2_SENS_VCX_PADS_NUM,
tp2855_data->pads);

I note the kernel messages state: "imx8_media_dev: module is from the staging directory, the quality is unknown, you have been warned."

Not sure what is happening here.

1. Should the imx8_media_dev driver have created the 4 x /dev/video* nodes for this 4 channel video decoder but is not handling this correctly ?

2. There are issues in the tp2855 driver that need addressing to get the 4 /dev/video* entries ?

[ZXC]

I also encountered this problem, did you debug successfully

[TerryBarnaby1]

Not directly. The IMX8mp has a very basic CSI2 camera interface hardware block unlike some other SOC's in the IMX range. It does not have the ability to see and do anything with the CSI2 virtual channel tags in the stream.

For our use we were using a TP2855 4 analogue channel front end device. We changed this to a TP2856 chip that supports two CSI2 interfaces and connected that to the IMX8mp's two CSI2 camera interfaces so we could have two simultaneous cameras in use.

With particular CSI input hardware and particular camera resolutions (all the same) it might be possible to de-interleave the the 4 CSI virtual channels in software somehow using what info there is in the IMX8's CSI2 registers. But even if possible, not sure of the software overhead of doing this.

Oh why couldn't NXP use the same hardware IP block types across the IMX8 range !

[TerryBarnaby1]

I don't suppose there is the means to get the second CSI2.2 interface to use the CSI2.1 interfaces clock and data[23] lanes at all ?

[TerryBarnaby1]

Thanks for the reply.

Wow that is a big drop off, why on earth was that done !

Please can this be made clear in the documentation, we never noticed this lack of support for MIPI CSI2 virtual channels on the IMX8MP.

Is there a chance future revisions of the IMX8MP will have this ability ?

We currently have a 4 virtual channel CSI stream going into the IMX8MP and we can view the video from virtual channel 0. Is the MIPI IP separating virtual channel 0 out and ignoring virtual channels 1,2,3 or is something else going on ?

[Rita_Wang]

Is there a chance future revisions of the IMX8MP will have this ability ?

No, sorry about that. You can choose the i.MX8QM or i.MX8QXP.

[TerryBarnaby1]

Unfortunately the i.MX8QM or i.MX8QXP do not have other features we need and we now have our first prototype IMX8MP boards up and running.

We currently have a 4 virtual channel CSI stream going into the IMX8MP and we can view the video from virtual channel 0 fine.

1. From random internet reading, as there doesn't appear to be any information from NXP as to what happens, I assume that the complete CSI streams data (all virtual channels) is fed to memory, but the drivers are taking just the first virtual channel somehow, is this the case ?

2. I have also seen that it might be possible for software to demux the virtual channels in the video stream. Is this actually possible and if so would there be a high CPU usage overhead in doing this ?

3. Is there any information on how this all works ?

4. An alternative, for us, is to feed the TP2855's CSI stream to the two IMX8MP's CSI inputs. The TP2855 allows for two CSI streams sharing the one clock. Now we would have to somehow feed the CSI clock into the two IMX8MP CSI clock inputs. Is it possible to drive the second CSI's clock from the other CSI's clock input or, as these pins are next to each other on the IMX8MP, is it possible to  turn off the termination on one of the clock inputs ?

5. Anyone have any ideas on a suitable differential clock to two differential clock chips that would keep the phase (I assume the CSI data lines have to be in phase with the clocks) ?

Terry

[Rita_Wang]

If you can not choose the i.MX8QM or i.MX8QXP to use, you can use the PCIE, but how to use in details I  do not know.

[Rita_Wang]

tp2855(4 chanle input, mipi csi-2 output)--->I.MX8MP MIPI CSI-2 interface.

The i.MX8MP MIPI CSI-2 does not have the ability to separate 4-channel camera video using virtual channels, which is determined by MIPI IP.

But i.MX8QM and i.MX8QXP can do it.

[TJOD]

Hi, 

Would these be valid MPNs (MIMX8QP5AVUFFAB or MIMX8QP6AVUFFAB) for using virtual channels?  If not, what are the part numbers? 

Regards,

Tom