2411324_en-US

はい

i.MX8M Plus – Is there any way to access MIPI CSI-2 Embedded Data (DT 0x12) together with RAW image

Hello NXP Community,

we are currently integrating a MIPI CSI-2 RAW image sensor with an i.MX8M Plus and would like to clarify whether there is any supported or technically possible way to make the sensor's CSI-2 Embedded Data (Data Type 0x12) available to software on the target.

Our camera stream has the typical CSI-2 structure:

Frame Start
Embedded Data        DT = 0x12
RAW image data       DT = 0x2A / 0x2B / 0x2C
Frame End

The image data is captured through the MIPI CSI-2 receiver → Gasket → ISI → memory/V4L2 path and this part is working correctly.

The problem is that we additionally need the Embedded Data belonging to each frame. The metadata contains frame-related sensor information, and therefore it is important that the metadata can be associated with the corresponding captured image.

According to the i.MX 8M Plus Applications Processor Reference Manual, the MIPI CSI-2 receiver provides configurable data-type handling through the CSI-2 receiver registers, in particular the MIPI_CSIx_ISP_CONFIG0 / DATAFORMAT configuration. The relevant MIPI CSI-2 Rx / CSIS register descriptions are in Chapter 13.5, MIPI CSI-2, and in older revisions of the Reference Manual the supported image data types are described around Section 13.5.6.13.

We also found NXP Application Note AN13857 – i.MX 8M Series MIPI Capture System, especially Section 7, "Embedded data support".

AN13857 states that a stream containing Embedded Data with DT=0x12 and image data with another data type is considered data type interleaving, and that the i.MX 8M series capture system does not support this mode. The suggested workaround is to configure the sensor to transmit the embedded data using the same data type as the image data.

Unfortunately, this workaround is not possible with our sensor. The sensor transmits standard CSI-2 Embedded Data using DT 0x12, while the image data uses the corresponding RAW data type. The Embedded Data data type cannot be changed to the RAW image data type.

However, we found an older NXP Community discussion specifically concerning the i.MX8MP where NXP TechSupport stated that the CSI hardware should support DT=0x12, while the BSP did not support metadata at that time:

https://community.nxp.com/t5/i-MX-Processors/I-MX8MP-capture-MIPI-embedded-data/td-p/1383845

There is also an older discussion about accessing CSI-2 Embedded Data on i.MX8M devices:

https://community.nxp.com/t5/i-MX-Processors/Does-iMX8M-CSI-support-metadata-embedded-data/m-p/97755...

Therefore, we would like to clarify whether the limitation described in AN13857 applies to the complete hardware path, or mainly to the currently supported ISI/V4L2 capture architecture.

Our main question is:

Is there any way on the i.MX8M Plus to receive or access the payload of CSI-2 packets with Data Type 0x12 while simultaneously capturing the RAW image stream, even if this requires modifications to the Linux driver?

For example, would any of the following approaches be technically possible?

Route DT=0x12 and the RAW image data through different CSI/ISI paths or channels.
Access the Embedded Data before the ISI, directly from the MIPI CSI-2 receiver.
Configure an additional CSI-2 receiver channel/interface for DT=0x12.
Use another DMA or memory path for the Embedded Data.
Extend the NXP Linux CSI/ISI driver to expose the Embedded Data as a V4L2 metadata node or separate /dev/videoX device.
Use any undocumented or currently unsupported hardware functionality of the CSI-2 receiver that would allow the DT=0x12 payload to reach memory.

We do not necessarily require an officially supported V4L2 metadata implementation. If the i.MX8M Plus hardware is capable of transferring the DT=0x12 payload to memory, we would also be interested in implementing the necessary kernel/driver changes ourselves.

Could NXP please clarify whether the restriction described in AN13857 is:

a fundamental hardware limitation of the i.MX8M Plus CSI-2 → Gasket → ISI architecture, meaning that the DT=0x12 payload cannot be transferred to memory while RAW image data is being captured,

or

a software/driver limitation, for which a custom driver implementation could provide access to the Embedded Data?

If it is a hardware limitation, is there any alternative mechanism on the i.MX8M Plus to obtain the Embedded Data payload without changing the camera's CSI-2 output format?

Thank you for any clarification or implementation hints.

Best regards,
Peter

Re: i.MX8M Plus – Is there any way to access MIPI CSI-2 Embedded Data (DT 0x12) together with RAW im

Hello,

The limitation in application note is a hardware architecture limitation of the i.MX8M Plus CSI. The downstream ISI block has no mechanism to separate DT=0x12 payload from the RAW stream.

Unfortunately, there is no supported or officially workaround that mentioned in application note.

Best regards.