topic Re: Sensor-based JPEG encoding on i.MX6UL in i.MX Processors

Sensor-based JPEG encoding on i.MX6UL

kosma_moczek — Tue, 07 Feb 2023 15:40:41 GMT

I'd like to use the OV5640's built-in JPEG encoder on a small cheap i.MX6ULL board. However, I'm struggling to find a way to configure the CSI peripheral for receiving variable-length compressed JPEG data - it seems it only supports fixed-size raw frames. Has anyone successfully used sensor-based compression with i.MX6UL or other i.MX chips?

Re: Sensor-based JPEG encoding on i.MX6UL

joanxie — Wed, 08 Feb 2023 05:03:01 GMT

what format do you want to set in the ov5640? what do you mean variable-length compressed JPEG data ?

Re: Sensor-based JPEG encoding on i.MX6UL

kosma_moczek — Wed, 08 Feb 2023 14:03:55 GMT

To elaborate a little bit:

1. OV5640 has an internal JPEG encoder and it can output the data over DVP/MIPI in different ways (see attachment). Some modes output a fixed size frame with "emulated" lines, others output a single line, and depending on the mode the frame can be padded with dummy data to make it appear fixed size.

2. Looking at IMX6ULLRM, it seems that the End of Frame Interrupt can only be generated based on the pixel/byte count:

Based on this, I think the only way to receive JPEG data from the sensor is to use one of the "emulated" modes where the sensor sends something that looks like a normal, fixed-size frame (RGB888 or Bayer), and then unpack the data in software? Am I correct? Can I program the emulated frame size on the sensor, and then set the same frame size on the CPU?

Another reason I asked is because I would appreciate code examples - looking at the kernel sources (mxc_v4l2_capture.c from Variscite sources) it seems that the MXC V4L2 driver doesn't support JPEG natively, so I suspect some custom driver work might be needed?

static inline int valid_mode(u32 palette) { return ((palette == V4L2_PIX_FMT_RGB565) || (palette == V4L2_PIX_FMT_BGR24) || (palette == V4L2_PIX_FMT_RGB24) || (palette == V4L2_PIX_FMT_BGR32) || (palette == V4L2_PIX_FMT_RGB32) || (palette == V4L2_PIX_FMT_YUV422P) || (palette == V4L2_PIX_FMT_UYVY) || (palette == V4L2_PIX_FMT_YUYV) || (palette == V4L2_PIX_FMT_YUV420) || (palette == V4L2_PIX_FMT_YVU420) || (palette == V4L2_PIX_FMT_NV12)); }

Notably missing from the list is V4L2_PIX_FMT_JPEG.

Re: Sensor-based JPEG encoding on i.MX6UL

kosma_moczek — Wed, 08 Feb 2023 16:37:00 GMT

Another piece of wisdom/research: the newest kernel sources from https://github.com/torvalds/linux have redone this driver, it's now called drivers/media/platform/nxp/imx7-media-csi.c. It's been significantly redone; I will try to run tests with the latest kernel (6.1.10).

Re: Sensor-based JPEG encoding on i.MX6UL

joanxie — Thu, 09 Feb 2023 04:24:49 GMT

yes, you are correct, for csi supported format on imx6ul, you can refer to the Table 19-3. CSI input data format of reference manual, and current CSI couldn't support JPEC data, maybe you need convert them to csi supported format

you also can check driver

https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.24_2.1.0