Capturing JPEG stream via MIPI CSI

[ofer_livny]

Hi

I have a working setup of an iMX6Q working on a custom board, connected to a camera the streams YUV via MIPI CSI.

The camera supports streaming compressed JPEG data, and I'm now trying to figure out how to correctly configure the CSI-IPU-MEM datapath.

In the reference manual there are no details about the JPEG8 stream support in the IPU (accept for 2 configuration bits with very little details).

Can someone elaborate on that topic?

To be more specific - assuming I configure my stream to JPEG, marked as generic data datatype, how can the IPU indicate an EOF so it can switch to write to a different buffer in memory?

Thanks,

Ofer

[Yuri]

  The IPU of i.MX6 provides the following :

"JPEG8 input is supported only if the data size for each frame can be fixed in advance.
If so, the full stream is transferred transparently to the system memory. The extraction
of embedded information and thumbnails is left to SW."

Have a great day,
Yuri

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[ofer_livny]

Hi Yuri

Thanks for the fast reply.

Can you share the source of the quote you provided? I cannot find it in my documents.

Handling JPEGs the way you describe is actually the same as handling all generic data types, right?

Isn't there a specialized logic in the IPU for dealing with JPEGs ? At least the reference manual states that there are two relevant configuration bits, and I believe supplying an external interrupt to signal the end of frame (of a variable size) is also possible. Can you confirm that?

Thanks again!

Ofer

[Yuri]

1.
  I used IPU (preliminary) specs, which are not intended for publishing, mainly because they
contain many details for different IPU versions, that can only fully disorient customers.

2.

  The fact of absence JPEG8 details in the RM in itself means, that this feature is considered
as one, which related to generic data. 

Have a great day,
Yuri

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[ofer_livny]

Hi Yuri

Assuming I have the ability to know when each JPEG frame streaming is done, can I use the CSI0_FORCE_EOF to let the IPU /IDMAC know its the end-of-frame?

Is there a recommended way to stream JPEG data over mipi csi ?

Thanks!

[Yuri]

Hi, Ofer Livny !

JPEG over MIPI/CSI-2 (as User-Defined 8-bit image format) will provide the EOF events.

~Yuri.

[ofer_livny]

Hi Yuri

Unfortunately I don't see these EOF events happen. Is there anything specific I need to configure to see them happen?

I am currently debugging this behavior - I can stream YUV/RGB successfully, but when it comes to JPEG as generic data - the EOF interrupts happen only when the DMA buffer is full (i.e. after more than 1 frame is streamed, but it is not guaranteed that the last frame in the buffer is whole). My sensor does not support fixed-sized JPEG, so this does not help.

Is there a specific kernel version / camera driver that this feature is supported in ? I didn't find any reference to JPEG streaming in the drivers found in the BSPs freescale supplies.

Ofer

[Yuri]

JPEG8 is intended to be working via MIPI CSI, which should provide VS signalling. Please check
if it is true for the case.

[ofer_livny]

Hi Yuri

As far as I know my sensor supports vsync: The streaming works fine when streaming YUV/RGB, only when streaming JPEG it doesn't.

Is there a way I can verify that on the IPU side? I'm afraid there's no MIPI CSI2 sniffer available to me for that purpose.

[Yuri]

What camera / configuration is used in the case ?

[ofer_livny]

I'm using an stv0987 image processor, streaming MIPI CSI-2 using 2 data lanes. I use the CSI-SMFC-IDMAC path (no IC).

I am able to stream YUV/RGB. But when streaming JPEG/Generic data - I do not see an end-of-frame indication per each frame. I only get it when the actual memory buffer I defined is filled (usually with an incomplete last frame) .

I can't find a way to monitor the vsync indication, as it is internal to the CSI-IPU.

Setting the FORCE_EOF bit does not seem to do anything. Same goes to setting the two not-so-documented JPEG8 bits in IPU_CSI0_SENS_CONF.  Resetting the IDMAC causes an "new frame before end of frame" indication.

Your help is much appreciated :smileyhappy:

[Yuri]

  I think EOF interrupt issue means, that the frame data size

is not configured correctly in IDMAC CPMEM. Please use such
parameters as Frame Width and Frame Height.

Have a great day,
Yuri

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[yan_]

Hi Yuri,

Since we're working in MIPI/CSI-2 mode and not in parallel, the bit-fields CSI0_DATA_WIDTH and CSI0_SENS_DATA_FORMAT of register

IPUx_CSI0_SENS_CONF are ignored. Because all relevant information should be supplied to IPU from MIPI controller by the help of  MCT_DI bus.

As result, we can't supply any frame data size or any kind of data format.

What we should configure is -

CSI0 Data Identifier Register (IPUx_CSI0_DI) and its configured for 1st Channel with value 0x30 - which means that incoming MIPI data type is JPEG.

From IMX6DQRM.pdf:

"37.4.3.2.2

High-speed serial interface - MIPI (Mobile Industry Processor

Interface).

In MIPI interface two values arrive in each cycle. Each value is 8 bit wide, which means

16 MSB bits of the data bus input are treated, while 4 LSB bits are ignored.

When working in this mode, the CSI can handle up to 4 streams of data. Each stream is

identified with DI (data identifier) that includes the virtual channel and the data type of

this stream. Each stream that is handled is defined in registers MIPI_DI0-3. Only the

main stream (MIPI_DI0) can be sent to all destination units while the other streams are

sent only to the SMFC as generic data.

In this mode SENS_DATA_FORMAT and DATA_WIDTH registers are ignored, since

this information is coming to the CSI via the MCT_DI bus."

Thanks a lot,

Yan

[dehuanxin]

just here to say thank you.

my sensor uses mipi user-defined type 0 (0x30).

i've been struggling for weeks to get eof to work and you point me to the right direction.

thank you very much!

[Yuri]

Yan, good day !

According to Table 19-1 (Input signals to CSI2IPU) of the i.MX6 DQ Reference Manual ()
the following data types are considered :
• 00h–07h—Synchronization short packet data types

• 08h–0Fh—Generic short packet data types

• 10h–17h—Generic long packet data types

• 18h–1Fh—YUV data

• 20h–27h—RGB data

• 28h–2Fh—RAW data

• 30h– 37h—User defined byte-based data

• 38h–3Fh—Reserved

As we can see, JPEG in itself is not present, it may be treated as RAW or User defined data.

~Yuri.

[ofer_livny]

Hi Yuri

You are correct. We configure the CSI2IPU to use defined byte-based data. (0x30)

However, down the pipe the IPU CSI has got JPEG related configuration, which is what I am asking about...

I opened a service request on this issue (SR# 1-2197709001).

Regards,

Ofer

[Yuri]

As mentioned in SR #1-2197709001 :
"When streaming JPEG frames to the IPU, it gives EOF interrupts
only when filling a memory buffer, and NOT per frame."

~Yuri.

[aravinthkumarja]

Hi Yuri,

Same thing we are using, but we didn't get EOF interrupts.

we got this error,

ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0

VIDIOC_DQBUF failed.

ERROR: v4l2 capture: VIDIOC_QBUF: buffer already queued

VIDIOC_QBUF failed

How can i fix this.,

[Yuri]

Hello,

  Please refer to the following :

CSI preview demo not working!

Unable to capture video on IMX6Q

Regards,

Yuri.

[ofer_livny]

Hi Yuri

Since there is no way for me to know the exact data size of each frame (I'm not able to compress the jpeg to specific fixed size), how should I configure correctly the CPMEM ? Do the frame width/height parameters really matter in my case?

Is it related to the EOLI (end of line interrupt) ?

Thanks!

Ofer

[ofer_livny]

Hi Yuri

Thanks again.

So in fact there's no difference between JPEG and generic data.

How about using an external interrupt to signal the end of a variable sized frame? Is it possible ?

To be more specific - if my software is able to poll on a camera register that signals that the stream has ended, can it write to an IPU register to force this EOF indication?