i.MX8M Mini support for JPEG encoding

[vincentz63]

I noticed that VPU H1 Register 14 (SWREG14) allows to select between VP8, JPEG or H264; does the VPU_H1 support JPEG? If so, how to use it? gst-inspect-1.0 only returns jpegenc, which is the standard software encoder...

The register is documented in i.MX 8M Mini Applications Processor Reference Manual, Rev. 1, 03/2019, page 4805.

[vincentz63]

Hi,

The only relevantLinux doc I have is "i.MX Linux® User's Guide, Rev. L4.14.98-2.0.0_ga, 04/2019", but it does not contain a section 9.5.4.

Section 7.3.3 does states support for MJPEG via GStreamer's vpuenc_jpeg, but that does not seem to exist for the i.MX8MMini. "gst-inspect-1.0 | grep vpu" yields "vpuenc_h264 and vpuenc_vp8".

GStreamer's v4l2src has an image/jpeg pad and its 'io-mode' property can be set to 'mmap'; is this what you're implying I should use to target the h/w JPEG encoder?

Thanks

[vincentz63]

Hello,

Any change you could look into my reply from 14th August?

Do you mean that the JPEG core of the i.MX8MMini was bought from Alma (who appear to sell an IP called JPEG-E-X)?

Do you mean that there's no GStreamer API for the JPEG encoder and that instead we must directly target the V4L2 API to make use of the encoder?

Thanks

JP

[amigen-jason]

Hello JP,

Were you ever able to confirm JPEG HW encode/decode support on the H1? 

[vincentz63]

-

[Bio_TICFSL]

Hello JP,

The JPEG Encoder consists of a JPEG-E-X core and a JPEG Encoder Wrapper (JPGENCWRP). Similarly, the JPEG Decoder consists of a JPEG decoder core (JPEG-DX) and its corresponding wrapper

The driver relies on the V4L2 framework, For more information on streaming I/O, see Streaming I/O (Memory Mapping).

3.2. Streaming I/O (Memory Mapping) — The Linux Kernel documentation 

See section 9.5.4 of you Linux ref manual. for usage.

Regards

[amigen-jason]

Do you then confirm that there is a HW encoder and decoder on the i.MX 8M Mini?  Is it contained within the H1 VPU?  If so, it is also present in the i.MX 8M Plus?

[bart_meeuwissen]

Hi Amigen-jason, 

Did you get any insight on the hardware jpeg encoder in the i.MX 8M Plus. At this point we also run into trouble trying to get it to work.

If I look at the documentation of VeriSilicon about the VPU used (Hantro VC8000E) it actually has support for (M)JPEG, but software libraries from VeriSilicon integrated in NXP BSP are lacking the functions for it (not in binary, but strangely the header files for it do exist). Furthermore, when I look into the iMX8MPlus reference manual the JPEG block is not shown in the block diagram of the VC8000E while it is in the documentation of VeriSilicion, did NXP remove it?

If anyone has information regarding this, we are interested.

regards,

Bart Meeuwissen

[amigen-jason]

I got confirmation that:

• The i.MX 8M Mini and Plus have the same decoder but the encoder is different
• The JPEG section was removed because JPEG support could not be added to the BSP due to a H1 hardware limitation.  Not sure what that means exactly other than, presumably, the Hantro really is missing JPEG support.  

What Hantro VC8000E documentation do you have access to?  Could I get a link if it is public?  

[bart_meeuwissen]

The block diagram of the Hantro VC8000E is on the public site of verisilicon:

https://www.verisilicon.com/en/IPPortfolio/HantroVC8000E

It does show a JPEG encoder. A similar diagram taken from the imx8mplus reference guide, has a modified block diagram:

Apart from the missing jpeg encoder they look very similar. When we realised this we where kind of shocked that it wouldn't be able to generate JPEG or MJPEG hardware accelerated. We are not sure if there are good alternatives, like:

• JPEG encoding in the ISP ? some registers hint to this feature
• JPEG encoding using the GPU, so far we haven't been successful in getting this to work
• Software encoding using a core works ofcourse, but is to slow for our application.

Somehow we can't believe that this processor would lack this feature. If anyone has a hint on this subject, we are very interested.

thanks,

Bart

[amigen-jason]

I agree, missing sort sort of JPEG acceleration is HUGE miss on this part.  In all of our applications we need USB-UVC support and adding h.264 support to the Linux kernel has been...difficult.  I have demonstrated h.264 over UVC with on a 4.14 kernel but didn't natively work in with all hosts (namely Windows built-in webcam viewer). 

I have successfully used MJPEG over UVC via libturbojpeg, which is the best SW encoder I can find.  The best architecture we can dream up given the lacking HW capabilities is to use the 2D GPU to CSC the image to YUV prior to feeding libturbojpeg so that at least the first stage of JPEG encoding is done in HW.  And as libturbojpeg uses NEON to accelerate encoding that's about the best I think we can do.

I'm still working with PRO support to find a better HW solution but it's slow and difficult to find someone who is really in the know.

[KlausHaag]

Did you find a solution here?