i.MX6Q: Can the VPU decode JPEG image?

[Tarek]

Hi,

I've tried the following Gst pipeline :

gst-launch filesrc location=sam_max.jpg typefind=true ! vpudec ! imagefreeze ! mfw_isink

The output is a corrupted display. Is there a way to make it work?

Please Note:I know that software decoding is working but that's not what I want.

Thank,

Tarek

[lily_zhang]

Please also try the following command line:

gst-launch filesrc location=./sample.jpg blocksize=file_size ! "image/jpeg" ! vpudec ! filesink location=temp.yuv

[timf]

I try this comand, and I got the following:

Also, Is there a way to decode jpeg in c code? Thanks.

/tmp # gst-launch filesrc location=./hd2.jpg blocksize=904339 ! "image/jpeg" ! vpudec ! filesink location=temp.yuv

Setting pipeline to PAUSED ...

[INFO] Product Info: i.MX6Q/D/S

vpudec versions :smileyhappy:

  plugin: 3.0.1

  wrapper: 1.0.28(VPUWRAPPER_ARM_LINUX Build on Sep  9 2013 17:27:45)

  vpulib: 5.4.6

  firmware: 2.1.5.32515

Pipeline is PREROLLING ...

[INFO] bitstreamMode 1, chromaInterleave 0, mapType 0, tiled2LinearEnable 0

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

[ERR] JpegDecodeHeader failure

Pipeline is PREROLLED ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

Got EOS from element "pipeline0".

Execution ended after 554667 ns.

Setting pipeline to PAUSED ...

Setting pipeline to READY ...

Setting pipeline to NULL ...

Freeing pipeline ...

/tmp #

[jack_mao]

hi,

      In fact, we don't suggest to decode jpeg with VPU,  VPU has capbility to handle the MJPEG stream,  but not fit fot jpeg decode. The customer commonly select the open source to handle this.

gst-launch filesrc location=JPEGFILE ! jpegdec ! freeze ! mfw_isink

jack

[timf]

In our particular video surveillance application, we need to decode dozens of JPEG per second. It is good to have VPU to help out.

BTW, with xiaoli.zhang's example and  jackmaoprevious reply. It turned out that vpudec CAN decode single JPEG file, if the JPEG file is in JFIF format, not EXIF format.

In short, for those who interested, most JPEG on web are in JFIF format. And those from digital cameras are in EXIF format. Open the file in hex mode, if it start with 0xFF 0xD8 0xFF 0xE0, it is in JFIF format.

Here is working example:

( Please replace file_size with actual size)

Generate yuv file:

  gst-launch filesrc location=./sample.jpg blocksize=file_size ! "image/jpeg" ! vpudec ! filesink location=temp.yuv

Display on screen:

gst-launch filesrc location=./sample.jpg blocksize=file_size ! "image/jpeg" ! vpudec ! freeze ! mfw_v4lsink

It would be good to have a gstreamer element that convert EXIF to JFIF.

Thanks you guys all for help.

[Tarek]

Hi Tim,

Are you using i.MX6 ?

I tried your suggestion with a JFIF file that starts with 0xFF 0xD8 0xFF 0xE0 but it didn't display the image !!!

Have you tried more than one file?

[timf]

Sorry for the late reply.

Yes, i am using iMX6, Sabrelite board.

I did test multiple JFIF images.

Remember to replace file_size to the actual file size in the command.

[Tarek]

Thanks Tim. You are right. I forgot to provide the correct file_size. now it's working for me.

gst-launch filesrc location=jpeg420exif.jpg blocksize=770000 !  "image/jpeg" ! vpudec ! imagefreeze ! mfw_isink

However when running the pipeline the CPU load is 29% which is too much? In that case there is not much point from using HW acceleration if it's not saving any CPU horsepower! 

Have you check the CPU load?

[jack_mao]

Hi,

    As I know, the VPU provide jpeg codec, but  do not provide image plugin based on it. You could refer to the mxc_vpu_test to see how  to decode it in code, otherwise you could find opensource image plugin to decode it by gstreamer.

best regards

Jack

[Tarek]

Hi Jack,

If you run gst-inspect vpudec command you will see that JPEG is supported in the current vpudec plug-in. I think we need someone to explain how to construct the pipeline for JPEG image.

SINK template: 'sink'

    Availability: Always

    Capabilities:

      video/mpeg

            mpegversion: 4

      video/x-h264

      video/x-h263

      video/mpeg

           systemstream: false

            mpegversion: { 1, 2 }

      video/x-wmv

             wmvversion: 3

                 format: WVC1

      video/x-wmv

             wmvversion: 3

      video/x-xvid

      video/x-vp8

      image/jpeg

Thanks,

Tarek

[jack_mao]

Hi,

      The image/jpeg means the MJPEG decode, not support the jpeg decode.

jack

[Tarek]

Hi Jack,

I will need to play a life Moving JPEG stream from IP camera. It looks like VPU is not happy with MJPEG formate sent from the camera but I might be able to modify it to fit the VPU needs.

Is there any more details about the MJPEG formate that is supported by the VPU?

Thanks,

Tarek

[jack_mao · Accepted solution]

hi,

    MJPEG must base on the following spec,

    Baseline ISO/IEC 10918-1 JPEG compliance

   JFIF 1.02 input format with up to 3 components

   8-bit samples for each component

   Support up to 4:4:4

Jack

[kamillulko]

Hello,

I used following pipeline to decode mjpeg stream from HD USB camera on SABRE lite board:

gst-launch -v v4l2src num-buffers=-1 ! 'image/jpeg,width=1280,height=720,framerate=30/1' ! queue ! vpudec ! queue ! mfw_v4lsink device=/dev/video16 sync=false

So I was trying to decode MJPEG but with same results as Tarek - image was corrupt, only top part displayed and remaining part of the screen was green. I think something is wrong with MJPEG decoding in current BSP.

[tarekel-sherbin]