How to use v4l2jpegdec in i.mx8x

[toshinariagata]

I cannot decode jpeg using v4l2jpegdec on i.mx8x.

i.MX 8DualX/8DualXPlus/8QuadXPlus Applications Processor Reference Manual(17.3.2 Features) says that the VPU supports JPEG decoding.

The following are the commands and files.

gst-launch-1.0 -v filesrc location='/home/root/test.jpg' ! v4l2jpegdec ! filesink location=test.nv12

The command resulted in the following error.
ERROR: from element /GstPipeline:pipeline0/GstFileSrc:filesrc0: Internal data stream error.

gst-launch-1.0 -v filesrc location='/home/root/test.jpg' ! jpegparse ! v4l2jpegdec ! filesink location=test.nv12

The command succeeded but cannot decode correctly, see test.nv12

Could you please tell me how to use v4l2jpegdec?

[joanxie]

try to add “videocovert " like:

xxx ! jpegparse ! v4l2jpegdec ! queue ! videoconvert ! xxx

[toshinariagata]

Hi. @joanxie 

Thank you for the reply.

I execute the following command, but I cannot decode jpeg.
Is there anything you can find out from the gstreamer logs?
I look forward to hearing from you.

gst-launch-1.0 -v filesrc location='/home/root/test.jpg' ! jpegparse ! v4l2jpegdec ! queue ! videoconvert ! filesink location=test

Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstJpegParse:jpegparse0.GstPad:src: caps = image/jpeg, parsed=(boolean)true, format=(string)I420, width=(int)640, height=(int)480, framerate=(fraction)1/1
/GstPipeline:pipeline0/v4l2jpegdec:v4l2jpegdec0.GstPad:sink: caps = image/jpeg, parsed=(boolean)true, format=(string)I420, width=(int)640, [  900.437676] [VPU Decoder] firmware version is 1.7.0
height=(int)480, framerate=(fraction)1/1

(gst-launch-1.0:646): GStreamer-CRITICAL **: 20:02:15.737: passed '0' as denominator for `GstFraction'
/GstPipeline:pipeline0/v4l2jpegdec:v4l2jpegdec0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)640, height=(int)480, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)bt709, framerate=(fraction)1/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)640, height=(int)480, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)bt709, framerate=(fraction)1/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)640, height=(int)480, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)bt709, framerate=(fraction)1/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)640, height=(int)480, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)bt709, framerate=(fraction)1/1
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)640, height=(int)480, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)bt709, framerate=(fraction)1/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)640, height=(int)480, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)bt709, framerate=(fraction)1/1
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.012520750
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Also, I think using jpegparse and videoconvert will increase the CPU usage due to the software processing.
If possible, is there any way to decode jpegs without using these?

[joanxie]

refer to the command like this:

$ gst-launch-1.0 v4l2src device=/dev/video0 ! jpegparse ! v4l2jpegdec ! queue ! videoconvert ! v4l2h264enc ! h264parse ! matroskamux ! filesink location=out.mkv

[george]

Hello, @joanxie,

toshinariagata-san is my customer.
He wants to clear whether NXP's BSP supports jpeg decode by H/W for i.MX8X.

I found a related article on this Community.
https://community.nxp.com/t5/i-MX-Processors/Hardware-accelerated-JPEG-decoder-on-the-i-MX8X/m-p/952...

Igor has answered a similar question and he advises to ask professional support as the feature is not supported by the BSP.

According to the latest BSP release notes, it still doesn't seem to be supported.

Igor's article was a long time ago, but can I understand that current BSPs also don't support jpeg decode?

BR,
George

[toshinariagata]

Hi. @joanxie.

Thank you for the reply.

I execute the following command, but I get an error.
Is there anything you can find out from the gstreamer logs?
I hope you will be able to provide the information.

root@imx8qxpc0mek:~# gst-launch-1.0 -v v4l2src device=/dev/video3 ! 'image/jpeg,width=1280,height=960,framerate=(fraction)30/1' ! jpegparse ! v4l2jpegdec ! queue ! videoconvert ! v4l2h264enc ! h264parse ! matroskamux ! filesink location=out.mkv
[  635.800098] usb 1-1.1: reset high-speed USB device number 4 using cdns-usb3
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = image/jpeg, width=(int)1280, height=(int)960, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = image/jpeg, width=(int)1280, height=(int)960, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstJpegParse:jpegparse0.GstPad:sink: caps = image/jpeg, width=(int)1280, height=(int)960, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = image/jpeg, width=(int)1280, height=(int)960, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstJpegParse:jpegparse0.GstPad:src: caps = image/jpeg, parsed=(boolean)true, format=(string)UYVY, width=(int)1280, height=(int)960, framerate=(fraction)30/1
/GstPipeline:pipeline0/v4l2jpegdec:v4l2jpegdec0.GstPad:sink: caps = image/jp[  638.625145] [VPU Decoder] firmware version is 1.7.0
eg, parsed=(boolean)true, format=(string)UYVY, width=(int)1280, height=(int)960, framerate=(fraction)30/1
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../../../../git/libs/gst/base/gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reas[  638.689578] [VPU Decoder] msg [0] 25 is missed! ctx is NULL
on not-negotiated (-4)
Execution ended after 0:00:02.493368875
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

I originally wanted to convert jpeg to h264, but found that v4l2jpegdec could not decode it, so I am asking.

[joanxie]

see this topic:

https://community.nxp.com/t5/i-MX-Processors/i-MX8X-transcode-m-jpeg-video-to-h264-with-gstreamer-us...