VPU for MJPG decoding in IMX8MP?

[jake4]

I have a USB camera that gives MJPG (Motion-JPEG, compressed ) 

And I'm using IM8MP with VPU Hantro

Is there any VPU function that we can use for decompression in this case?

This document breifly describes

https://docs.nxp.com/bundle/RM00294/page/topics/imx_6_vpu_main_features.html

This pipeline take 60% cpu

gst-launch-1.0 v4l2src device=/dev/video2 ! image/jpeg,width=1280,height=720 ! jpegdec ! imxvideoconvert_g2d ! fakesink

Is there any VPU function we can rely on to reduce the CPU?

[jake4]

Hi 

Even when i try to run this 

gst-launch-1.0 videotestsrc ! imxvideoconvert_g2d ! queue ! vpuenc_hevc ! multiqueue ! vpudec ! fakesink
Setting pipeline to PAUSED ...
====== VPUDEC: 4.10.0 build on Nov 27 2025 18:04:02. ======
wrapper: 3.0.0 (VPUWRAPPER_ARM64_LINUX Build on Nov 27 2025 18:03:51)
vpulib: 1.1.1
firmware: 1.1.1.836202415
====== VPUENC: 4.10.0 build on Nov 27 2025 18:04:02. ======
wrapper: 3.0.0 (VPUWRAPPER_ARM64_LINUX Build on Nov 27 2025 18:03:52)
vpulib: 1.1.1
firmware: 1.1.1.43690
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
0:07:59.5 / 99:99:99.

It takes 199 % of CPU is the CPU getting used?

perf shows 

7.82% libgstvideo-1.0.so.0.2600.0 [.] 0x00000000000219e4
6.63% libhantro_vc8000e.so.1 [.] EncAsicFrameStart
5.76% libgstvideo-1.0.so.0.2600.0 [.] 0x00000000000219f8
5.49% [kernel] [k] DecRefreshRegs
3.92% libgstvideo-1.0.so.0.2600.0 [.] 0x00000000000219e0
3.89% libgstvideo-1.0.so.0.2600.0 [.] 0x00000000000219f0
3.89% libgstvideo-1.0.so.0.2600.0 [.] 0x0000000000021a00
3.05% libhantro.so.1 [.] SwShowBits
2.94% [kernel] [k] DecFlushRegs.isra.0
2.69% libgstvideo-1.0.so.0.2600.0 [.] 0x00000000000219ec
1.45% libc.so.6 [.] 0x00000000000a3194
1.23% libc.so.6 [.] 0x00000000000a3188
1.21% orcexec.0La54a (deleted) [.] 0x0000000000000040
1.19% libgstvideo-1.0.so.0.2600.0 [.] 0x00000000000219e8
1.16% libhantro.so.1 [.] DWLPrivateAreaReadByte
0.80% libhantro_vc8000e.so.1 [.] VCEncStrmEncodeExt
0.77% orcexec.0La54a (deleted) [.] 0x0000000000000044
0.69% orcexec.0La54a (deleted) [.] 0x0000000000000038
0.56% libhantro.so.1 [.] SwFlushBits
0.53% [kernel] [k] _raw_spin_unlock_irqrestore
0.48% libgstvideotestsrc.so [.] 0x0000000000008450
0.48% libgstvideotestsrc.so [.] 0x0000000000008448
0.46% libc.so.6 [.] 0x00000000000a3184
0.46% libhantro_vc8000e.so.1 [.] EncAsicGetRegisterValue
0.46% libc.so.6 [.] 0x00000000000a318c
0.45% orcexec.0La54a (deleted) [.] 0x0000000000000034
0.35% libhantro.so.1 [.] FlushDecRegisters
0.34% libhantro.so.1 [.] RefreshDecRegisters
0.33% [kernel] [k] el0_svc
0.32% libgobject-2.0.so.0.8400.4 [.] g_type_check_instance_is_fundamentally_a
0.32% libhantro.so.1 [.] HevcDecDecode
0.31% libhantro.so.1 [.] SetDecRegister
0.30% orcexec.0La54a (deleted) [.] 0x000000000000012c

Not sure VPU is being utilized. How to confirm?

lsof | grep hantro
gst-launc 1516 root mem REG 179,1 224656 2117 /usr/lib/libhantro.so.1
gst-launc 1516 root mem REG 179,1 353928 2322 /usr/lib/libhantro_vc8000e.so.1
gst-launc 1516 root mem CHR 510,0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 root 15u CHR 510,0 0t0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 root 31u CHR 511,0 0t0 161 /dev/mxc_hantro
gst-launc 1516 1517 gst-launc root mem REG 179,1 224656 2117 /usr/lib/libhantro.so.1
gst-launc 1516 1517 gst-launc root mem REG 179,1 353928 2322 /usr/lib/libhantro_vc8000e.so.1
gst-launc 1516 1517 gst-launc root mem CHR 510,0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 1517 gst-launc root 15u CHR 510,0 0t0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 1517 gst-launc root 31u CHR 511,0 0t0 161 /dev/mxc_hantro
gst-launc 1516 1518 gst-launc root mem REG 179,1 224656 2117 /usr/lib/libhantro.so.1
gst-launc 1516 1518 gst-launc root mem REG 179,1 353928 2322 /usr/lib/libhantro_vc8000e.so.1
gst-launc 1516 1518 gst-launc root mem CHR 510,0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 1518 gst-launc root 15u CHR 510,0 0t0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 1518 gst-launc root 31u CHR 511,0 0t0 161 /dev/mxc_hantro
gst-launc 1516 1519 pool-spaw root mem REG 179,1 224656 2117 /usr/lib/libhantro.so.1
gst-launc 1516 1519 pool-spaw root mem REG 179,1 353928 2322 /usr/lib/libhantro_vc8000e.so.1
gst-launc 1516 1519 pool-spaw root mem CHR 510,0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 1519 pool-spaw root 15u CHR 510,0 0t0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 1519 pool-spaw root 31u CHR 511,0 0t0 161 /dev/mxc_hantro
gst-launc 1516 1520 multiqueu root mem REG 179,1 224656 2117 /usr/lib/libhantro.so.1
gst-launc 1516 1520 multiqueu root mem REG 179,1 353928 2322 /usr/lib/libhantro_vc8000e.so.1
gst-launc 1516 1520 multiqueu root mem CHR 510,0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 1520 multiqueu root 15u CHR 510,0 0t0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 1520 multiqueu root 31u CHR 511,0 0t0 161 /dev/mxc_hantro
gst-launc 1516 1521 queue0:sr root mem REG 179,1 224656 2117 /usr/lib/libhantro.so.1
gst-launc 1516 1521 queue0:sr root mem REG 179,1 353928 2322 /usr/lib/libhantro_vc8000e.so.1
gst-launc 1516 1521 queue0:sr root mem CHR 510,0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 1521 queue0:sr root 15u CHR 510,0 0t0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 1521 queue0:sr root 31u CHR 511,0 0t0 161 /dev/mxc_hantro
gst-launc 1516 1522 videotest root mem REG 179,1 224656 2117 /usr/lib/libhantro.so.1
gst-launc 1516 1522 videotest root mem REG 179,1 353928 2322 /usr/lib/libhantro_vc8000e.so.1
gst-launc 1516 1522 videotest root mem CHR 510,0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 1522 videotest root 15u CHR 510,0 0t0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 1522 videotest root 31u CHR 511,0 0t0 161 /dev/mxc_hantro
gst-launc 1516 1523 gmain root mem REG 179,1 224656 2117 /usr/lib/libhantro.so.1
gst-launc 1516 1523 gmain root mem REG 179,1 353928 2322 /usr/lib/libhantro_vc8000e.so.1
gst-launc 1516 1523 gmain root mem CHR 510,0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 1523 gmain root 15u CHR 510,0 0t0 162 /dev/mxc_hantro_vc8000e
gst-launc 1516 1523 gmain root 31u CHR 511,0 0t0 161 /dev/mxc_hantro

[Manuel_Salas]

Hello @jake4 

I hope you are doing very well.

Could you please try the below pipeline:

gst-launch-1.0 v4l2src device=/dev/video2 ! image/jpeg,width=1280,height=720 ! v4l2jpegdec ! imxvideoconvert_g2d ! fakesink

It is not recommended to use videotestsrc. This is a pure software generator. It uses the CPU to create every single pixel of the test pattern.

Best regards,

Salas.

[jake4]

Is there any other recommended method to acheive this?

we have a USB camera that gives the MPJG compressed image, we need to process it with hardware acceleration in order not to load the CPU,  is there any recommendation?

[jake4]

Hi @Manuel_Salas
getting this
gst-launch-1.0 v4l2src device=/dev/video3 ! image/jpeg,width=1280,height=720 ! v4l2jpegdec ! imxvideoconvert_g2d ! fakesink
WARNING: erroneous pipeline: no element "v4l2jpegdec"

Thanks

gst-inspect-1.0 | grep v4l2
video4linux2: v4l2convert: V4L2 Video Converter
video4linux2: v4l2deviceprovider (GstDeviceProviderFactory)
video4linux2: v4l2h264dec: V4L2 H264 Decoder
video4linux2: v4l2h264enc: V4L2 H.264 Encoder
video4linux2: v4l2h265dec: V4L2 H265 Decoder
video4linux2: v4l2h265enc: V4L2 H.265 Encoder
video4linux2: v4l2radio: Radio (video4linux2) Tuner
video4linux2: v4l2sink: Video (video4linux2) Sink
video4linux2: v4l2src: Video (video4linux2) Source
video4linux2: v4l2vp8dec: V4L2 VP8 Decoder
video4linux2: v4l2vp9dec: V4L2 VP9 Decoder

[Manuel_Salas]

Hello @jake4 

Please try the below:

$ gst-launch-1.0 v4l2src device=/dev/video3 ! image/jpeg,width=1280,height=720 ! jpegparse ! vpudec ! imxvideoconvert_g2d ! fakesink

The vpudec uses the Hantro VPU.

Let me know if this worked for you.

Best regards,

Salas.

[jake4]

Hi @Manuel_Salas 

Running this gives the error as below.

gst-launch-1.0 v4l2src device=/dev/video2 ! image/jpeg,width=1280,height=720 ! jpegparse ! vpudec ! imxvideoconvert_g2d ! fakesink
WARNING: erroneous pipeline: could not link jpegparse0 to vpudec0

gst-inspect-1.0 vpudec gives this as below, so seems it cant handle jpeg 

Thanks

Factory Details:

Rank secondary (128)
Long-name IMX VPU-based video decoder
Klass Codec/Decoder/Video
Description Decode compressed video to raw data
Author Multimedia Team <[email protected]>

Plugin Details:
Name vpu
Description VPU video codec
Filename /usr/lib/gstreamer-1.0/libgstvpu.so
Version 4.10.0
License LGPL
Source module imx-gst1.0-plugin
Binary package Freescle Gstreamer Multimedia Plugins
Origin URL http://www.freescale.com

GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoDecoder
+----GstVpuDec

Element Flags:

Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-h265
video/x-vp9
video/x-vp8
video/x-h264

SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
format: { (string)NV12, (string)I420, (string)YV12, (string)Y42B, (string)NV16, (string)Y444, (string)NV24, (string)NV12_10LE }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]

[jake4]

Hi @Manuel_Salas 

I'm using IMX8MPlus does that support jpeg VPU decoding?

Thanks

[jake4]

Hi joanxie

Can you please advise?

Thanks