This objective of this article is to achieve 4K Encode using 2 1920X2160 Encodes on i.MX8M Plus. The idea is to capture a 4K camera feed using i.MX8M Plus EVK and measure the power consumption by reading the GROUP_SOC_FULL data displayed by the BCU tool.
The VPU hardware in i.MX8M Plus doesn’t support 4K Encoder. i.MX8M Plus captures 4K 30fps stream on MIPI CSI interface and split frame into 2 halves and encodes them separately and send them over the network.
We use GStreamer plug-ins to videocrop each 4K frame into two 1920X2160 frames (two halves). Each half is encoded using VPU encoder in H.264 format and transmitted over network. Both halves are decoded separately using the VPU decoder. Decoded halves are then stitched together into reconstructed 4KP30 frame of 3840X2160 resolution. The resulted video feed is then presented on 4K display. The quadrants must be synchronized both at sender side and receiver side.
A conceptual block diagram is shown below.
Sender's pipeline:
gst-launch-1.0 v4l2src device=<CAMERA_PATH> ! "video/x-raw,height=2160,width=3840,framerate=15/1" ! tee name=t \
t. ! videocrop top=0 left=0 right=1920 bottom=0 ! imxvideoconvert_g2d videocrop-meta-enable=true ! v4l2h264enc ! rtph264pay ! udpsink host=<RECEIVER_IP> port=5004 \
t. ! videocrop top=0 left=1920 right=0 bottom=0 ! imxvideoconvert_g2d videocrop-meta-enable=true ! v4l2h264enc ! rtph264pay ! udpsink host=<RECEIVER_IP> port=5005
Receiver's pipeline:
gst-launch-1.0 \
imxcompositor_g2d name=c \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=2160 \
sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=2160 ! waylandsink sync=false \
udpsrc port=5004 ! "application/x-rtp, encoding-name=(string)H264" ! rtpjitterbuffer ! rtph264depay ! h264parse ! v4l2h264dec ! c.sink_0 \
udpsrc port=5005 ! "application/x-rtp, encoding-name=(string)H264" ! rtpjitterbuffer ! rtph264depay ! h264parse ! v4l2h264dec ! c.sink_1
The power measurement obtained was ~1.9W.
Following steps were used to reduce the power:
Part 1: DDR data rate at 2400MTS.
We reduced DDR data rate from 4000MTS to 2400MTS. Change following lines in ~/uboot-nxp/board/freescale/imx8mp_evk/Makefile. Generate a new flash.bin file using a toolchain and flash bootloader.
obj-$(CONFIG_IMX8M_LPDDR4_FREQ0_2400MTS) += lpddr4_timing.o
This reduces the power measurement to ~1.64W.
Part 2: DDR data rate at 2400MTS with nominal drive mode (i.e. VDD_SOC at 0.85V).
In overdrive mode, VDD_SOC is 0.95V. we switch to nominal drive mode where in VDD_SOC is 0.85V. Change following lines in ~/uboot-nxp/board/freescale/imx8mp_evk/Makefile
obj-$(CONFIG_IMX8M_LPDDR4_FREQ0_2400MTS) += lpddr4_timing.o
Add following lines in ~/uboot-nxp/configs/imx8mp_evk_defconfig
CONFIG_IMX8M_LPDDR4_FREQ0_2400MTS=y
CONFIG_IMX8M_VDD_SOC_850MV=y
Also, include all the chages from Part 1.
With new flash.bin, it reduced the power measurement is ~1.35W.
Part 3: Disable Ethernet
While keeping the above optimizations, we disable the ethernet in dts and use below pipeline.
Sender's pipeline:
gst-launch-1.0 v4l2src device=<CAMERA_PATH> ! "video/x-raw,height=2160,width=3840,framerate=15/1" ! tee name=t \
t. ! videocrop top=0 left=0 right=1920 bottom=0 ! imxvideoconvert_g2d videocrop-meta-enable=true ! v4l2h264enc ! rtph264pay ! fakesink \
t. ! videocrop top=0 left=1920 right=0 bottom=0 ! imxvideoconvert_g2d videocrop-meta-enable=true ! v4l2h264enc ! rtph264pay ! fakesink
Also, include all the chages from Part 2.
This reduces the power measurement to ~1.28W.
Part 4: Reduced the input resolution from the camera.
We reduced camera resolution from 4K 3840X2160 to 2560X1440. The camera resolution was reduced within GStreamer pipeline
Sender's pipeline:
gst-launch-1.0 v4l2src device=<CAMERA_PATH> ! "video/x-raw,height=1440,width=2560,framerate=30/1" ! tee name=t \
t. ! videocrop top=0 left=0 right=1280 bottom=0 ! imxvideoconvert_g2d videocrop-meta-enable=true ! v4l2h264enc ! rtph264pay ! udpsink host=<RECEIVER_IP> port=5004 \
t. ! videocrop top=0 left=1280 right=0 bottom=0 ! imxvideoconvert_g2d videocrop-meta-enable=true ! v4l2h264enc ! rtph264pay ! udpsink host=<RECEIVER_IP> port=5005
Receiver's pipeline:
gst-launch-1.0 \
imxcompositor_g2d name=c \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1280 sink_0::height=1440 \
sink_1::xpos=1280 sink_1::ypos=0 sink_1::width=1280 sink_1::height=1440 ! waylandsink sync=false \
udpsrc port=5004 ! "application/x-rtp, encoding-name=(string)H264" ! rtpjitterbuffer ! rtph264depay ! h264parse ! v4l2h264dec ! c.sink_0 \
udpsrc port=5005 ! "application/x-rtp, encoding-name=(string)H264" ! rtpjitterbuffer ! rtph264depay ! h264parse ! v4l2h264dec ! c.sink_1
Also, include all the chages from Part 3.
This reduces the power measurement to ~1.068W.
Part 5: Reduced the input resolution with Ethernet disabled.
We reduced camera resolution from 4K 3840X2160 to 2560X1440 along with disabling ethernet in dts.
Sender's pipeline:
gst-launch-1.0 v4l2src device=<CAMERA_PATH> ! "video/x-raw,height=1440,width=2560,framerate=30/1" ! tee name=t \
t. ! videocrop top=0 left=0 right=1280 bottom=0 ! imxvideoconvert_g2d videocrop-meta-enable=true ! v4l2h264enc ! rtph264pay ! fakesink \
t. ! videocrop top=0 left=1280 right=0 bottom=0 ! imxvideoconvert_g2d videocrop-meta-enable=true ! v4l2h264enc ! rtph264pay ! faeksink
Also, include all the chages from Part 4.
This reduces the power measurement to ~1.005W.
The modified dts files are attached for reference.