iMX8MM Encoder High Bandwidth

[tom_perman]

We are developing a product which uses the iMX8M Mini to encode 1080p30 video, captured from its camera interface. 

We have a requirement that when the input is static (or almost static), the bandwidth of the encoded video stream needs to be <100kbps. This should be achievable with an efficient H.264 encoder, and we already have a solution developed which achieves this using the iMX6DL. 

However when we test the iMX8MM we find that for some static inputs the bandwidth of encoded video is >5Mbbps. This doesn't make sense to us, since there is no change in the input from frame to frame, and therefore the output encoded video bandwidth should be low, meeting our requirement. 

Has anyone else noticed behaviour like this for whilst using the iMX8MM encoder? Is there anyone at NXP who can help us with this?

Here is the test command we have used, showing the set of encoder parameters we use. road_30s.bgr0 is 900 frames of raw video, where every frame is an identical image of a road. out.h264 is the output encoded video, which had a bitrate of ~6Mbps. This seems at least an order of magnitude higher than it should be. 

/unit_tests/VPU/hantro/h264_testenc \
    --input /tmp/road_30s.bgr0 \
    --output /tmp/out.h264 \
    --firstPic 1 \
    --lastPic 900 \
    --outputRateNumer 30 \
    --outputRateDenom 1 \
    --inputRateNumer 30 \
    --inputRateDenom 1 \
    --lumWidthSrc 1920 \
    --lumHeightSrc 1080 \
    --inputFormat 11 \
    --colorConversion 1 \
    --viewMode 0 \
    --level 42 \
    --byteStream 1 \
    --quarterPixelMv 0 \
    --enableCabac 0 \
    --bitPerSecond 10000000 \
    --picRc 1 \
    --mbRc 1 \
    --gopLength 30 \
    --intraPicRate 0 \
    --intraQpDelta 0 \
    --fixedIntraQp 51 \
    --refFrameAmount 1 \
    --longTermPicRate 15

[ari-san_24]

Is this the only level where the H264 profile can be changed? What if we don't have an HX280 but a VC8000e?

[cristiano_santoni]

Hi James,
I'm Cristiano, a colleague of Tom, working on the same project.
Thanks for your suggestion, switching to the Main profile does makes things better in terms of bandwidth.
We've also found that increasing the qpMin to 12 helps and we are now at an average bitrate of 950 kbps, with the P-Frames stabilizing at a size of 3.5KB (860kbps @ 30 fps).
Here's the command line we're currently using:

/unit_tests/VPU/hantro/h264_testenc \
  --input /tmp/road_30s.bgr0 \
  --output /tmp/out.h264 \
  --firstPic 1 \
  --lastPic 900 \
  --outputRateNumer 30 \
  --outputRateDenom 1 \
  --inputRateNumer 30 \
  --inputRateDenom 1 \
  --lumWidthSrc 1920 \
  --lumHeightSrc 1080 \
  --inputFormat 11 \
  --colorConversion 1 \
  --viewMode 0 \
  --level 42 \
  --byteStream 1 \
  --quarterPixelMv 0 \
  --enableCabac 1 \
  --bitPerSecond 10000000 \
  --picRc 1 \
  --mbRc 1 \
  --gopLength 30 \
  --qpMin 12 \
  --qpMax 51 \
  --intraPicRate 0 \
  --intraQpDelta 0 \
  --fixedIntraQp 51 \
  --refFrameAmount 1 \
  --longTermPicRate 15

The average size and content of the P-Frames, however, is still confusing: analyzing the produced frames we noticed that after about 20 frames that iMX8MM stabilizes both MB QP values (12 for almost all MBs) and frame sizes (3.5KB), without trying to save on MBs which have already reached qpMin.
By comparison the i.MX6DL sends very little information once the best quality has been reached, resulting in an average P-frames bitrate of just 50 kbps.
Attached you can find screenshots of ffplay with with the option "-debug vis_mb_type" that shows the difference in produced MBs between the i.MX6DL and i.MX8MM. These screenshots have been taken after frame 30, at which point both stream are stable there are no further changes in the produced frames until the end of the stream.

Any idea why the rate control is behaving this way and whether there is an option to push the efficiency of the encoding further?

[jamesbone]

I think the profile in the Linux version that you are using it is in Base Profile and it is making the delay.

In VPU_H1 Register 18 fields (SWREG18).

From the above register definition,
  - Setting Bit 21 will make encoder set in High Profile
  - Setting Bit 18 will make encoder set in Main Profile
  - Clearing Bit 18 will make encoder set in Base Profile

In the kernel corresponding driver or Hantro VPU Encoder is "drivers/mxc/hantro_845_h1/hx280enc.c"

You can try changing the H264 Profile to HIGH by configuring VPU_H1 Register 18, in the "hx280enc_init(void)" function present in the encoder driver file "drivers/mxc/hantro_845_h1/hx280enc.c" as follows,

static int __init hx280enc_init(void)
{
u32 vpu_h1_reg18;

---
---

+    vpu_h1_reg18 = readl(hx280enc_data.hwregs + 0x48);
+    pr_info("hx280enc: vpu_h1_reg18 <%x>\n", vpu_h1_reg18);
+    vpu_h1_reg18 |= (1 << 21);                                 /*H.264 Transform 8x8 enable. High Profile H.264. transform8x8Mode*/
+    pr_info("hx280enc: vpu_h1_reg18 <%x>\n", vpu_h1_reg18);
+    writel(vpu_h1_reg18, hx280enc_data.hwregs + 0x48); /* SWREG18 */

}