vpuenc framerate data missing (vui parameters, timing_info)

[josepholivier]

I am using the VPU on the imx6 to generate H.264-encoded video that is ultimately displayed on digital televisions. The H.264 stream is packaged into an MPEG transport stream and then udp-streamed to the destination.

Display on many TV's is jerky and ugly. I believe this is because the H.264 stream lacks framerate data. Specifically, in the H.264 SPS (sequence parameter set) packet, the vui_parameters_present_flag is always false. Videos which display properly always have this flag set, and correct values in the timing_info structure (timing_info_present_flag, num_units_in_tick, time_scale). It appears that this data is not being generated properly by the VPU (or vpuenc gstreamer plugin).

My pipeline uses an appsrc to produce frames instead of the videotestsrc below, but is otherwise identical to:

gst-launch              \

     videotestsrc       \

   ! video/x-raw-rgb,bpp=16,depth=16,endianness=1234,red_mask=63488,green_mask=2016,blue_mask=31,width=1280,height=720,framerate=30/1 \

   ! mfw_ipucsc \

   ! vpuenc codec=6 seqheader-method=3 framerate-nu=30000 framerate-de=1000 \

   ! mpegtsmux \

   ! udpsink host=[host-ip] port=7890

Here's the version output for my imx/fsl gstreamer plugins:

IPU_CSC_CORE_LIBRARY_VERSION_INFOR_01.00.

MFW_GST_IPU_CSC_PLUGIN 3.0.7 build on Jan  8 2014 15:20:51.

[INFO] Product Info: i.MX6Q/D/S

vpuenc versions :smileyhappy:

plugin: 3.0.7

wrapper: 1.0.35(VPUWRAPPER_ARM_LINUX Build on Jan  8 2014 14:33:14)

vpulib: 5.4.12

firmware: 2.1.9.36350

I'm using Yocto Poky Dylan 9.0.0.

[adrian_alonso]

Hi Joseph,

One last detail can you tell me which iMX6 target board are you using?

Regards

[imxcommunitysco]

Branched to internal group.

[karina_valencia]

AdrianAlonso Apr 8, 2014 3:06 PM (in response to imxcommunityscout)

Hi,

Tested iMX6 YoctoProject rootfs are based on "dora" release branch;

Latest Dylan compatibility is a work on progress which will be available at

the same time Dylan reach a stable state release by end on April 2014.

Can you build and test based on "dora" branch release.

Regards

Adrian

[josepholivier]

I have built and tested on my hardware using Dora, still with no VUI parameter data. My version number output is now:

IPU_CSC_CORE_LIBRARY_VERSION_INFOR_01.00.

MFW_GST_IPU_CSC_PLUGIN 3.0.9 build on Apr 10 2014 15:46:09.

[ 2880.479949] No such IOCTL, cmd is 22032

[ 2880.483981] No such IOCTL, cmd is 22032

Setting pipeline to PAUSED ...

[INFO] Product Info: i.MX6Q/D/S

vpuenc versions :smileyhappy:

  plugin: 3.0.9

  wrapper: 1.0.40(VPUWRAPPER_ARM_LINUX Build on Apr 10 2014 15:35:17)

  vpulib: 5.4.18

  firmware: 2.3.10.40778

As you can see, we've improved our vpuenc plugin, wrapper, vpulib and firmware version. Looking at the code, the VPUENC gstreamer plugin receives the SPS and PPS data directly from the VPU encoder hardware, and this data does not include VUI parameters.

There is also a message regarding IOCTLs, which stems from my using an older Linux kernel than the Dora release. This was done to test faster, since it's unlikely that the kernel version directly affects action at the H.264 SPS VUI data level.

I have slightly modified my gstreamer launch script and pipeline like so:

#!/bin/bash

HOST=$1

PORT=$2

if [[ -z $HOST ]]; then

  echo No host provided

  echo Usage: \n\tgstreamer-pipelines.sh HOST PORT

  exit 1

fi

if [[ -z $PORT ]]; then

  echo No port provided

  echo Usage: \n\tgstreamer-pipelines.sh HOST PORT

  exit 1

fi

gst-launch-0.10 -v \

  videotestsrc pattern="smpte" horizontal-speed="1" !\

  video/x-raw-rgb,bpp=16,depth=16,endianness=1234,red_mask=63488,green_mask=2016,blue_mask=31,width=1280,height=720,framerate=30/1 !\

  queue !\

  mfw_ipucsc !\

  queue !\

  vpuenc codec=avc bitrate=4000000 seqheader-method=3 framerate-nu=30000 framerate-de=1000 !\

  queue !\

  mpegtsmux !\

  udpsink host=$HOST port=$PORT