Ryan Erb

i.MX6 GStreamer-imx Plugins - Tutorial & Example Pipelines

Discussion created by Ryan Erb on Sep 23, 2015
Latest reply on Apr 22, 2017 by Mike Taylor

The Freescale i.MX6 has many video capabilities that are best accessed through GStreamer.

 

Gateworks, the leading supplier of Powerful ARM based Single Board Computer solutions using the Freescale i.MX6, has invested countless engineering hours researching and mastering GStreamer for the i.MX series of processors. Gateworks would like to share this GStreamer research with the rest of the i.MX community of developers! Visit the Gateworks Software Wiki GStreamer Page.

 

There are two main versions of GStreamer: 0.10 and 1.0. Version 1.0 is now the latest standard. gstreamer-1.0

 

The i.MX6 processor has hardware blocks such as the IPU (image processing unit), VPU (video processing unit), and GPU (graphical processing unit). The main advantage of using these hardware blocks is that there is no CPU cost for decoding/encoding a stream because another hardware block in the i.MX6 takes care of it. This leaves your CPU free to deal with other programs etc.

 

The GStreamer app works with 'plugins'. A plugin comprises of elements that can do work on a media stream. For example, the imxvpudec is a VPU based decoder plugin.

 

This post is specifically about the plugins. There are different versions and sets of plugins available.

 

Gateworks has chosen to use the GStreamer-imx plugins for the following reasons:

 

  • Open Source Development model: The project is on github and is very active
  • The main developer has been a GStreamer contributer for some time now and is very active in the GStreamer community
  • The source is very well documented and easy to follow
  • Things are done in a very standard GStreamer way

 

 

Plugin List

For a thorough description of each plugin, and why to use it, please visit the Gateworks Software Wiki GStreamer Page

 

TypePlugin(s)Element(s)Comments
Audio DecoderimxaudioimxuniaudiodecUses i.MX uniaudio codecs for decoding
Audio Encoderimxaudioimxmp3audioencUses i.MX for MP3 audio encoding
Device Sourcesimxv4l2videoimxv4l2videosrcGet camera source via v4l2
Video DecoderimxvpuimxvpudecVPU Based decoder
Video Encoderimxvpuimxvpuenc_mjpeg; imxvpuenc_mpeg4; imxvpuenc_h264; imxvpuenc_h263VPU Based encoders
Video Render (sink)imxg2d; imxpxp; imxeglvivsink; imxipuimxg2dvideosink; imxpxpvideosink; imxeglvivsink; imxipuvideosinkg2d1, ipu1, pxp2, and egl (overlay) video sinks
Video Converterimxg2d; imxpxp; imxipuimxg2dvideotransform; imxpxpvideotransform; imxipuvideotransformg2d, pxp, egl and ipu video filter/converter/scalars3
Video Compositingimxg2d; imxipuimxg2dcompositor, imxipucompositorgpu/ipu accelerated compositing

1. The g2d sink is very flexible in the types of input video it can take, but doesn't have the ability to convert to as many formats as the IPU can. On the other hand, the IPU is very picky with it's input (e.g. requiring a 1px offset) and the kernel driver is very undocumented, but as stated before, it can convert between many colorspace formats.

2. Note that the PXP sinks are only applicable to the i.mx6solo and i.mx6dl processors.

3. Please see note 1 above.

 

 

Plugin Example Pipeline

For example, to encode a video from a camera on /dev/video2 into h.264 and save it to a file:

 

#Take camera input /dev/video2, encode it to h264 at a bitrate of 10mbit/s (CBR) and save to a file.
gst-launch-1.0 imxv4l2videosrc device=/dev/video2 ! imxvpuenc_h264 bitrate=10000 ! filesink location=/tmp/file.mp4

 

Many more pipeline examples are described and listed on the Gateworks Software Wiki GStreamer Pipelines page

 

Summary

Using GStreamer 1.0 with the GStreamer-imx plugins is a powerful way to access and apply the multimedia capabilities of the Freescale i.MX6 processors!

 

If there are other examples you would like to see, please add to the discussion!

 

 

Arpan Chakravarty

Kevin Wong

Outcomes