gstreamer mfw_v4lsink YUY2 not displaying anything

[markurup]

I'm having some trouble with mfw_v4lsink in gstreamer:

Working:

root@flexpad /$ gst-launch-0.10 -v videotestsrc ! video/x-raw-yuv,format=\(fourcc\)I420 ! mfw_v4lsink &

root@flexpad /$ MFW_GST_V4LSINK_PLUGIN 1.9.6 build on Jan 27 2012 11:18:05.

Setting pipeline to PAUSED ...

/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)320, height=(int)240, framerate=(fraction)30/1

Pipeline is PREROLLING ...

>>V4L_SINK: Actually buffer status:

        hardware buffer : 12

        software buffer : 0

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)320, height=(int)240, framerate=(fraction)30/1

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)320, height=(int)240, framerate=(fraction)30/1

/GstPipeline:pipeline0/MFW_GST_V4LSINK_INFO_T:mfw_gst_v4lsink_info_t0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)320, height=(int)240, framerate=(fraction)30/1

Pipeline is PREROLLED ...

Setting pipeline to PLAYING ...

full screen size:800x480

[V4L Update Display]: left=0, top=0, width=800, height=480

New clock: GstSystemClock

Not working (it runs with no error, but nothing is displayed on screen):

root@flexpad /$ gst-launch-0.10 -v videotestsrc ! video/x-raw-yuv,format=\(fourc

c\)YUY2 ! mfw_v4lsink &

root@flexpad /$ MFW_GST_V4LSINK_PLUGIN 1.9.6 build on Jan 27 2012 11:18:05.

Setting pipeline to PAUSED ...

/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)YUY2, width=(int)320, height=(int)240, framerate=(fraction)30/1

Pipeline is PREROLLING ...

>>V4L_SINK: Actually buffer status:

        hardware buffer : 12

        software buffer : 0

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)YUY2, width=(int)320, height=(int)240, framerate=(fraction)30/1

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)YUY2, width=(int)320, height=(int)240, framerate=(fraction)30/1

/GstPipeline:pipeline0/MFW_GST_V4LSINK_INFO_T:mfw_gst_v4lsink_info_t0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)YUY2, width=(int)320, height=(int)240, framerate=(fraction)30/1

Pipeline is PREROLLED ...

Setting pipeline to PLAYING ...

full screen size:800x480

[V4L Update Display]: left=0, top=0, width=800, height=480

New clock: GstSystemClock

mfw_v4lsink shows YUV (I420) but not YUV (YUY2), although inspecting the caps says it should support it:

root@flexpad /$ gst-inspect mfw_v4l_sink

No such element or plugin 'mfw_v4l_sink'

root@flexpad /$ gst-inspect | grep mfw 

mfw_aacdecoder:  mfw_aacdecoder: Freescale AAC Decoder Plugin

mfw_mp3decoder:  mfw_mp3decoder: freescale mp3 decoder

mfw_h264decoder:  mfw_h264decoder: Freescale    H264 decoder

mfw_mpeg4aspdecoder:  mfw_mpeg4aspdecoder: Freescale MPEG4 Decoder

mfw_v4lsink:  mfw_v4lsink: Freescale: V4L Sink

root@flexpad /$ gst-inspect mfw_v4lsink

MFW_GST_V4LSINK_PLUGIN 1.9.6 build on Jan 27 2012 11:18:05.

Factory Details:

  Long name:    Freescale: V4L Sink

  Class:        Sink/Video

  Description:  Video rendering device plugin used to display YUV/RGB data with the support to input cropping

  Author(s):    Multimedia Team <shmmmw@freescale.com>

  Rank:         unknown (258)

Plugin Details:

  Name:                 mfw_v4lsink

  Description:          Video display plugin based on V4L2

  Filename:             /usr/lib/gstreamer-0.10/libmfw_gst_v4lsink.so

  Version:              1.9.6

  License:              unknown

  Source module:        gst-fsl-plugin

  Binary package:       Gstreamer Multimedia Plugins (Freescale)

  Origin URL:           http://www.freescale.com

GObject

+----GstObject

       +----GstElement

             +----GstBaseSink

                   +----GstVideoSink

                         +----MFW_GST_V4LSINK_INFO_T

Pad Templates:

  SINK template: 'sink'

    Availability: Always

    Capabilities:

      video/x-raw-yuv

                 format: I420

                  width: [ 1, 2147483647 ]

                 height: [ 1, 2147483647 ]

              framerate: [ 0/1, 2147483647/1 ]

      video/x-raw-yuv

                 format: YV12

                  width: [ 1, 2147483647 ]

                 height: [ 1, 2147483647 ]

              framerate: [ 0/1, 2147483647/1 ]

      video/x-raw-yuv

                 format: YUYV

                  width: [ 1, 2147483647 ]

                 height: [ 1, 2147483647 ]

              framerate: [ 0/1, 2147483647/1 ]

      video/x-raw-yuv

                 format: YUY2

                  width: [ 1, 2147483647 ]

                 height: [ 1, 2147483647 ]

              framerate: [ 0/1, 2147483647/1 ]

      video/x-raw-yuv

                 format: NV12

                  width: [ 1, 2147483647 ]

                 height: [ 1, 2147483647 ]

              framerate: [ 0/1, 2147483647/1 ]

      video/x-raw-yuv

                 format: 422P

                  width: [ 1, 2147483647 ]

                 height: [ 1, 2147483647 ]

              framerate: [ 0/1, 2147483647/1 ]

      video/x-raw-rgb

                    bpp: [ 1, 32 ]

                  depth: [ 1, 32 ]

Element Flags:

  no flags set

Element Implementation:

  Has change_state() function: mfw_gst_v4lsink_change_state

  Has custom save_thyself() function: gst_element_save_thyself

  Has custom restore_thyself() function: gst_element_restore_thyself

Element has no clocking capabilities.

Element has no indexing capabilities.

Element has no URI handling capabilities.

Pads:

  SINK: 'sink'

    Implementation:

      Has chainfunc(): gst_base_sink_chain

      Has custom eventfunc(): gst_base_sink_event

      Has bufferallocfunc(): gst_base_sink_pad_buffer_alloc

    Pad Template: 'sink'

Element Properties:

  name                : The name of the object

                        flags: readable, writable

                        String. Default: null Current: "mfw_gst_v4lsink_info_t0"

  preroll-queue-len   : Number of buffers to queue during preroll

                        flags: readable, writable

                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 Current: 0

  sync                : Sync on the clock

                        flags: readable, writable

                        Boolean. Default: true Current: true

  max-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)

                        flags: readable, writable

                        Integer64. Range: -1 - 9223372036854775807 Default: -1 Current: 20000000

  qos                 : Generate Quality-of-Service events upstream

                        flags: readable, writable

                        Boolean. Default: false Current: true

  async               : Go asynchronously to PAUSED

                        flags: readable, writable

                        Boolean. Default: true Current: true

  ts-offset           : Timestamp offset in nanoseconds

                        flags: readable, writable

                        Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0 Current: 0

  last-buffer         : The last buffer received in the sink

                        flags: readable

                        MiniObject of type "GstBuffer"

  blocksize           : Size in bytes to pull per buffer (0 = default)

                        flags: readable, writable

                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096 Current: 4096

  render-delay        : Additional render delay of the sink in nanoseconds

                        flags: readable, writable

                        Unsigned Integer64. Range: 0 - -1 Default: 0 Current: 0

  show-preroll-frame  : Whether to render video frames during preroll

                        flags: readable, writable

                        Boolean. Default: true Current: true

  disp-width          : gets the width of the image to be displayed

                        flags: readable, writable

                        Integer. Range: 0 - 2147483647 Default: 0 Current: 0

  disp-height         : gets the height of the image to be displayed

                        flags: readable, writable

                        Integer. Range: 0 - 2147483647 Default: 0 Current: 0

  axis-top            : gets the top co-ordinate of the origin of display

                        flags: readable, writable

                        Integer. Range: 0 - 2147483647 Default: 0 Current: 0

  axis-left           : gets the left co-ordinate of the origin of display

                        flags: readable, writable

                        Integer. Range: 0 - 2147483647 Default: 0 Current: 0

  crop-left-by-pixel  : set the input image cropping in the left (width)

                        flags: readable, writable

                        Integer. Range: 0 - 2147483647 Default: 0 Current: 0

  crop-right-by-pixel : set the input image cropping in the right (width)

                        flags: readable, writable

                        Integer. Range: 0 - 2147483647 Default: 0 Current: 0

  crop-top-by-pixel   : set the input image cropping in the top (height)

                        flags: readable, writable

                        Integer. Range: 0 - 2147483647 Default: 0 Current: 0

  crop-bottom-by-pixel: set the input image cropping in the bottom (height)

                        flags: readable, writable

                        Integer. Range: 0 - 2147483647 Default: 0 Current: 0

  dump-location       : Location of the file to write cropped video YUV stream.Enable it will output image to file instead of V4L device

                        flags: readable, writable

                        String. Default: null Current: null

  setpara             : set parameter of V4L2, 1: Set V4L 2: Set Color

                        flags: readable, writable

                        Integer. Range: 0 - 3 Default: 0 Current: 0

  force-aspect-ratio  : Force Aspect Ratio

                        flags: readable, writable

                        Boolean. Default: false Current: false

>>V4L finalize: class finalized.

root@flexpad /$

Is this a general issue, or am I missing something/doing it wrong?

ps. I could use a ffmpegcolorspace element, but that uses alot of cpu time, so I would rather avoid it, especially when mfw_v4lsink says it supports YUY2!

Regards

Mark

[senykthomas]

If this is X11 based, skip my answer as I've nothing to say about X11 :smileywink:

If it's directly to framebuffer then you need to use gst-plugins-gl (from freescale!) and replace mfw_v4lsink with glimagesink.

glimagesink opens the framebuffer for EGL/OpenGL, maps the frames to textures and renders the textures with opengl directly into the framebuffer.

This is working for me on wandsolo and nitrogen6x.

[markurup]

It's not X11, but doesn't opengl require some hardware support?

Ofcourse I forgot to specify what hardware i'm on. It's the i.MX287 (Ka-Ro electronics GmbH: TX28 ARM9 i.MX28 SODIMM Modul mit Freescale i.MX287 CPU).

It as no hardware acceleration at all, but it does have a Pixel Pipeline (PXP):

http://cache.freescale.com/files/32bit/doc/brochure/FLYRIMXPRDCMPR.pdf

Thanks anyway.

[markurup]

While looking at why the sink can't accept YUY2, I'm also looking at why my source can't supply I420.

If I connect my source (a usb camera) to my host computer (x86_64 - arch linux) I CAN get the image as I420:

[urup@c4a012 ~]$ gst-launch-0.10 -v v4l2src ! video/x-raw-yuv,width=640,height=480,format=\(fourcc\)YUY2 ! ffmpegcolorspace ! ximagesink

Setting pipeline to PAUSED ...

/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)YUY2, width=(int)640, height=(int)480, interlaced=(boolean)false, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)YUY2, width=(int)640, height=(int)480, interlaced=(boolean)false, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)YUY2, width=(int)640, height=(int)480, interlaced=(boolean)false, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1

/GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:src: caps = video/x-raw-rgb, bpp=(int)32, depth=(int)24, endianness=(int)4321, red_mask=(int)65280, green_mask=(int)16711680, blue_mask=(int)-16777216, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlaced=(boolean)false

/GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)YUY2, width=(int)640, height=(int)480, interlaced=(boolean)false, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1

/GstPipeline:pipeline0/GstXImageSink:ximagesink0.GstPad:sink: caps = video/x-raw-rgb, bpp=(int)32, depth=(int)24, endianness=(int)4321, red_mask=(int)65280, green_mask=(int)16711680, blue_mask=(int)-16777216, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlaced=(boolean)false

^CCaught interrupt -- handling interrupt.

Interrupt: Stopping pipeline ...

Execution ended after 4256866749 ns.

Setting pipeline to PAUSED ...

Setting pipeline to READY ...

/GstPipeline:pipeline0/GstXImageSink:ximagesink0.GstPad:sink: caps = NULL

/GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:src: caps = NULL

/GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:sink: caps = NULL

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = NULL

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = NULL

/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = NULL

Setting pipeline to NULL ...

Freeing pipeline ...

[urup@c4a012 ~]$

While doing the same to the i.MX28 gives me:

root@flexpad /$ gst-launch-0.10 v4l2src -v device=/dev/video1 ! video/x-raw-yuv,width=640,height=480,format=\(fourcc\)I420 ! fakesink &

root@flexpad /$ Setting pipeline to PAUSED ...

ERROR: Pipeline doesn't want to pause.

ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Could not negotiate format

Additional debug info:

gstbasesrc.c(2584): gst_base_src_start (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:

Check your filtered caps, if any

Setting pipeline to NULL ...

Freeing pipeline ...

root@flexpad /$

Any clues as to what the difference is (somewhere in the v4l2 layer maybe)?

[senykthomas]

Sorry, than I'm out. I've no experience with the i.MX28.

But in general you need someone to take the frame and render/blit/display it 'on-screen'.

For i.MX6 this is not mfw_v4lsink. This is either X11 (taking the output/window from mfw_v4lsink) or the glimagesink (being at the end of the gst pipeline)

[markurup]

After reading a bit more in another manual (i.MX28 EVK Linux Reference Manual), it says that the PXP only accepts RGB555, RGB565, RGB24, YUV420 (planar), and YUV422P (planar) input formats, so I suspect that the mfw_v4lsink element simply lies in it capabilities?

Am I correct in this assumption?