i.MX53 gstreamer on framebuffer?

[sidebranch]

Are there examples on how to use the gstreamer codec plugins on the i.MX53 without X Windows running on the target?

Has someone played decoded video to the framebuffer?

Thanks,

Leon.

[sidebranch]

Yes it seems you are right that _MX51 must be defined. However, the configure user message suggests otherwise:

configure.ac:251:  AC_MSG_ERROR(No target platform specified! Use ./configure PLATFORM=(MX28/MX233/MX25/MX27/MX31/MX35/MX37/MX51/MX53/MX50/MX5X/MX6) to retry)

If specifying PLATFORM=MX53 this will define _MX53 as a preprocessor symbol during compilation. However, there is no check for _MX53, only for MX51 and smaller platforms.

So I think MX53/MX50/MX5X/MX6 will fail (and possible are Freescale internal new developments).

Thanks Kan HU, this fixed my problem.

[KanHU]

You maybe simply comment these sentences to see what happened. 

maybe the some macro is not defined, it should defined _MX51

[sidebranch]

If I disable this piece of code, the decoding suddenly works:

if (v4l_info->buffers_required > max_frames) {
            v4l_info->buffers_required = max_frames;
        }

I wonder why I need this change in my build, as nobody else seems to have this problem.

Regards,

Leon.

[sidebranch]

I have continued my analysis why no video is played out from my build:

In the "gst-fsl-plugin-2.0.3.tar.gz" package, the source file "mfw_gst_v4lsink.c" and the function "mfw_gst_v4lsink_buffer_alloc()":

        max_frames = MAX_V4L_ALLOW_SIZE_IN_BYTE / frame_buffer_size;

        if (v4l_info->buffers_required > max_frames) {
            v4l_info->buffers_required = max_frames;
        }

resulted in v4l_info->buffers_required = 2.

which resulted in mfw_gst_v4l.c: mfw_gst_v4l2_output_setup() to not allocate any buffers at all; this loop condition was never true:

    while (v4l_info->buffers_required >= MIN_BUFFER_NUM) {

    }

I have added extra Gstreamer debugging prints to demonstrate what happens. I do not have a clue yet why the Ubuntu provided image behaves differently. Unfortunately I am running into LTIB problems I have to solve, as that would provide a third basis.

0:00:01.058964251  1231    0xf39c0 DEBUG            mfw_v4lsink mfw_gst_v4lsink.c:1890:mfw_gst_v4lsink_buffer_alloc: Decoded Width = 1920, Decoded Height = 1088
0:00:01.059034001  1231    0xf39c0 WARN             mfw_v4lsink mfw_gst_v4lsink.c:1213:mfw_gst_v4lsink_set_format: set to nv12 mode
0:00:01.059086376  1231    0xf39c0 DEBUG            mfw_v4lsink mfw_gst_v4lsink.c:1902:mfw_gst_v4lsink_buffer_alloc: Decoder maximal reserved 12 buffers.
0:00:01.059323751  1231    0xf39c0 INFO             mfw_v4lsink mfw_gst_v4lsink.c:1933:mfw_gst_v4lsink_buffer_alloc: MAX_V4L_ALLOW_SIZE_IN_BYTE / frame_buffer_size = max_frames (7340032 / 3133440 = 2)
0:00:01.059380376  1231    0xf39c0 DEBUG            mfw_v4lsink mfw_gst_v4lsink.c:1936:mfw_gst_v4lsink_buffer_alloc: Decoder maximal support 2 buffers.
0:00:01.059429626  1231    0xf39c0 DEBUG            mfw_v4lsink mfw_gst_v4lsink.c:1940:mfw_gst_v4lsink_buffer_alloc: v4l_info->buffers_required reduced to max_frames = 2
0:00:01.059703626  1231    0xf39c0 INFO             mfw_v4lsink mfw_gst_v4l.c:1007:mfw_gst_v4l2_set_fmt: mfw_gst_v4l2_set_fmt()
0:00:01.059797751  1231    0xf39c0 INFO             mfw_v4lsink mfw_gst_v4l.c:1137:mfw_gst_v4l2_set_fmt: v4l_info->buffers_required = 2
0:00:01.059887126  1231    0xf39c0 INFO             mfw_v4lsink mfw_gst_v4l.c:942:mfw_gst_v4l2_output_setup: v4l_info->buffers_required = 2
0:00:01.059941126  1231    0xf39c0 INFO             mfw_v4lsink mfw_gst_v4l.c:943:mfw_gst_v4l2_output_setup: MIN_BUFFER_NUM = 5
0:00:01.059990126  1231    0xf39c0 INFO             mfw_v4lsink mfw_gst_v4l.c:974:mfw_gst_v4l2_output_setup: v4l_info->swbuffer_max = 0
0:00:01.060038751  1231    0xf39c0 ERROR            mfw_v4lsink mfw_gst_v4l.c:1141:mfw_gst_v4l2_set_fmt: Error in mfw_gst_v4lsink_output_setup

[sidebranch]

I have included the log for that.

I think it goes wrong at this point, I just cannot figure out why yet. The full log is in the attachment.

0:00:03.544888354  1233    0xd2fc0 ERROR            mfw_v4lsink mfw_gst_v4l.c:1130:mfw_gst_v4l2_set_fmt: Error in mfw_gst_v4lsink_output_setup
>>V4L_SINK: Actually buffer status:
    hardware buffer : 0
    software buffer : 0
0:00:03.545144729  1233    0xd2fc0 ERROR            mfw_v4lsink mfw_gst_v4l_buffer.c:630:mfw_gst_v4l2_buffer_init: Failed to allocate buffer pool container
0:00:03.545243854  1233    0xd2fc0 WARN             mfw_v4lsink mfw_gst_v4l_buffer.c:400:mfw_gst_v4l2_new_buffer: Try new buffer failed, ret 2 No such file or directory queued 0
0:00:03.545417854  1233    0xd2fc0 DEBUG            mfw_v4lsink mfw_gst_v4l_buffer.c:404:mfw_gst_v4l2_new_buffer: Finally assign a sw buffer from queue, left:0.
0:00:03.575604104  1233    0xd2fc0 DEBUG            mfw_v4lsink mfw_gst_v4l_buffer.c:69:mfw_gst_v4lsink_buffer_finalize: Buffer 0 maybe dropped.

[KanHU]

could you try "export GST_DEBUG=mfw_vpudecoder:4,mfw_v4lsink:4,GST_CAPS:4", 

I need more information for analysis. 

[sidebranch]

No this is independent of Linaro. We are working on the Yocto Project and the OpenEmbedded Core.

This is mainly voluntary work by me in the community as I need this for a project, whilst Alonso Lazcano Adrian (B38018) is working to get Freescale teams to work on mainlining.

Freescale has already taken over my work for PowerPC mainlining in Yocto.

We can re-use the Linaro work. We mostly work on the build infrastructure for Yocto, not necessarely on kernel/user space fixes.

However I do want to verify that what I check in and bring up stream actually works.

[sidebranch]

Sorry the log was not complete. I've deleted the old log and attached the new log instead.

[KanHU]

really strange, like the vpu did not set good caps to v4lsink

btw, I'm wondering which team you are with for upstream to which project? linaro?  

[sidebranch]

Thank you for looking along. I build from the meta-freescale repository at GitHub, this is for OpenEmbedded.  I know this is not LTIB, but I am working with the Freescale team to bring this upstream. This should match the 11.09.01 source code release.

Attached is the log for

export GST_DEBUG=mfw_vpudecoder:4,GST_CAPS:4

time gst-launch filesrc location=/trailer_1080p.mov ! qtdemux ! mfw_vpudecoder ! mfw_v4lsink

[KanHU]

where you build from? 

can you post log with "export GST_DEBUG=mfw_vpudecoder:4,GST_CAPS:4"

[sidebranch]

My own build gives "hardware buffer : 0" and gives no codec video output to the screen.

The Ubuntu SD card gives "hardware buffer : 12" and works.

What can be the reason for this difference?

I have built and executed the alpha tool just to be sure it is no alpha/colorkey issue.

[KanHU]

v4l sink is default output to fb2

composite for fb0 and fb2 is controlled by alpha value or colorkey 

there's ioctl for alpha setting 

struct mxcfb_gbl_alpha g_alpha; 

fd = open("/dev/fb0", O_RDWR, 0);

g_alpha.alpha = 128;
g_alpha.enable = 1;
ioctl(fd, MXCFB_SET_GBL_ALPHA, done, &g_alpha);

the alpha is from 0 to 255.

the hardware buffer number is for internal debug only, you can ignore the information. 

[WolfgangW]

Sorry, I didn't find out where to set transparency either.

Trying with gst-launch, my output looks mostly similar:

root@freescale ~$ gst-launch filesrc location=/mnt/cdrom/hd_other_samsung_led_motion.m2ts ! ffdemux_mpegts ! mfw_vpudecoder ! mfw_v4lsink
MFW_GST_V4LSINK_PLUGIN 2.0.3 build on Nov 25 2011 23:46:26.
Setting pipeline to PAUSED ...
[INFO] Product Info: i.MX53
VPU Version: firmware 13.4.41; libvpu: 5.3.2
MFW_GST_VPU_DECODER_PLUGIN 2.0.3 build on Nov 25 2011 23:45:31.
Pipeline is PREROLLING ...
>>V4L_SINK: Actually buffer status:
hardware buffer : 11
software buffer : 0
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
full screen size:1920x1080
[V4L Update Display]: left=0, top=0, width=1920, height=1080
New clock: GstSystemClock
video4linux video16: Bypassing IC
mxc_ipu mxc_ipu: Channel already disabled 10
mxc_ipu mxc_ipu: Channel already uninitialized 10
mxc_ipu mxc_ipu: Channel already disabled 10
mxc_ipu mxc_ipu: Channel already uninitialized 10
VPU blocking: timeout.
VPU blocking: timeout.
...

I am not sure if the number of hardware buffers at this point is meaningful? And, in my case, there is the message " video4linux video16: Bypassing IC" - what is IC? Might be image combiner...?

BTW, this is a full HD video you're playing... Did you see and follow this post?

[sidebranch]

Thank you. To answer your last question first: Yes, this is exactly what I mean with framebuffer playout. I was unsure if the V4L2 output used the fb device.

I have tested the following on the supplied Ubuntu SD image:

time gst-launch filesrc location=/trailer_1080p.mov ! qtdemux ! mfw_vpudecoder ! mfw_v4lsink

I see the video play correctly in the background, even when I kill the X Windows server at the same time. This is OK.

In my self-built image, I think I have an issue with the transparency, but opposite to your situation, in my case the framebuffer remains visible, but the video is invisible.

Where can this transparency be set?

MFW_GST_V4LSINK_PLUGIN  build on Dec 14 2011 13:56:30.
Setting pipeline to PAUSED ...
[INFO]    Product Info: i.MX53
VPU Version: firmware 13.4.41; libvpu: 5.3.2
MFW_GST_VPU_DECODER_PLUGIN  build on Dec 14 2011 13:56:16.
Pipeline is PREROLLING ...
>>V4L_SINK: Actually buffer status:
    hardware buffer : 0
    software buffer : 0
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
full screen size:1024x768
[V4L Update Display]: left=0, top=0, width=1024, height=768
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 32461977000 ns.
Setting pipeline to PAUSED ...
total time:0:00:32.480843375 ,Render fps:0
Setting pipeline to READY ...
Setting pipeline to NULL ...
Total rendered:0

>>VPU_DEC: State: Ready to Null
Freeing pipeline ...
[--->FINALIZE vpu_dec

[WolfgangW]

Short update:

Playback on raw framebuffer (actually: overlay via v4l) works with gst-launch as well as gplay in the same way.

With my self-compiled filesystem image, the overlay is also used without transparency, so the framebuffer contents is not visible while playing video - I remember this was different with the original Freescale image.

However, on second reading I am not sure if this is what you mean by "played decoded video to the framebuffer"?

[WolfgangW]

Hi Leon,

I only tried with gplay lately, but this one works flawlessly without any X running. Have to check with gst-launch tomorrow, but AFAIR it also worked without X. Only thing is that I did not yet figure out how to set up overlay correctly (blinking cursor of the framebuffer visible on the video...)

Best regards,

Wolfgang