Hello,
we would like to develop an equipment with GPU accelerated full-HD video H.264 blended with 0.5alpha still picture over it. No Android.
Questions:
a) Is this intended equipment possible to do with GPU acceleration?
b) Is this possible to do it with g2d?
c) Is v4l2 supported by NXP in future? How to resize the movie in 800x600 window? Please see 5 below.
Thank you,
FH
We use for testing:
1) i.MX6 Series SABRE for SmartDevices (MCIMX6DL-SDP) with LCD touch 1024x768
2) Linux imx6dlsabresd 4.1.15-2.1.0+g30278ab with compiled imx-gstreamer plugins
3) /sys/class/graphics/fb0: DISP3 BG - DI1 U:1024x768p-60 ldb 2-layer-fb-bg
/sys/class/graphics/fb1: DISP3 FG overlay 2-layer-fb-fg
/sys/class/graphics/fb2: DISP3 BG U:1920x1080p-60 hdmi 1-layer-fb
- (to fb0) movie: 1920x765@23.976FPS(M4V codec (M4V /M4A /mp42/isom))
- (to fb1) still picture: PNG image data, 1024 x 768, 8-bit/color RGBA, non-interlaced
- loc_alpha.alpha_in_pixel = 1;
loc_alpha.enable = 1;
fd = open("/dev/fb0", O_RDWR, 0);
ioctl(fd, MXCFB_SET_LOC_ALPHA, &loc_alpha);
ioctl(fd, MXCFB_WAIT_FOR_VSYNC, 0);
ioctl(fd, FBIOBLANK, FB_BLANK_UNBLANK);
4) no HW changes, no SW changes, in Yocto added compile imx-gstreamer plugins
5) expected: alpha blending, smooth video play, no CPU at 100%
observed: alpha blending is correct (picture and movie are mixed)
background video play is visible, good colours and size, but not smooth play
(=like movie on a powerless hardware without GPU acceleration)
CPU is on 11%
When we use v4l2:
"gst-launch-1.0 filesrc location=N_movie.m4v ! qtdemux ! queue ! vpudec ! imxv4l2sink device=/dev/video16"
the movie play smoothly, but there is unfortunately no way to resize it inside 800x600 window
6) echo 0 > /sys/class/graphics/fb1/blank
echo 0 > /sys/class/graphics/fb0/blank
set_alpha.bin
WINDOW="window-x-coord=100 window-y-coord=100 window-width=800 window-height=600"
gst-launch-1.0 filesrc location=2N_picture_05alpha.png ! pngdec ! imagefreeze ! queue ! imxg2dvideosink force-aspect-ratio=true framebuffer=/dev/fb1
gst-launch-1.0 filesrc location=2N_movie.m4v typefind=true ! qtdemux ! queue ! vpudec ! imxg2dvideosink framebuffer=/dev/fb0 $WINDOW
7) always not smoothly video overplay
8) LCD touch 1024x768, like here
Hi Frantisek
one can test video capabilities with Demo Image
https://www.nxp.com/webapp/Download?colCode=L4.1.15_2.0.0_MX6QDLSOLO&appType=license&location=null
follow sect.7 Multimedia, sect.7.3.15 Video conversion attached Linux Guide
Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------
Hi Igor,
thank you for the reply.
I realized (in L4.1.15_2.1.0) that if I use decodebin!videoconvert instead of qtdemux!vpudec, everythink works fine
as expected, accelerated (smooth background video play, alpha blending image on top of it, resize, CPU at 20%):
gst-launch-1.0 filesrc location=2N_movie.m4v typefind=true ! qtdemux ! queue ! vpudec ! imxg2dvideosink framebuffer=/dev/fb0 $WINDOW
gst-launch-1.0 filesrc location=2N_movie.m4v typefind=true ! decodebin ! queue ! videoconvert ! imxg2dvideosink framebuffer=/dev/fb0 $WINDOW
So my previous a), b) and c) questions are solved now.
Can you please explain, what is here wrong with qtdemux!vpudec?
My next questions are:
Thank you,
and best regards,
FH
I think you may be mixing plugins from two different sources (which aren't compatible), the FSL gst-1.0 plugs are from here and include vpudec. There's also an open source implementation gstreamer-imx that includes imxg2dvideosink. The g2d api allow overlaying of multi buffers when blitting to the output surface however I'm not sure if this is implement by the gst-plugin (simple demo calling g2d api directly). Since gstreamer-imx source code is available then I guess worst case is that you will need to support it yourself.
Thank you, answers after hours of investigation:
a) yes, this equiptment is possible to do with GPU acceleration - we made as Linux native app:
b) yes, G2D is working perfectly (in mxc_vpu_test example we reused codes written for Android to be used without Android)
c) resize is done by G2D also
FH