topic Re: Frame grab from mfw_isink in i.MX Processors

Frame grab from mfw_isink

TalleyHo — Tue, 01 Nov 2011 16:07:46 GMT

I'm looking to capture a snapshot of a running video. My sink is mfw_isink. When requested, I grab the last-buffer from the sink and store it to a file.

So....

gst-launch playbin2 uri=uri video-sink=mfw_isink

From code:

g_object_get(G_OBJECT(m_vsink), "last-buffer", &buffer, NULL);

The following caps come from the buffer:

Caps video/x-raw-yuv, format=(fourcc)NV12, width=(int)720, height=(int)480, width_align=(int)64, height_align=(int)64, framerate=(fraction)30000/1001, pixel-aspect-ratio=(fraction)4/3, crop-top-by-pixel=(int)0, crop-left-by-pixel=(int)0, crop-right-by-pixel=(int)48, crop-bottom-by-pixel=(int)32, num-buffers-required=(int)10, field=(int)0

Last buffer size: 589824

Last buffer timestamp: 0:00:05.068854819

My confusion and concerns stem from the num-buffers-required field. When I try to use the IPU unit test to convert, the resulting file doesn't appear to be a full frame. The buffer size also doesn't seem to match up w/ what I expected size wise. I can tell what was supposed to be on screen, but it certainly isn't exactly what was on screen. It also doesn't look like it is only 1/10th of the screen either. What I find interesting, is that the actual video on screen is not equal to the caps provided.

When running this on the PC using Xvimagesink, the last-buffer field provided a full frame. Shouldn't this be the case here? If not, how would I go about getting a full screen grab of a running video?

Mx53/BSP 11.05

Re: Frame grab from mfw_isink

KanHU — Wed, 02 Nov 2011 14:03:56 GMT

one gstbuffer will contain whole frame including crop area.

the buffersize is calculated for (width+crop-left-by-pixel+crop-right-by-pixel)*(height+crop-top-by-pixel+crop-bottom-by-pixel)*1.5 for 12 bits per pixel of NV12 format.

If you need grab the snapshot, you need also eliminate the crop area

Re: Frame grab from mfw_isink

TalleyHo — Sat, 03 Dec 2011 00:52:51 GMT

Hi Kan,

Thanks for the suggestion. The cropping explains the strangeness in gimp. However, I get color banding on the bottom/right sides (maybe the cropped area, not sure) and the images are in black and white.

I'm using imagemagic to convert: convert -size 768x512 -sampling-factor 2x2 -depth 8 /nfsshare/screenshot.yuv (displayed image is 720x480, with 48/32 cropping areas)

What else is necessary? The format changes NV12/I420 depending on what source is decoding, but that doesn't seem to have any bearing on the resulting image. Other than the cropping area, doing the exact same thing w/ xvimagesink produces a nice snapshot.

Re: Frame grab from mfw_isink

KanHU — Tue, 06 Dec 2011 14:10:26 GMT

you mean the image grabbed is back and white? it looks like you get the wrong u,v data.

the I420 and NV12 are all 4:2:0 YUV data. the difference is that I420 is U, V planar and NV12 is UV interleave means like UVUV.

take I420 as example, the u, v area also has cropped field which crop-left/right, crop-top/bottom is half of Y.

I'm not sure imagemagic can process NV12 format, if you use mfw_vpudecoder before mfw_isink , you can set fmt property of mfw_vpudecoder for switch vpu outformat from NV12 to I420.

Re: Frame grab from mfw_isink

TalleyHo — Tue, 06 Dec 2011 15:05:37 GMT

I'll put a caps in place to force the format. I've been using playbin2. I haven't found a method to how it picks the format.

The resulting image is black and white. However, there are 'green' borders around the right/bottom side. Even the ipu unit test won't decode it properly. Or, at least, I haven't found the correct parameters to give it to decode it correctly. I ultimately need it ARGB/32 format.

Re: Frame grab from mfw_isink

KanHU — Sat, 17 Dec 2011 15:41:15 GMT

you can set fmt property of mfw_vpudecoder, 0 for NV12, 1 for I420. if you need other format, you need other csc plugin, like mfw_ipucsc.