Frame grab from mfw_isink

[TalleyHo]

I'm looking to capture a snapshot of a running video.  My sink is mfw_isink.  When requested, I grab the last-buffer from the sink and store it to a file.  

So.... 

gst-launch playbin2 uri=uri video-sink=mfw_isink

From code:

g_object_get(G_OBJECT(m_vsink), "last-buffer", &buffer, NULL);

The following caps come from the buffer:

Caps video/x-raw-yuv, format=(fourcc)NV12, width=(int)720, height=(int)480, width_align=(int)64, height_align=(int)64, framerate=(fraction)30000/1001, pixel-aspect-ratio=(fraction)4/3, crop-top-by-pixel=(int)0, crop-left-by-pixel=(int)0, crop-right-by-pixel=(int)48, crop-bottom-by-pixel=(int)32, num-buffers-required=(int)10, field=(int)0

Last buffer size: 589824

Last buffer timestamp: 0:00:05.068854819

My confusion and concerns stem from the num-buffers-required field.  When I try to use the IPU unit test to convert, the resulting file doesn't appear to be a full frame.  The buffer size also doesn't seem to match up w/ what I expected size wise.  I can tell what was supposed to be on screen, but it certainly isn't exactly what was on screen.  It also doesn't look like it is only 1/10th of the screen either.  What I find interesting, is that the actual video on screen is not equal to the caps provided.

When running this on the PC using Xvimagesink, the last-buffer field provided a full frame.  Shouldn't this be the case here?  If not, how would I go about getting a full screen grab of a running video?

Mx53/BSP 11.05

[KanHU]

you can set fmt property of mfw_vpudecoder, 0 for NV12, 1 for I420. if you need other format, you need other csc plugin, like mfw_ipucsc.

[TalleyHo]

I'll put a caps in place to force the format.  I've been using playbin2.  I haven't found a method to how it picks the format.

The resulting image is black and white.  However, there are 'green' borders around the right/bottom side.  Even the ipu unit test won't decode it properly.  Or, at least, I haven't found the correct parameters to give it to decode it correctly.  I ultimately need it ARGB/32 format.

[KanHU]

you mean the image grabbed is back and white? it looks like you get the wrong u,v data.

the I420 and NV12 are all 4:2:0 YUV data. the difference is that I420 is U, V planar and NV12 is UV interleave means like UVUV. 

take I420 as example, the u, v area also has cropped field which crop-left/right, crop-top/bottom is half of Y.

I'm not sure imagemagic can process NV12 format, if you use mfw_vpudecoder before mfw_isink , you can set fmt property of mfw_vpudecoder for switch vpu outformat from NV12 to I420.

[TalleyHo]

Hi Kan,

  Thanks for the suggestion.  The cropping explains the strangeness in gimp.  However, I get color banding on the bottom/right sides (maybe the cropped area, not sure) and the images are in black and white.  

I'm using imagemagic to convert: convert -size 768x512 -sampling-factor 2x2 -depth 8 /nfsshare/screenshot.yuv (displayed image is 720x480, with 48/32 cropping areas)

What else is necessary?  The format changes NV12/I420 depending on what source is decoding, but that doesn't seem to have any bearing on the resulting image.  Other than the cropping area, doing the exact same thing w/ xvimagesink produces a nice snapshot.

[KanHU]

one gstbuffer will contain whole frame including crop area.

the buffersize is calculated for (width+crop-left-by-pixel+crop-right-by-pixel)*(height+crop-top-by-pixel+crop-bottom-by-pixel)*1.5 for 12 bits per pixel of NV12 format.

If you need grab the snapshot, you need also eliminate  the crop area