Efficient Streaming of image sensor using gray formats on i.MX 8M Mini

[mmarohnic]

Hello!

I have implemented a mono sensor on  i.MX 8M Mini platform and can stream via gstreamer like this. 

gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=GRAY8,width=1920,height=1080" ! videoconvert ! fpsdisplaysink text-overlay=true video-sink=autovideosink sync=false

However, I can only get 3 fps. Is there any method available to speed this up (hardware acceleration, etc.) 

With v4l2-ctl I am able to stream much larger values.
I would also need this kind of acceleration for GRAY16_BE format.

[joanxie]

how about using command like 

"v4l2-ctl -d /dev/video0 --set-fmt-video=width=xxx,height=xxx,pixelformat=xxx --stream-mmap"

xxx is for specific value you need to set

[mmarohnic]

As mentioned in the description the problem is to get the visualization with decent fps via gstreamer and not by using v4l2-ctl.

[joanxie]

why I ask you to change to the v4l2, because gstreamer didn't show the real performance, as I known, fpsdisplaysink  text-overlay=true would cause frame rate drop, otherwise you can test by " fpsdisplaysink "sync=false" "text-overlay=false"", you should get higher frame rate, this plugin is opensource, if you need test the frame rate, I suggest that you can use v4l2, if you insist on using gstreamer, try to change other sink plugin

[mmarohnic]

I can stream with:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! "video/x-raw,format=GRAY8,width=1920,height=1080,framerate=90/1" ! fpsdisplaysink text-overlay=false vi
deo-sink=fakesink sync=false signal-fps-measurements=true

and get a full frame rate (the same as with v4l2-ctl). With waylanandsink and videotestscr

one can stream max to 60fps like this:

gst-launch-1.0 -v videotestsrc ! video/x-raw,format=YUY2,framerate=70/1,width=1280,height=720 ! queue ! fpsdisplaysink sync=false video-sink="waylandsink sync=false" signal-fps-measurements=true text-overlay=false fps-update-interval=10000

The problem seems to be that videoconvert is taking too much time in 

gst-launch-1.0 -v v4l2src device=/dev/video0 ! "video/x-raw,format=GRAY8,width=1920,height=1080,framerate=90/1" ! videoconvert ! fpsdisplaysink text-overlay=false video-sink="waylandsink sync=false" sync=false signal-fps-measurements=true

Could videoconvert be replaced with https://github.com/Freescale/gstreamer-imx/blob/master/README.md imxvideoconvert (or some other plugin) that uses gpu accelration to achive max fps with visualization? Is there a source code available for implementing this kind of operation using g2d api (https://github.com/nxp-imx/g2d-samples)? (or some other way to achive this?)

Thanks!

[joanxie]

you can test imxvideoconvert_g2d, the command like 

gst-launch-1.0 v4l2src device=/dev/videox ! video/x-raw,format=xx ,width=xx,height=xx ! queue ! imxvideoconvert_g2d ! waylandsink window-width=xx window-height=xx--no-position