Variable framerate encoder (Night vision)

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Variable framerate encoder (Night vision)

Jump to solution
1,567 Views
janiscoders
Contributor III

Hi, we have a project using IMX53 with ov5640 camera, where night vision must be enabled. Everything works fine, except when camera chip goes into night mode. The set-up is like this - ov5640 driver is modified to enable night mode (video becomes black n' white and when it is really dark, then camera chip drops frame rate to around 8-10 fps). IMPORTANT - I can't control the frame rate in driver, because that is determined by the Omnivision chip internally.

The problem is like this - if it is light enough then camera is returning video with 30 fps and I can record it with this command:

gst-launch mfw_v4lsrc capture-width=640 capture-height=480 capture-mode=0  num-buffers=150 ! queue ! mfw_vpuencoder codec-type=2 bitrate=0 ! queue  ! mux. alsasrc num-buffers=150 ! mfw_mp3encoder bitrate=64 ! mux. avimux name=mux  ! filesink location=currentlyRecording.mp4

BUT if it becomes dark, then camera chip automatically changes framerate to around 8-10 fps and video then is encoded in such a way that everything moves 3 times faster. There is no way to provide such a frame rate for encoder. Can someone please provide a solution to overcome this.

If camera was producing 8-10 fps all the time then I can encode it successfully by changing frame rate using videorate plugin, BUT the problem is that frame rate can change depending on brightness while video is being recorded.


NOTE. When I am live streaming using this command:


gst-launch -v mfw_v4lsrc fps-n=30 capture-width=640 capture-height=480 capture-mode=1  ! queue ! mfw_vpuencoder codec-type=0 ! rtpmp4vpay ! udpsink host=192.168.0.102 port=5000  sync=true

, then viewing this video on computer looks fine even when frame rate is changing dynamically. How does rtpm does this? Could it be possible to do the same when recording into a file?

Thank you




Labels (3)
0 Kudos
1 Solution
985 Views
janiscoders
Contributor III

We solved this problem by ourselves - the problem was to change raw colorspace, so that videorate plugin can process this data and adjust framerate to 30 fps. This can be done using mfw_ipucsc, which is hardware accelerated or ffmpegcolorspace, which isn't.

The command is this:

gst-launch -e mfw_v4lsrc capture-width=640 capture-height=480 capture-mode=0 ! queue ! mfw_ipucsc ! 'video/x-raw-yuv,format=(fourcc)I420' ! videorate ! 'video/x-raw-yuv,framerate=(fraction)30/1'  ! mfw_vpuencoder codec-type=2 bitrate=0 ! queue ! mux. alsasrc ! mfw_mp3encoder bitrate=64 ! queue ! mux. matroskamux name=mux ! filesink location=testAudio_with_e.mkv


Things to note - we do not provide num-buffers, so this video must be stopped by using CTRL-C. Otherwise the audio wasn't in sync with video.

View solution in original post

0 Kudos
1 Reply
986 Views
janiscoders
Contributor III

We solved this problem by ourselves - the problem was to change raw colorspace, so that videorate plugin can process this data and adjust framerate to 30 fps. This can be done using mfw_ipucsc, which is hardware accelerated or ffmpegcolorspace, which isn't.

The command is this:

gst-launch -e mfw_v4lsrc capture-width=640 capture-height=480 capture-mode=0 ! queue ! mfw_ipucsc ! 'video/x-raw-yuv,format=(fourcc)I420' ! videorate ! 'video/x-raw-yuv,framerate=(fraction)30/1'  ! mfw_vpuencoder codec-type=2 bitrate=0 ! queue ! mux. alsasrc ! mfw_mp3encoder bitrate=64 ! queue ! mux. matroskamux name=mux ! filesink location=testAudio_with_e.mkv


Things to note - we do not provide num-buffers, so this video must be stopped by using CTRL-C. Otherwise the audio wasn't in sync with video.

0 Kudos