camera on CSI

JoshuaParrish · ‎11-18-2011

Hello all,

I've recently been trying to set up an ov7725 on the csi 0 line.

I have it coming up and being recognized as a device.

However, when I try to get a picture out I get nothing.

I've used a couple different tests. but nothing gets a real response.

The output I have from /unit_tests/mxc_v4l2_overlay.out is

g_display_width=240, g_display_height =320

g_display_top=0,g_display_left=0

sensor chip is ov7725

sensor frame size is 640x480

sensor frame format is UYVY

frame_rate is 30

frame buffer width 1024, height 768, bytesperline 2048

Then about 3 seconds later I get a green pattern at the top of my screen.

My sensor by default puts out 60 frames per second (I have used i2c to control it down to 30 fps). It outputs in 640x480 in YUV format.

The driver I'm using is a clone of the ov3640 driver included with the BSP.

The big difference between the two drivers is that I have eliminated all the i2c set up data.

and replaced ov3640 with ov7725.

The main issue here is that I am not getting a picture. Could anyone offer suggestions?

daiane_angolini · ‎12-02-2011

great.

Please, place over here the link to new thread.

JoshuaParrish · ‎12-02-2011

I'll probably start on new thread. Since the original question was answered quite some time ago. Thanks again for your help.

daiane_angolini · ‎12-02-2011

I don´t have any new idea. Maybe it´s a vpu encoder bug.......

Let´s hope anyone else can help on this issue...

JoshuaParrish · ‎12-01-2011

So I finally had mfw_vpuencoder come back up.

Unfortunately, I only have preview giving a good image.

The image that comes out of mfw_vpuencoder for all available codecs is a green picture.

I've uploaded the movie.

JoshuaParrish · ‎11-30-2011

Daiane,

I'm currently running the Ubuntu shipped fs (An incredibly modified version of the one that shipped in the demo box).

I unloaded the debs into a directory and dpkg -i *.deb .

The std_avc one comes out with a crazy amount of color distortion. colors @ the top are correct; yet others are very wrong.

As far as another movie goes, I posted movies for both the std_mpeg and the H263 encoding; both videos come out with quite similar encoding artifacts (right side color effect).

JoshuaParrish · ‎11-30-2011

-rw-r--r-- 1 root root 38952 2011-09-28 01:32 libmfw_gst_aacdec.so

-rw-r--r-- 1 root root 38976 2011-09-28 01:32 libmfw_gst_aacplusdec.so

-rw-r--r-- 1 root root 34756 2011-09-28 01:32 libmfw_gst_ac3dec.so

-rw-r--r-- 1 root root 64216 2011-11-30 09:56 libmfw_gst_aiur_demux.so

-rw-r--r-- 1 root root 13984 2011-09-28 01:32 libmfw_gst_audio_pp.so

-rw-r--r-- 1 root root 18136 2011-09-28 01:32 libmfw_gst_deinterlace.so

-rw-r--r-- 1 root root 13980 2011-09-28 01:32 libmfw_gst_downmix.so

-rw-r--r-- 1 root root 34724 2011-09-28 01:32 libmfw_gst_h264dec.so

-rw-r--r-- 1 root root 22164 2011-09-28 01:32 libmfw_gst_ipu_csc.so

-rw-r--r-- 1 root root 30696 2011-09-28 01:32 libmfw_gst_isink.so

-rw-r--r-- 1 root root 34756 2011-09-28 01:32 libmfw_gst_mp3dec.so

-rw-r--r-- 1 root root 14032 2011-11-30 09:56 libmfw_gst_mp3enc.so

-rw-r--r-- 1 root root 22336 2011-09-28 01:32 libmfw_gst_mpeg2dec.so

-rw-r--r-- 1 root root 26472 2011-09-28 01:32 libmfw_gst_mpeg4aspdec.so

-rw-r--r-- 1 root root 59388 2011-09-28 01:32 libmfw_gst_v4lsink.so

-rw-r--r-- 1 root root 22200 2011-09-28 01:32 libmfw_gst_v4lsrc.so

-rw-r--r-- 1 root root 26560 2011-09-28 01:32 libmfw_gst_vorbisdec.so

-rw-r--r-- 1 root root 67880 2011-09-28 01:32 libmfw_gst_vpu_dec.so

-rw-r--r-- 1 root root 52320 2011-09-28 01:32 libmfw_gst_vpu_enc.so

-rw-r--r-- 1 root root 18216 2011-09-28 01:32 libmfw_gst_wma10dec.so

-rw-r--r-- 1 root root 22296 2011-09-28 01:32 libmfw_gst_wma8enc.so

-rw-r--r-- 1 root root 26508 2011-09-28 01:32 libmfw_gst_wmv9mpdec.so

-rw-r--r-- 1 root root 22336 2011-09-28 01:32 libmfw_gst_wmvdec.so

daiane_angolini · ‎11-30-2011

well... you should cp all tarball to /opt/freescale/pkg. then upgrade the version for all new packages into the respective spec file.

then it´s only ./ltib -c and let it rebuilt....

I´ve been thinking about this issue. If it´s a bug, it might be fixed on newest release. So, it would be nice give it a test.

Have you posted any other movie encoded with another codec and with good quality?

JoshuaParrish · ‎11-30-2011

gst-inspect | grep mfw
(gst-plugin-scanner:8593): GStreamer-WARNING **: Failed to load plugin '/usr/lib/gstreamer-0.10/libmfw_gst_ac3dec.so': lib_ac3_dec_arm11_elinux.so.2.8.0: cannot open shared object file: No such file or directory
(gst-plugin-scanner:8593): GStreamer-WARNING **: Failed to load plugin '/usr/lib/gstreamer-0.10/libmfw_gst_vpu_dec.so': libvpu.so.0: cannot open shared object file: No such file or directory
(gst-plugin-scanner:8593): GStreamer-WARNING **: Failed to load plugin '/usr/lib/gstreamer-0.10/libmfw_gst_vpu_enc.so': libvpu.so.0: cannot open shared object file: No such file or directory
(gst-plugin-scanner:8593): GStreamer-WARNING **: Failed to load plugin '/usr/lib/gstreamer-0.10/libmfw_gst_wmvdec.so': lib_WMV789_dec_arm11_elinux.so.0.1.3: cannot open shared object file: No such file or directory
(gst-plugin-scanner:8593): GStreamer-WARNING **: Failed to load plugin '/usr/lib/gstreamer-0.10/libmfw_gst_wma8enc.so': lib_wma_muxer_arm11_ELINUX.so: cannot open shared object file: No such file or directory
(gst-plugin-scanner:8593): GStreamer-WARNING **: Failed to load plugin '/usr/lib/gstreamer-0.10/libmfw_gst_downmix.so': lib_downmix_arm11_elinux.so.2.0.0: cannot open shared object file: No such file or directory
(gst-plugin-scanner:8593): GStreamer-WARNING **: Failed to load plugin '/usr/lib/gstreamer-0.10/libmfw_gst_wma10dec.so': lib_wma10_dec_arm12_elinux.so.3.4.0: cannot open shared object file: No such file or directory
(gst-plugin-scanner:8593): GStreamer-WARNING **: Failed to load plugin '/usr/lib/gstreamer-0.10/libmfw_gst_isink.so': libipu.so.0: cannot open shared object file: No such file or directory
(gst-plugin-scanner:8593): GStreamer-WARNING **: Failed to load plugin '/usr/lib/gstreamer-0.10/libmfw_gst_wmv9mpdec.so': lib_WMV9MP_dec_MP_arm11_elinux.so.2.2.4: cannot open shared object file: No such file or directory
(gst-plugin-scanner:8593): GStreamer-WARNING **: Failed to load plugin '/usr/lib/gstreamer-0.10/libmfw_gst_ipu_csc.so': libipu.so.0: cannot open shared object file: No such file or directory
(gst-plugin-scanner:8593): GStreamer-WARNING **: Failed to load plugin '/usr/lib/gstreamer-0.10/libgstopengl.so': libEGL.so.1: cannot open shared object file: No such file or directorymfw_deinterlacer: mfw_deinterlacer: Mfw De-interlacemfw_v4lsink: mfw_v4lsink: Freescale: V4L Sinkmfw_audio_pp: mfw_audio_pp: Freescale Audio Post-process Filtermfw_aacplusdecoder: mfw_aacplusdecoder: Freescale AAC Decoder Pluginmfw_mp3encoder: mfw_mp3encoder: freescale mp3 encodermfw_mp3decoder: mfw_mp3decoder: freescale mp3 decodermfw_h264decoder: mfw_h264decoder: Freescale H264 decodermfw_aacdecoder: mfw_aacdecoder: Freescale AAC Decoder Pluginmfw_mpeg2decoder: mfw_mpeg2decoder: Freescale MPEG2 Decodermfw_vorbisdecoder: mfw_vorbisdecoder: Freescale vorbis Decoder Pluginmfw_v4lsrc: mfw_v4lsrc: Freescale Video Source plug-inmfw_mpeg4aspdecoder: mfw_mpeg4aspdecoder: Freescale MPEG4 Decoder

JoshuaParrish · ‎11-30-2011

Daiane,

Did that, unfortunately I now have no ability to even run mfw_vpuencoder. that plugin has disappeared.

JoshuaParrish · ‎11-30-2011

Alright I will go with that one.

daiane_angolini · ‎11-30-2011

11.09 release notes tells imx53 is supported, at least for ubuntu....

JoshuaParrish · ‎11-30-2011

11.05 or 11.09? I heard 11.09 shouldn't be used on the imx53; only the imx6 series.

daiane_angolini · ‎11-30-2011

Are you using the latest mmcodec package?

Can you upgrade it?

I don´t know if it´s a bug. But let´s try the latest release....

JoshuaParrish · ‎11-29-2011

Forgot to attach the video.

JoshuaParrish · ‎11-29-2011

I captured this using

gst-launch -v --gst-debug=2 mfw_v4lsrc num-buffers=350 capture-width=640 capture-height=480 ! videorate ! 'video/x-raw-yuv,width=640,height=480,framerate=(fraction)30/1' ! queue ! mfw_vpuencoder codec-type=0 intra-qp=10 qp=10 ! queue ! mux. alsasrc num-buffers=300 typefind=1 ! audioconvert ! mfw_mp3encoder ! queue ! mux. avimux name=mux ! filesink location=test.mp4

[the output is at the bottom]

[the video output has been attached]

What's interesting is the side keeps bleeding into the rest of the scene. This is the best video I can get.

My efforts at trying v4l2src do not work. (Which is kind of a bummer)

ffmpegcolorspace does not work either. (I had really high hopes for that. )

I think this comes down to encoder error on the vpu. What do you think?

MFW_GST_V4LSRC_PLUGIN 1.9.7-1 build on Jan 28 2011 13:26:16.BLN_MAD-MMCODECS_MP3E_ARM_02.02.00_ARM12 build on Apr 19 2011 14:18:53.MFW_GST_MP3_ENCODER_PLUGIN 1.9.7-1 build on Jan 28 2011 13:29:54.Setting pipeline to PAUSED ...[INFO] Product Info: i.MX53VPU Version: firmware 13.4.24; libvpu: 5.1.4MFW_GST_VPU_ENCODER_PLUGIN 1.9.7-1 build on Jan 28 2011 13:24:19./GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: actual-buffer-time = 139319/GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: actual-latency-time = 34829/GstPipeline:pipeline0/GstAlsaSrc:alsasrc0.GstPad:src: caps = audio/x-raw-int, endianness=(int)1234, signed=(boolean)true, width=(int)32, depth=(int)32, rate=(int)44100, channels=(int)2/GstPipeline:pipeline0/MFWGstV4LSrc:mfwgstv4lsrc0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)NV12, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1Pipeline is live and does not need PREROLL ...Setting pipeline to PLAYING ...New clock: GstAudioSrcClock/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:src: caps = audio/x-raw-int, endianness=(int)1234, signed=(boolean)true, width=(int)16, depth=(int)16, rate=(int)44100, channels=(int)2/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:sink: caps = audio/x-raw-int, endianness=(int)1234, signed=(boolean)true, width=(int)32, depth=(int)32, rate=(int)44100, channels=(int)2/GstPipeline:pipeline0/MfwGstMp3EncInfo:mfwgstmp3encinfo0.GstPad:sink: caps = audio/x-raw-int, endianness=(int)1234, signed=(boolean)true, width=(int)16, depth=(int)16, rate=(int)44100, channels=(int)2/GstPipeline:pipeline0/MfwGstMp3EncInfo:mfwgstmp3encinfo0.GstPad:sink: caps = audio/x-raw-int, endianness=(int)1234, signed=(boolean)true, width=(int)16, depth=(int)16, rate=(int)44100, channels=(int)2/GstPipeline:pipeline0/MfwGstMp3EncInfo:mfwgstmp3encinfo0.GstPad:src: caps = audio/mpeg, mpegversion=(int)1, layer=(int)3, rate=(int)44100, channels=(int)2/GstPipeline:pipeline0/MfwGstMp3EncInfo:mfwgstmp3encinfo0.GstPad:src: caps = audio/mpeg, mpegversion=(int)1, layer=(int)3, rate=(int)44100, channels=(int)20:00:00.795767988 3172 0x192090 WARN videorate gstvideorate.c:587:gst_video_rate_event:<videorate0> Got segment but doesn't have GST_FORMAT_TIME value/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)NV12, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)NV12, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)NV12, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1/GstPipeline:pipeline0/GstQueue:queue2.GstPad:sink: caps = audio/mpeg, mpegversion=(int)1, layer=(int)3, rate=(int)44100, channels=(int)2/GstPipeline:pipeline0/GstQueue:queue2.GstPad:src: caps = audio/mpeg, mpegversion=(int)1, layer=(int)3, rate=(int)44100, channels=(int)20:00:00.801315128 3172 0x18b7c0 WARN avimux gstavimux.c:844:gst_avi_mux_audsink_set_caps:<mux> unparsed MPEG audio input (?), doing CBR muxing/GstPipeline:pipeline0/GstAviMux:mux.GstPad:audio_00: caps = audio/mpeg, mpegversion=(int)1, layer=(int)3, rate=(int)44100, channels=(int)2/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)NV12, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)NV12, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)NV12, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)NV12, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1/GstPipeline:pipeline0/MfwGstVPU_Enc:mfwgstvpu_enc0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)NV12, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1/GstPipeline:pipeline0/MfwGstVPU_Enc:mfwgstvpu_enc0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)NV12, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1/GstPipeline:pipeline0/MfwGstVPU_Enc:mfwgstvpu_enc0.GstPad:src: caps = video/mpeg, height=(int)480, width=(int)640, framerate=(fraction)30/1, mpegversion=(int)4, systemstream=(boolean)false/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = video/mpeg, height=(int)480, width=(int)640, framerate=(fraction)30/1, mpegversion=(int)4, systemstream=(boolean)false/GstPipeline:pipeline0/GstQueue:queue1.GstPad:src: caps = video/mpeg, height=(int)480, width=(int)640, framerate=(fraction)30/1, mpegversion=(int)4, systemstream=(boolean)false/GstPipeline:pipeline0/GstAviMux:mux.GstPad:video_00: caps = video/mpeg, height=(int)480, width=(int)640, framerate=(fraction)30/1, mpegversion=(int)4, systemstream=(boolean)falseGot EOS from element "pipeline0".Execution ended after 12647232153 ns.Setting pipeline to PAUSED ...Setting pipeline to READY .../GstPipeline:pipeline0/GstAviMux:mux.GstPad:audio_00: caps = NULL/GstPipeline:pipeline0/GstAviMux:mux.GstPad:video_00: caps = NULL/GstPipeline:pipeline0/GstQueue:queue2.GstPad:src: caps = NULL/GstPipeline:pipeline0/GstQueue:queue2.GstPad:sink: caps = NULL/GstPipeline:pipeline0/GstQueue:queue1.GstPad:src: caps = NULL/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = NULL/GstPipeline:pipeline0/MfwGstMp3EncInfo:mfwgstmp3encinfo0.GstPad:src: caps = NULL/GstPipeline:pipeline0/MfwGstMp3EncInfo:mfwgstmp3encinfo0.GstPad:sink: caps = NULL/GstPipeline:pipeline0/MfwGstVPU_Enc:mfwgstvpu_enc0.GstPad:src: caps = NULL/GstPipeline:pipeline0/MfwGstVPU_Enc:mfwgstvpu_enc0.GstPad:sink: caps = NULL/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:src: caps = NULL/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:sink: caps = NULL/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = NULL/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = NULL/GstPipeline:pipeline0/GstAlsaSrc:alsasrc0.GstPad:src: caps = NULL/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = NULL/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = NULL/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:src: caps = NULL/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:sink: caps = NULL/GstPipeline:pipeline0/MFWGstV4LSrc:mfwgstv4lsrc0.GstPad:src: caps = NULLSetting pipeline to NULL ...Freeing pipeline ...

JoshuaParrish · ‎11-29-2011

I'm glad your supervisor checks this. They should know that with out your input my project would be dead.

Ogg theora

Yes it is pretty horrible performance, but testing with a usb cam a few months ago I was down to 64% avg cpu time.

It's really a back up for mpeg4

H264 and MPEG4

I have tried the ffmpegcolorspace before on the earlier mfw_v4lsrc ! mfw_v4lsink problem with no luck.

Now, with this problem it will at least link through and the script will parse, however it doesn't seem to change anything.

Forgot to send this reply while I was testing.

The only place I can put in the ffmpegcolorspace command is between mfw_v4lsrc and the mfw_vpuencoder. This is good becuase the preview shows a good image everytime. But it seems as though somthing is happening in the vpu that is changing the colors. The odd part is not all the colors are off; some show up as the correct color.

gst-launch -v --gst-debug=2 mfw_v4lsrc num-buffers=50 typefind=1 capture-width=640 capture-height=480 preview=1 preview-width=640 preview-height=480 bg=1 ! queue ! ffmpegcolorspace ! mfw_vpuencoder codec-type=2 ! queue ! avimux name=mux ! filesink location=test.avi

daiane_angolini · ‎11-29-2011

Hi Joshua.

One nice thing about being public is that everyone can see it ;)

So, many thanks for your nice words, my supervisor will see it over here.

ogg theora/vorbis

Do you know mx53 has no hardware acceleration for encoding/decoding OGG theora/vorbis, don´t you? Have you tested its performance using only software?

H264 & MPEG4

A friend of mine just give me a good idea. Can you give ffmpegcolorspace a try?

JoshuaParrish · ‎11-29-2011

Daiane,

I wanted to say once again that appreciate your help. You have been a really big to me. If possible I would like to get contact information for your supervisor via a pm. You have been one of the only Freescale agents to help with my current problems, as my technical support ticket with Freescale has yet to be answered.

Anyway on to your questions:

unit tests work

gstreamer preview works

encoding h263(codec 1) works

encoding mpeg(codec 0) works

encoding avc (codec 2) does not work (at least not as it should, bad color).

Yes, I need MPEG4.

The goal is to get both an ogg theora/vorbis to cover Chrome and Firefox and a .mp4 playable in safari and IE the reason for this is to be native with HTML5. The only luck I've had with mp4 is using the mpeg part 10 AVC encoding and the mp3 encoder.

That is the way I would like to go. If using codec 0 works; I'll do that too; but, I need both video and audio.

With the codec 2 command I included the output in gstreamer.txt

I also included its video output as well.

I'm working in avi right now as I have had a lot of luck just seeing a video.

However, I'm willing to change to what ever I needed.

I also tested codec 0; it too works well. Unfortunately I do not know if that can be used for my purposes. I believe it is only used for plain video (no audio) I may be wrong about that.

I hope this makes my goal more apparent.

daiane_angolini · ‎11-29-2011

I only posted the code because I was about to reply your other comment.

Let me see if I understood.

unit_test works

gstreamer preview works (mfw_v4lsrc ! mfw_v4lsink someway)

encoding an H263 works

So, now you need to enconde MPEG4. Is that right?

Have you took the message from gst-launch during mp4 encoding?

I will take a deeper look on mfw_vpuencoder looking for any color configuration.

JoshuaParrish · ‎11-29-2011

Daiane,

I have to be able to make a video and audio mp4 format.

codec-type : selects the codec type for encoding flags: readable, writable Enum "MfwGstVpuEncCodecs"

Default: 2, "std_avc" Current: 2, "std_avc"

(0): std_mpeg4 - STD_MPEG4

(1): std_h263 - STD_H263

(2): std_avc - STD_AVC

(7): std_mjpg - STD_MJPG

So at least for me it looks like H263 is codec 1. Unfortunately, H263 will not work as an mp4, so I would need to find out this issue.

I am slightly confused about the code you posted. You'd like to hard code NV12? Or are you saying this tidbit is hard codded into the program?