Add support for RPi camera V2 in imx93

[dhruvinrajpura]

Hello,

I am using imx93-11x11 evk and I have RPi camera V2, I want to add support for that in yocto build with core-image-base I am building,

Can anyone provide me patch for dts and h/w connection I required to do to Up camera module?

[joanxie]

Pls refer to this application onte

https://docs.nxp.com/bundle/UM11933/page/topics/getting_started.html

[dhruvinrajpura]

@joanxie 

I resolved issue and getting /dev/video0

root@imx93-11x11-lpddr4x-evk:~# media-ctl -p
Media controller API version 6.6.23

Media device information
------------------------
driver          mxc-md
model           FSL Capture Media Device
serial          
bus info        platform:42800000.bus:camera
hw revision     0x0
driver version  6.6.23

Device topology
- entity 1: mxc_isi.0 (16 pads, 2 links, 0 routes)
            type V4L2 subdev subtype Unknown flags 0
	pad0: Sink
		<- "mxc-mipi-csi2.0":4 [ENABLED]
	pad1: Sink
	pad2: Sink
	pad3: Sink
	pad4: Sink
	pad5: Sink
	pad6: Sink
	pad7: Sink
	pad8: Sink
	pad9: Sink
	pad10: Sink
	pad11: Sink
	pad12: Source
		-> "mxc_isi.0.capture":0 [ENABLED]
	pad13: Source
	pad14: Source
	pad15: Sink

- entity 18: mxc_isi.0.capture (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video0
	pad0: Sink
		<- "mxc_isi.0":12 [ENABLED]

- entity 22: mxc-mipi-csi2.0 (8 pads, 2 links)
             type Node subtype V4L flags 0
             device node name /dev/v4l-subdev0
	pad0: Sink
		<- "imx219 2-0010":0 [ENABLED,IMMUTABLE]
	pad1: Sink
	pad2: Sink
	pad3: Sink
	pad4: Source
		-> "mxc_isi.0":0 [ENABLED]
	pad5: Source
	pad6: Source
	pad7: Source

- entity 31: imx219 2-0010 (1 pad, 1 link, 0 routes)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev1
	pad0: Source
		[stream:0 fmt:SRGGB10_1X10/3280x2464 field:none colorspace:raw xfer:none quantization:full-range
		 crop.bounds:(8,8)/3280x2464
		 crop:(8,8)/3280x2464]
		-> "mxc-mipi-csi2.0":0 [ENABLED,IMMUTABLE]

root@imx93-11x11-lpddr4x-evk:~# 




root@imx93-11x11-lpddr4x-evk:~# v4l2-ctl --list-device
FSL Capture Media Device (platform:42800000.bus:camera):
	/dev/media0

mxc-isi-cap_v1 (platform:4ae40000.isi:cap_devic):
	/dev/video0

root@imx93-11x11-lpddr4x-evk:~# 

But I am not able to capture ? How can I capture? Can you anlyze above and give me commands for capture?

[HankChang888]

In the end, was the porting successful? I’m currently also trying to port it, and I can now see the list of video devices, but the streamed video data is empty.

[dhruvinrajpura]

Hello @joanxie 

For this application note, default ap1302 is enabled, for this full image is required or it will be there in minimal image as well?, As you know I am adding support for imx219 so for that I am building minimal image so if i just add dts patch will it work or not?

[joanxie]

minimal is A small image that only allows a device to boot. I used multimedia before,  I don't test minimal for imx93 camera porting , it should be ok

[dhruvinrajpura]

Hi @joanxie 

Can you check once?

[joanxie]

you can started with minimal image, then add extra module by yourselves, our AN is based multimedia or full images, not based on minimal, if you have further issue about camera porting on the different images, pls create a new case, and attach your detailed issue

[dhruvinrajpura]

Hi @joanxie 

Thanks for reply,

This is for ap1302 I am using camera module: RPi camera V2 and that uses sensor Sony IMX219, so if I directly connect that with above steps, will it work?

[joanxie]

normally customer use IMX219 output as raw data, imx93 couldn't handle this directly, if you just need capture, imx93 can support it, then you use SW to convert raw data to other format, pls check your mail, I send the patch to you for reference

[HankChang888]

Hi joanxie ,

Could you please share the patch you mentioned for reference?

[dhruvinrajpura]

Hi @joanxie 

Applied your given patches attaching error log file that I am getting while compiling.

[joanxie]

how about changing int imx219_probe(struct i2c_client *client, const struct i2c_device_id *id) to int imx219_probe(struct i2c_client *client) ?

[dhruvinrajpura]

Hello @joanxie 

Yes we can, but before this also I got some error in different drivers as well, I am using 6.6-scarthgap can you check on that, I need to add your MAKEFILE patch separately as lines num also not matching

[joanxie]

as I mentioned in the mail, this isn't official version, just for reference, you need porting to your own bsp, not just apply the whole patch directly, you can change your source code based on your own bsp

[dhruvinrajpura]

Hello @joanxie 

I applied patch in dts and probe successfully imx219 but getting below while loading driver imx8_media_dev

root@imx93-11x11-lpddr4x-evk:~# dmesg | grep -E 'imx219|mx8-img-md'
[ 2.529849] imx219: starting imx219_probe() function
[ 2.529859] imx219: Allocated memory for imx219 struct
[ 2.529888] imx219: Initialized CCI regmap
[ 2.529898] imx219: Got system clock (xclk)
[ 2.529901] imx219: xclk frequency validated: 24000000 Hz
[ 2.529916] imx219 2-0010: supply VANA not found, using dummy regulator
[ 2.536651] imx219 2-0010: supply VDIG not found, using dummy regulator
[ 2.543291] imx219 2-0010: supply VDDL not found, using dummy regulator
[ 2.549915] imx219: Regulators acquired
[ 2.550112] imx219: Optional reset GPIO requested
[ 2.557779] imx219: Sensor set to streaming mode
[ 2.558012] imx219: Sensor returned to standby mode
[ 2.558181] imx219: Subdev flags and function set
[ 2.558185] imx219: Media entity pads initialized
[ 2.558189] imx219: Subdev initialization finalized
[ 2.558199] imx219: Subdev sensor registered asynchronously
[ 2.558296] imx219: Runtime PM initialized
[ 7.541231] mx8-img-md: Registered mxc_isi.0.capture as /dev/video0
[ 7.567996] mx8-img-md: Registered sensor subdevice: imx219 2-0010 (1)
[ 7.600977] mx8-img-md: created link [mxc_isi.0] => [mxc_isi.0.capture]
[ 7.643109] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.0]
[ 7.685370] mx8-img-md: subdev_notifier_complete error exit
[ 7.697350] mx8-img-md: Unregistered all entities

[joanxie]

refer to your error message and limited information, pls debug imx8-media-dev.c, you can add more print to check what the root cause is,

[dhruvinrajpura]

@joanxie 

I resolved that one and getting /dev/video0 now but while capturing I am getting issue

I am giving you all logs for reference

root@imx93-11x11-lpddr4x-evk:~# media-ctl -p
Media controller API version 6.6.23

Media device information
------------------------
driver          mxc-md
model           FSL Capture Media Device
serial          
bus info        platform:42800000.bus:camera
hw revision     0x0
driver version  6.6.23

Device topology
- entity 1: mxc_isi.0 (16 pads, 2 links, 0 routes)
            type V4L2 subdev subtype Unknown flags 0
	pad0: Sink
		<- "mxc-mipi-csi2.0":4 [ENABLED]
	pad1: Sink
	pad2: Sink
	pad3: Sink
	pad4: Sink
	pad5: Sink
	pad6: Sink
	pad7: Sink
	pad8: Sink
	pad9: Sink
	pad10: Sink
	pad11: Sink
	pad12: Source
		-> "mxc_isi.0.capture":0 [ENABLED]
	pad13: Source
	pad14: Source
	pad15: Sink

- entity 18: mxc_isi.0.capture (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video0
	pad0: Sink
		<- "mxc_isi.0":12 [ENABLED]

- entity 22: mxc-mipi-csi2.0 (8 pads, 2 links)
             type Node subtype V4L flags 0
             device node name /dev/v4l-subdev0
	pad0: Sink
		<- "imx219 2-0010":0 [ENABLED,IMMUTABLE]
	pad1: Sink
	pad2: Sink
	pad3: Sink
	pad4: Source
		-> "mxc_isi.0":0 [ENABLED]
	pad5: Source
	pad6: Source
	pad7: Source

- entity 31: imx219 2-0010 (1 pad, 1 link, 0 routes)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev1
	pad0: Source
		[stream:0 fmt:SRGGB10_1X10/3280x2464 field:none colorspace:raw xfer:none quantization:full-range
		 crop.bounds:(8,8)/3280x2464
		 crop:(8,8)/3280x2464]
		-> "mxc-mipi-csi2.0":0 [ENABLED,IMMUTABLE]

root@imx93-11x11-lpddr4x-evk:~# 




root@imx93-11x11-lpddr4x-evk:~# v4l2-ctl --list-device
FSL Capture Media Device (platform:42800000.bus:camera):
	/dev/media0

mxc-isi-cap_v1 (platform:4ae40000.isi:cap_devic):
	/dev/video0

root@imx93-11x11-lpddr4x-evk:~# 

$ gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! queue ! tcpclientsink host=192.168.68.209 port=>video/x-raw,format=NV12,width=1920,height=1080 ! videoconvert ! queue ! tcpclientsink host=192.168.68.209 port=5000

Setting pipeline to PAUSED ...

$ gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=1920,height=1080 ! videoconvert ! queue ! tcpclientsink host=192.168.68.209 port=5000

Setting pipeline to PAUSED ...

$ gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=480 ! videoconvert ! autovideosink

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0: Failed to initializ[ 46.432183] mxc_isi.0: Call subdev s_power fail!
e egl: EGL_NOT_INITIALIZED
Additional debug info:
/usr/src/debug/gstreamer1.0-plugins-base/1.24.7.imx/ext/gl/gstglimagesink.c(1140): _ensure_gl_setup (): /GstGLImageSinkBin:autovideosink0-actual-sink-glimage/GstGLImageSink:sink
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
Additional debug info:
/usr/src/debug/gstreamer1.0-plugins-good/1.24.7.imx/sys/v4l2/gstv4l2src.c(956): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.016970917
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
/usr/src/debug/gstreamer1.0/1.24.7.imx/libs/gst/base/gstbasesrc.c(3177): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Freeing pipeline ...

[ 158.156378] mxc_isi.0: Call subdev s_power fail!
[ 328.749218] mxc_isi.0: Call subdev s_power fail!
[ 350.954960] mxc_isi.0: Call subdev s_power fail!
[ 509.022065] mxc_isi.0: Call subdev s_power fail!
[ 826.091379] mxc_isi.0: Call subdev s_power fail!
[ 1043.341463] mxc_isi.0: Call subdev s_power fail!
[ 1454.303027] mxc_isi.0: Call subdev s_power fail!

[dhruvinrajpura]

Hello,

Okay, will check patch, actually I have RPI camera module v2 and from h/w spec I got to know it using imx219

I have the camera with flex cable attached in it want to up that one.