Integration of IMX219 camera sensor with NXP EVK board

[Jyo]

Hello All 

Camera: 8MP Sony IMX219 Camera Module for Raspberry Pi
Board: NXP i.MX8MP EVK
Interface: Connected directly via MIPI CSI2

I am currently working on enabling the IMX219 camera module on the NXP i.MX8MP EVK board. The camera driver is initializing correctly, and the sensor is being detected. However, I'm unable to get the camera stream to display over HDMI using Wayland.

gst-launch-1.0 v4l2src device=/dev/video3 ! 'video/x-raw,format=NV12,width=640,height=480' ! waylandsink

the process gets stuck during the ISP setup
[ 43.766523] enter isp_mi_stop
[ 44.098470] enter isp_mi_stop
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[ 46.078055] enter isp_set_stream 1


And below is the configuration output 

root@localhost:~# media-ctl -p

Media controller API version 6.6.36

Media device information

------------------------

driver     mxc-md

model      FSL Capture Media Device

serial

bus info    platform:32c00000.bus:camera

hw revision   0x0

driver version 6.6.36

Device topology

- entity 1: mxc-mipi-csi2.0 (8 pads, 1 link)

      type Node subtype V4L flags 0

      device node name /dev/v4l-subdev0

    pad0: Sink

        <- "imx219 1-0010":0 [ENABLED,IMMUTABLE]

    pad1: Sink

    pad2: Sink

    pad3: Sink

    pad4: Source

    pad5: Source

    pad6: Source

    pad7: Source

- entity 10: imx219 1-0010 (1 pad, 1 link, 0 routes)

       type V4L2 subdev subtype Sensor flags 0

       device node name /dev/v4l-subdev1

    pad0: Source

        [stream:0 fmt:unknown/0x0]

        -> "mxc-mipi-csi2.0":0 [ENABLED,IMMUTABLE]

root@localhost:~# v4l2-ctl --list-device

[ 176.158206] enter isp_mi_stop

 ():

    /dev/v4l-subdev0

    /dev/v4l-subdev2

    /dev/v4l-subdev3

 (csi0):

    /dev/v4l-subdev1

FSL Capture Media Device (platform:32c00000.bus:camera):

    /dev/media0

mxc-isi-m2m_v1 (platform:32e00000.isi:m2m_devic):

    /dev/video2

VIV (platform:viv0):

    /dev/video3

viv_media (platform:vvcam-video.0):

    /dev/media1

Failed to open /dev/video0: Device or resource busy

root@localhost:~#

 Could you please help me identify what might be going wrong?

[AshutoshNama]

Dear Jyo,
The IMX219 sensor is detected, and the media pipeline is mostly set up. The issue you're hitting during the isp_set_stream 1 phase suggests that the ISP (Image Signal Processor) isn't correctly linked to the sensor stream, or there's a missing format negotiation.

Here’s a short checklist to solve the issue:

Solution Steps:
Ensure the Format Is Set on Each Media Pad:

Run this:

bash
Copy
Edit
media-ctl -d /dev/media0 -V '"imx219 1-0010":0 [fmt:SRGGB10_1X10/640x480]'
media-ctl -d /dev/media0 -V '"mxc-mipi-csi2.0":0 [fmt:SRGGB10_1X10/640x480]'
media-ctl -d /dev/media0 -V '"mxc-mipi-csi2.0":4 [fmt:SRGGB10_1X10/640x480]'
Link Camera to ISI (Image Signal Interface):

Manually ensure the links are enabled. Example:

bash
Copy
Edit
media-ctl -d /dev/media0 --links '"mxc-mipi-csi2.0":4->"mxc-isi.0":0[1]'
Set Correct Format for ISI Output:

bash
Copy
Edit
media-ctl -d /dev/media0 -V '"mxc-isi.0":1 [fmt:NV12/640x480]'
Use the ISI Output Node, Not the VIV Sink:

Try this:

bash
Copy
Edit
gst-launch-1.0 v4l2src device=/dev/video2 ! 'video/x-raw,format=NV12,width=640,height=480' ! waylandsink
(/dev/video2 seems to be the mxc-isi-m2m output. /dev/video3 belongs to VIV, and may not be your actual camera stream.)

Jyo, If still need help feel free to share the problem statement

Best Regards,
Ashutosh Nama.

[Jyo]

Hello AshutoshNama 

Thanks for your reply . 
I have tried your changes . 

media-ctl -d /dev/media0 --links '"mxc-mipi-csi2.0":4->"mxc-isi.0":0[1]' 
But the above command returning -22 failure . 

Looks like some configuration is  missing from  ISI side.  

[AshutoshNama]

Jyo, Can you DM me so I can share you some helpful details, I'm running out of DM limit. 
my mail is ashutoshnama26@gmail.com

Best Regards,
Ashutosh Nama.

[Jyo]

Hi 

I followed the same  link , 
Please find  complete logs 

[Jyo]

Below are the config output 

root@localhost:/opt/imx8-isp/bin# media-ctl -p
Media controller API version 6.6.36

Media device information
------------------------
driver mxc-md
model FSL Capture Media Device
serial
bus info platform:32c00000.bus:camera
hw revision 0x0
driver version 6.6.36

Device topology
- entity 1: mxc-mipi-csi2.0 (8 pads, 1 link)
type Node subtype V4L flags 0
device node name /dev/v4l-subdev0
pad0: Sink
<- "imx219 1-0010":0 [ENABLED,IMMUTABLE]
pad1: Sink
pad2: Sink
pad3: Sink
pad4: Source
pad5: Source
pad6: Source
pad7: Source

- entity 10: imx219 1-0010 (1 pad, 1 link, 0 routes)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev1
pad0: Source
[stream:0 fmt:unknown/0x0]
-> "mxc-mipi-csi2.0":0 [ENABLED,IMMUTABLE]

root@localhost:/opt/imx8-isp/bin#
root@localhost:/opt/imx8-isp/bin#
root@localhost:/opt/imx8-isp/bin#
root@localhost:/opt/imx8-isp/bin#
root@localhost:/opt/imx8-isp/bin# v4l2-ctl --list-device
():
/dev/v4l-subdev0
/dev/v4l-subdev2
/dev/v4l-subdev3

(csi0):
/dev/v4l-subdev1

FSL Capture Media Device (platform:32c00000.bus:camera):
/dev/media0

mxc-isi-m2m_v1 (platform:32e00000.isi:m2m_devic):
/dev/video2

VIV (platform:viv0):
/dev/video3

vsi_v4l2dec (platform:vsi_v4l2dec):
/dev/video1

vsi_v4l2enc (platform:vsi_v4l2enc):
/dev/video0

viv_media (platform:vvcam-video.0):
/dev/media1


But its stuck while capturing the image . 
root@localhost:/opt/imx8-isp/bin# gst-launch-1.0 v4l2src device=/dev/video3 ! "video/x-raw ,format=YUY2, width=1920, height=1080" ! waylandsink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

Also I have uploaded the logs , Please check once and provide your guidence . 

[joanxie]

pls refer to the link as below, to check if you integrate is correct, then use the command to test again, if still failed, pls share your logfile with me

gst-launch-1.0 -v v4l2src device=/dev/video2 ! "video/x-raw,format=YUY2,width=1920,height=1080" ! queue ! waylandsink

https://github.com/nxp-imx-support/meta-imx8mp-isp-imx219

[Jyo]

The driver is working correctly and is able to read the chip ID. However, the CSI interface is not able to capture the camera buffer.

And from the config output , v4l2 driver also configure properly . 
But somehow MIPI CSI is not getting camera buffer properly . 

Not sure what causing the issue .