Integrating Sony IMX586 Camera on IMX8MP platform

[BillR1]

Hi, I have followed the standard porting proceedure for the IMX586 sensor.  It seems like for the most part "all systems go" except for the error at the end when the the adapter-0 cannot be opened.  

Start the ISP media server
-----------------------------------------------------------------------

root@imx8mp:~# /opt/imx8-isp/bin/start_isp.sh
Starting isp_media_server for IMX586
RUN_SCRIPT=/opt/imx8-isp/bin/run.sh
RUN_SCRIPT_PATH=/opt/imx8-isp/bin
Trying configuration "imx586_4k_60"...
Removing vvcam-video...
Removing vvcam-isp...
Removing vvcam-dwe...
Removing imx8-media-dev...
Removing ov2775...
Removing os08a20...
Removing imx586 basler-camera-driver-vvcam...
Loading module imx586 ...
imx586
Loaded /lib/modules/6.1.36+/extra/imx586.ko
Loading module imx8-media-dev ...
imx8_media_dev
Loaded /lib/modules/6.1.36+/kernel/drivers/staging/media/imx/imx8-media-dev.ko
Loading module vvcam-dwe ...
vvcam_dwe
Loaded /lib/modules/6.1.36+/extra/vvcam-dwe.ko
Loading module vvcam-isp ...
vvcam_isp
Loaded /lib/modules/6.1.36+/extra/vvcam-isp.ko
Loading module vvcam-video ...
vvcam_video
Loaded /lib/modules/6.1.36+/extra/vvcam-video.ko
Starting isp_media_server with configuration file CAMERA0

...

Kernel output during the above
---------------------------------------------------------------------

[ 142.618951] enter isp_hw_remove
[ 142.622407] vvcam isp driver removed
[ 142.626234] enter isp_hw_remove
[ 142.629903] vvcam isp driver removed
[ 142.678850] enter viv_dwe_exit_module
[ 142.682740] enter dwe_hw_remove
[ 142.686024] enter fake_pdev_release
[ 142.689957] vvcam dewarp driver removed
[ 142.742167] : Unregistered all entities
[ 142.818522] Enter imx586_remove
[ 142.821677] imx586_power_off is bypassed, power fixed on for now
[ 142.827719] imx586 4-001a: imx586_regulator_disable todo
[ 142.966247] Enter imx586_probe
[ 142.969347] Setting mclk to 25 MHz
[ 142.974656] camera sensor chip id reported as 0x586
[ 142.979584] media_entity_pads_init()
[ 142.983180] v4l2_async_register_subdev_sensor()
[ 143.009429] imx586_probe camera mipi imx586, is found
[ 143.053308] imx8_media_dev: module is from the staging directory, the quality is unknown, you have been warned.
[ 143.064125] mx8-img-md: Registered sensor subdevice: imx586 4-001a (1)
[ 143.070692] mx8-img-md: created link [imx586 4-001a] => [mxc-mipi-csi2.0]
[ 143.077497] mxc-md 32c00000.bus:camera: mxc_md_create_links
[ 143.096969] Enter imx586_priv_ioctl, command 2154321408 (0x80685600)
[ 143.097687] Entering mipi_csis_imx8mp_phy_reset
[ 143.103634] Enter imx586_query_capability
[ 143.112371] imx586_query_capability, adapter 30ad0000.i2c (4) is found
[ 143.126133] enter viv_dwe_init_module
[ 143.130124] enter dwe_hw_probe
[ 143.133621] vvcam dewarp driver probed
[ 143.177634] enter viv_isp_init_module
[ 143.181614] enter isp_hw_probe
[ 143.185075] vvcam isp driver registered
[ 143.189129] enter isp_hw_probe
[ 143.192676] vvcam isp driver registered
[ 143.242058] enter viv_video_init_module
[ 143.282512] enter isp_mi_stop
[ 143.286685] enter isp_mi_stop

root@imx8mp:~# v4l2-ctl --list-devices
():
/dev/v4l-subdev0
/dev/v4l-subdev2
/dev/v4l-subdev3

():
/dev/v4l-subdev4
/dev/v4l-subdev5

(csi0):
/dev/v4l-subdev1

FSL Capture Media Device (platform:32c00000.bus:camera):
/dev/media0

VIV (platform:viv0):
/dev/video2

VIV (platform:viv1):
/dev/video3

vsi_v4l2dec (platform:vsi_v4l2dec):
/dev/video1

vsi_v4l2enc (platform:vsi_v4l2enc):
/dev/video0

viv_media (platform:vvcam-video.0):
/dev/media1

Then try something

root@imx8mp:~# v4l2-ctl --device=/dev/video2 --all

Output from media server:

ADAPTER: AdaptOpen:open sensor failed in v4l2 mode! check sensor driver, cfg is imx586-vvcam
HAL-ALTERA: HalOpen: open adapter-0 failed in HAL
[CITF][ERR]: CitfApi create error!
[CITF][ERR]: initHardware new CitfApi 0 fail!
ERROR : [NativeSensor] mDriver.initHardware failed, id 0
ERROR : [MediaPipeline] NativeSensor open error!
ERROR : [V4l2Event] initialize MediaPipeline error!

I assume the following is incomplete because it isn't calling my driver properly.

root@imx8mp:~# v4l2-ctl --device=/dev/video2 --all
Driver Info:
Driver name : viv_v4l2_device
Card type : VIV
Bus info : platform:viv0
Driver version : 6.1.36
Capabilities : 0x84201001
Video Capture
Video Capture Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Extended Pix Format
Priority: 2
Video input : 0 (camera: ok)
Format Video Capture:
Width/Height : 0/0
Pixel Format : ''
Field : None
Bytes per Line : 0
Size Image : 0
Colorspace : SMPTE 170M
Transfer Function : Default (maps to Rec. 709)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Full Range)
Flags :
Crop Capability Video Capture:
Bounds : Left 0, Top 0, Width 0, Height 0
Default : Left 0, Top 0, Width 0, Height 0
Pixel Aspect: 1/1
Selection Video Capture: crop, Left 0, Top 0, Width 0, Height 0, Flags:
Selection Video Capture: crop_default, Left 0, Top 0, Width 0, Height 0, Flags:
Selection Video Capture: crop_bounds, Left 0, Top 0, Width 0, Height 0, Flags:
Selection Video Capture: compose, Left 0, Top 0, Width 0, Height 0, Flags:
Selection Video Capture: compose_default, Left 0, Top 0, Width 0, Height 0, Flags:
Selection Video Capture: compose_bounds, Left 0, Top 0, Width 0, Height 0, Flags:
Streaming Parameters Video Capture:
Capabilities : timeperframe
Frames per second: invalid (0/0)
Read buffers : 0

User Controls

viv_ext_ctrl 0x0098f901 (str) : min=0 max=65535 step=1 value='' flags=has-payload

The driver output during the above query shows  the following. 

[ 966.535532] Enter imx586_priv_ioctl, command 0x80685600
[ 966.546583] Enter imx586_query_capability
[ 966.550606] imx586_query_capability, adapter 30ad0000.i2c (4) is found
[ 966.565855] enter isp_mi_stop
[ 966.645776] enter isp_mi_stop

Seems like the only private ioctl that gets called in my driver is VIDIOC_QUERYCAP. Seems like this should be something basic I am missing.  I noticed mention in HAL-ALTERA of adapter-0 and my adapter is reported as 4, if they are the same thing. 

Bill

[joanxie]

refer to your logfile, open sensor failed in v4l2 mode! check sensor driver,it seems this error is related to your camera driver you need check your camera driver firstly

[BillR1]

Well I did check the driver originally first.  Then I decided to print out the v4l2 capabilities and got the following so I think I am on to something here. 

[63477.135304] Enter imx586_query_capability
[63477.143351] Driver: imx586
[63477.146058] Card:
[63477.148082] Bus Info: csi0
[63477.150800] Version: 0
[63477.153160] Capabilities: 0
[63477.155961] Device Caps: 0
[63477.158678] imx586_query_capability, adapter 30ad0000.i2c (4) is found

[BillR1]

Unfortunately missing capabilities was not the problem. (Also the other drivers do not do it either).  Also it appears that the error message is coming from the proprietary libcam_engine.so driver so I can not look to the source to find out what its not happy about. This is the point just before it should load imx586.drv.

[65062.927912] Entering mipi_csis_imx8mp_phy_reset
[65062.928038] Enter imx586_priv_ioctl, command -2140645888   #query caps
[65062.938077] imx586_query_capability, adapter 30ad0000.i2c (4) is found
[65062.944634] Driver: imx586
[65062.947379] Card: IMX586 Card
[65062.950369] Bus Info: csi0
[65062.953169] Version: 65536
[65062.955904] Capabilities: 67108865
[65062.959345] Device Caps: 67108865
[65063.037301] enter isp_mi_stop
ADAPTER: AdaptOpen:open sensor failed in v4l2 mode! check sensor driver, cfg is imx586-vvcam
HAL-ALTERA: HalOpen: open adapter-0 failed in HAL
[CITF][ERR]: CitfApi create error!
[CITF][ERR]: initHardware new CitfApi 0 fail!
ERROR : [NativeSensor] mDriver.initHardware failed, id 0
ERROR : [MediaPipeline] NativeSensor open error!
ERROR : [V4l2Event] initialize MediaPipeline error!

[joanxie]

you should focus on the v4l2 mode in the sensor driver of ISP, you can refer to the chapter 6.2 Camera Sensor Porting Setup in V4L2 Mode of enclosed file, you also can refer to the IMX219 package as reference

https://github.com/voxelbotics/imx8mp-imx219-sensor-enablement

[BillR1]

I made a regular v4l2 style driver on ISI. Its revealed some issues but now its not just black box of the isp_media_server proprietary libs telling me to check my driver. I have gotten a little further with it for now.

Bill