imx8mp isp video node not cteated

[mukesh_kumar]

Hi,

I'm porting imx290 sensor module with imx8mplus of 5.10.35 distro
I followed the camera sensor porting guide and made the driver changes in vvcam  accordingly, but when board is booted it is not creating any videonode even the driver's probe function is success.

below are the log messages from the time the modules starts loading:


[ 6.208014] imx290_vvcam: loading out-of-tree module taints kernel.
[ 6.224028] entering th func:imx290_probe
[ 6.318075] eimx290_vvcam 1-0042: supply DOVDD not found, using dummy regulator
[ 6.339949] eimx290_vvcam 1-0042: supply DVDD not found, using dummy regulator
[ 6.349485] eimx290_vvcam 1-0042: supply AVDD not found, using dummy regulator
[ 6.382951] imx8_media_dev: module is from the staging directory, the quality is unknown, you have been warned.
[ 6.443331] enter viv_isp_init_module
[ 6.448757] enter isp_hw_probe
[ 6.453073] enter viv_dwe_init_module
[ 6.456791] vvcam isp driver registered
[ 6.457080] enter dwe_hw_probe
[ 6.461686] imx-sdma 30e10000.dma-controller: firmware found.
[ 6.466027] vvcam dewarp driver probed
[ 6.470277] imx-sdma 30bd0000.dma-controller: firmware found.
[ 6.479325] imx-sdma 30bd0000.dma-controller: loaded firmware 4.6
[ 6.839512] caam algorithms registered in /proc/crypto
[ 6.896947] caam 30900000.crypto: caam pkc algorithms registered in /proc/crypto
[ 6.896962] caam 30900000.crypto: registering rng-caam
[ 6.898300] Device caam-keygen registered
[ 6.921808] random: crng init done
[ 6.921819] random: 7 urandom warning(s) missed due to ratelimiting
[ 7.309550] EXT4-fs (mmcblk2p2): mounted filesystem with ordered data mode. Opts: (null)
[ 7.383698] RTL8211F Gigabit Ethernet 30be0000.ethernet-1:01: attached PHY driver [RTL8211F Gigabit Ethernet] (mii_bus:phy_addr=30be0000.)
[ 7.475098] imx-dwmac 30bf0000.ethernet eth1: PHY [stmmac-1:01] driver [RTL8211F Gigabit Ethernet] (irq=POLL)
[ 7.487079] imx-dwmac 30bf0000.ethernet eth1: No Safety Features support found
[ 7.487093] imx-dwmac 30bf0000.ethernet eth1: IEEE 1588-2008 Advanced Timestamp supported
[ 7.487260] imx-dwmac 30bf0000.ethernet eth1: registered PTP clock
[ 7.487510] imx-dwmac 30bf0000.ethernet eth1: configuring for phy/rgmii-id link mode
[ 7.519467] 8021q: adding VLAN 0 to HW filter on device eth1
[ 8.143730] audit: type=1006 audit(1616581522.276:2): pid=639 uid=0 old-auid=4294967295 auid=0 tty=(none) old-ses=4294967295 ses=1 res=1
[ 8.192190] returnvalue after initilizing media entity pads of the camera subdev= 0
[ 8.196776] returnvalue after asynconous registering of the camera subdev= 0
[ 8.204482] entering th func:imx290_retrieve_capture_properties
[ 8.211560] imx290 detected.

When i give lsmod: check it is creating the following modules , vvcam_video module is not loaded

Module                 Size Used by

vvcam_isp 65536 0
vvcam_dwe 28672 0
imx8_media_dev 20480 0
imx290_vvcam 147456 0

when i tried removing the imx8_media_dev module and modprobe it again 

imx290_vvcam 147456 1

which made the module getting utilised, but still there is no video node created.
when i modprobe vvcam_video then it is loading but is not been utilised again as

vvcam_video 40960 0

please help me out to solve this issue. 

Thank you in advance,

Mukesh Kumar

[khang_letruong]

Hi @mukesh_kumar ,

Firstly, make sure that your vvcam kernel driver (mx290_vvcam.ko generated from imx290_vvcam.c for example) is loaded correctly when you execute the run.sh script within isp-imx framework. You might also modify it to add your imx290 sensor.

Then make sure that the following line is found in dmesg (i.e. the pipeline I2C sensor -> mipi-csi2 --> isp is well linked in dts) :

[    7.884122] mx8-img-md: created link [imx290 1-001a] => [mxc-mipi-csi2.0]

Similar to ov2775 or others, you can check in the below code snippet within probing function that all V4L2 functions return successfully :

        sd = &sensor->subdev;
        v4l2_i2c_subdev_init(sd, client, &ov2775_subdev_ops);
        sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
        sd->entity.function = MEDIA_ENT_F_CAM_SENSOR;
        sensor->pads[OV2775_SENS_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;

        retval = media_entity_pads_init(&sd->entity, OV2775_SENS_PADS_NUM,
                                        sensor->pads);
        sd->entity.ops = &ov2775_sd_media_ops;
        if (retval < 0)
                goto probe_err_power_down;

        retval = v4l2_async_register_subdev_sensor_common(sd);
        if (retval < 0) {
                dev_err(&client->dev,"%s--Async register failed, ret=%d\n", __func__,retval);
                goto probe_err_entity_cleanup;
        }

Br,
K

[mukesh_kumar]

Hi @khang_letruong 

Thanks for reply,
I have been through the run scripts, to add support for imx290 to it, it needs a .drv file and .xml files 

I tried generating one for imx290 using the help of the discussions at https://community.nxp.com/t5/i-MX-Processors/ISP-working-in-i-MX8M-Plus/m-p/1307705

but when i give the command

./build-isp-all release Full

it is building partially and not generating any .drv file for imx290 , even though the required files are added at isi/drv/ folder.

Thanks and regards,

mukesh

[khang_letruong]

Hi @mukesh_kumar,

Maybe you are referring the upper layer (ISI layer in isp-imx) for generating the .drv to be used by isp_media_server as well as scripts for loading the necessary kernel modules and calibration's configurations of the sensor itself. This is however another subject than the initial of this ticket : the creation of /dev/videoX following a successful probing of your kernel module which is imx290_vvcam.

I find myself that the discussion in the link you mentioned is well enough for implementing the upper layer including the .drv of your imx290.

BR,

K.

[khang_letruong]

Hi @mukesh_kumar,

Maybe you forgot to enable the &isp_0 node or &isp_1 node in your dts.

Regards,
K.

[Bio_TICFSL]

Good!

[mukesh_kumar]

Hi @khang_letruong 

Thanks for replying to the query,
I'm using isp_0 node and enabled the same in device tree source as shown below

&cameradev {
status = "okay";
};

&isi_0 {
status = "disable";
};

&isp_0 {
status = "okay";
};

&dewarp {
status = "okay";
};

I'm unable to understand which module/service  is invoking the vvcam_video module in basler case and not happening in ov2775 and imx290 as mentioned in my earlier reply.

Thanks and Regards,

Mukesh

[mukesh_kumar]

Hi, 

I have taken the reference of ov2775_mipi_v3.c file present in vvcam for the mentioned imx290 driver.
I observed that when i load imx8mp-evk-basler.dtb file  while booting
vvcam_isp 65536 1
vvcam_dwe 28672 1
vvcam_video 40960 1
imx8_media_dev 20480 0
basler_camera_driver_vvcam 20480 0

it is loading the modules ----> vvcam_video, vvcam_isp, vvcam_dwe and it is getting utilised, even no camera has been connected.

if I load  imx8mp-evk-ov2775.dtb file while booting, 
it is loading the modules ---->   with no device be, even no camera has been connected
vvcam_isp 65536 0
vvcam_dwe 28672 0
imx8_media_dev 20480 0
os08a20 32768 0 

I cant understand which is triggering to load vvcam_video incase of basler dtb even without camera.
please help me to understand this as in my case i'm unable to load the vvcam_video module while booting.

Also imx8_media_dev module isloading even before my imx290 module loads, how can i let the imx8_media_dev to load after my module probe got success?
i tried creating a file modules under /etc/ folder and kept the order as imx290 followed by imx8_media_dev, but it didn't worked.

can someone help me to slove these two things.

Thankyou,
Mukesh kumar.

[quercuspau]

Hi,

Is it not possible to use the already available driver for imx290 with isp?

I am planning to use the same sensor with imx8mp and ISP. However i haven't started yet

BR

[mukesh_kumar]

Hi @quercuspau 

According to sensor porting guide, the sensors supporting the isp should be present at vvcam, but there is no imx290 driver present at vvcam

Regards, 

mukesh

[quercuspau]

Hi @mukesh_kumar and @khang_letruong 

If I understood correctly, you used as reference the ov2775 module and adapt it with the imx290 driver register configuration. No expected changes on the device tree camera side. Is it ok to use the same xml and drv from ov2775 for imx290?

Regards

[khang_letruong]

Hi @quercuspau,

To be honest, the ov2775 together wirh os08a20 are the 2 good and open examples compared to Basler cameras.

You might need to adapt the device tree  in term of I2C bus/node (and its properties such as gpio, address), mipi-csi2 node (and its properties such as number of lanes, ...)   that your custom sensor connects to.

BR,
K

[quercuspau]

Hi @khang_letruong ,

As far as I read, you managed to run your camera with the ISP, how good does look your capturing picture? Also in some post I read that you contract some NXP camera assistance company and after receiving the xml and drv files, you still have some problems with the image calibration parameters. Have you able to solve them? If I can ask, you much cost the assistance company to tune your camera parameters?

BR

[khang_letruong]

Hi @quercuspau ,

ISP works well now. I have no more problem. For other info, sorry that I cannot disclose.

Best regards,

K.