topic imx477 sensor driver for i.MX8M-Plus in i.MX Processors

imx477 sensor driver for i.MX8M-Plus

malik_cisse — Sat, 01 May 2021 11:35:36 GMT

Hi,

I am writing imx477 sensor driver for i.mx8m-plus based on some preexisting driver in kernel_source_code/drivers/media/i2c/imx319.c (which is part of official linux kernel)

Unfortunatelly I get some kernel error: "mxc-md: probe of 32c00000.bus:camera failed with error -515"

As can be seen in dmesg shippet below.

I have the feeling that some system callback functions aremissing.

Any insight? Thank you

dmesg output:

...

[ 3.229148] bus: 'i2c': driver_probe_device: matched device 2-001a with driver imx477
[ 3.229153] bus: 'i2c': really_probe: probing driver imx477 with device 2-001a
[ 3.229181] imx477 2-001a: no pinctrl handle
[ 3.229209] imx477 2-001a: ########## Probing IMX477 Driver ##########
[ 3.236113] link-frequencies 0 value 450000000
[ 3.240597] imx477 2-001a: 2-001a supply VANA not found, using dummy regulator
[ 3.247872] devices_kset: Moving 2-001a to end of list
[ 3.247876] PM: Moving i2c:2-001a to end of list
[ 3.247883] imx477 2-001a: Linked as a consumer to regulator.0
[ 3.247897] imx477 2-001a: 2-001a supply VDIG not found, using dummy regulator
[ 3.255179] imx477 2-001a: 2-001a supply VDDL not found, using dummy regulator
[ 3.273479] imx477 2-001a: $$$ Read I2C: IMX477 CHIP ID = 477 $$$
[ 3.279644] imx477 2-001a: ########## End Probing IMX477##########
[ 3.285839] driver: 'imx477': driver_bound: bound to device '2-001a'
[ 3.285844] devices_kset: Moving 2-001a to end of list
[ 3.285851] PM: Moving i2c:2-001a to end of list
[ 3.285877] bus: 'i2c': really_probe: bound device 2-001a to driver imx477

...

[ 3.660712] platform 32c00000.bus:camera: Retrying from deferred list
[ 3.661350] bus: 'platform': driver_probe_device: matched device 32c00000.bus:camera with driver mxc-md
[ 3.661358] bus: 'platform': really_probe: probing driver mxc-md with device 32c00000.bus:camera
[ 3.661376] mxc-md 32c00000.bus:camera: no pinctrl handle
[ 3.661518] device: 'video0': device_add
[ 3.661570] PM: Adding info for No Bus:video0
[ 3.661672] mx8-img-md: Registered mxc_isi.0.capture as /dev/video0
[ 3.668100] mx8-img-md: Registered sensor subdevice: imx477 2-001a (1)
[ 3.674700] mx8-img-md: created link [mxc_isi.0] => [mxc_isi.0.capture]
[ 3.681355] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.0]
[ 3.687817] mx8-img-md: subdev_notifier_complete error exit
[ 3.693453] mxc-md 32c00000.bus:camera: Sensor register failed
[ 3.699324] mxc-md: probe of 32c00000.bus:camera failed with error -515

Re: imx477 sensor driver for i.MX8M-Plus

igorpadykov — Sun, 02 May 2021 23:01:33 GMT

Hi Malik

one can try to use ov2775 as example and
i.MX 8M Plus Camera Sensor Porting User Guide
https://source.codeaurora.org/external/imx/linux-imx/tree/arch/arm64/boot/dts/freescale/imx8mp-evk-dual-ov2775.dts?h=lf-5.10.y

https://www.nxp.com/products/processors-and-microcontrollers/arm-processors/i-mx-applications-processors/i-mx-8-processors/i-mx-8m-plus-arm-cortex-a53-machine-learning-vision-multimedia-and-industrial-iot:IMX8MPLUS?tab=Documentation_Tab

Camera Enablement Package
https://www.baslerweb.com/en/sales-support/downloads/software-downloads/basler-camera-enablement-package-fuer-nxps-i-mx-8m-plus-prozessor-ga/

prozessor-ga/

Best regards
igor

Re: imx477 sensor driver for i.MX8M-Plus

malik_cisse — Mon, 03 May 2021 06:58:42 GMT

I will look at those examples.

Thank you Igor

Malik

Re: imx477 sensor driver for i.MX8M-Plus

gghy — Fri, 07 May 2021 13:18:08 GMT

Hi, i'm having the same problem with a imx334 sensor, if i find anything useful i will keep you updated

Re: imx477 sensor driver for i.MX8M-Plus

malik_cisse — Mon, 10 May 2021 08:00:54 GMT

Hi,

I finally solved the missing callback function issue.

One needs to implement dummy ".link_setup" callback function returning 0. The system is expecting this callback. This is a bug in the Kernel driver.

static const struct media_entity_operations ar0144_entity_ops = {
.link_setup = ar0144_link_setup,
};

static int ar0144_link_setup(struct media_entity *entity,
struct media_pad const *local,
struct media_pad const *remote,
u32 flags)
{

return 0;
}

Re: imx477 sensor driver for i.MX8M-Plus

malik_cisse — Mon, 10 May 2021 09:02:59 GMT

Thank you Igor.

Yes, Basler Cam is definitelly a good source of knowledge to get started.

Do you know if one can buy the NXP devkit + Basler cam?

Thx, Malik

Re: imx477 sensor driver for i.MX8M-Plus

igorpadykov — Mon, 10 May 2021 13:15:47 GMT

one can look at

https://www.nxp.com/design/development-boards/i-mx-evaluation-and-development-boards/evaluation-kit-for-the-i-mx-8m-plus-applications-processor:8MPLUSLPD4-EVK#buy

https://www.baslerweb.com/en/embedded-vision/embedded-vision-portfolio/embedded-vision-kits/daa3840-30mc-imx8mp-evk/

Best regards
igor

Re: imx477 sensor driver for i.MX8M-Plus

Goko_gr — Tue, 28 Jun 2022 05:49:13 GMT

Hi Malik,

We've got the same subdev_notifier_complete error exit with adv7280 chip. No NXP employees have supported us so far as expected, so if you can help us about our issue as well, it would be very appreciated. Thanks for your help. You can find the link of our issue below.

https://community.nxp.com/t5/i-MX-Processors/iMX8MP-MIPI-CSI2-Problem-with-ADV7280-and-Camera/m-p/1477496#M191752

Re: imx477 sensor driver for i.MX8M-Plus

malik_cisse — Tue, 28 Jun 2022 07:24:49 GMT

Hi,

This is long time ago.

Not sure how I solved this particular issue.

One of the key factors was that I needed to implement callback functions that are needed in the imx8mp BSP which are not present in other drivers.

For example .link_setup callback was missing but no error indicated that as you can see further in this thread.
One idea is to take a known functioning driver (e.g Basler) and check if all callbacks there are implemented in your own driver.

Re: imx477 sensor driver for i.MX8M-Plus

Goko_gr — Tue, 28 Jun 2022 13:21:06 GMT

Hi Malik,

Thanks a lot for your answer. We are also using iMX8MP and i wonder which driver file name have you modified and made the link operation ? In our case, we found a similar code part as yours, but it already returns zero.

static const struct media_entity_operations csi_entity_ops = {
.link_setup = csi_link_setup,
.link_validate = v4l2_subdev_link_validate,
.get_fwnode_pad = csi_get_fwnode_pad,
};

static int csi_link_setup(struct media_entity *entity,
const struct media_pad *local,
const struct media_pad *remote, u32 flags)
{

struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
struct csi_priv *priv = v4l2_get_subdevdata(sd);
struct v4l2_subdev *remote_sd;
int ret = 0;

dev_dbg(priv->dev, "link setup %s -> %s\n", remote->entity->name,
local->entity->name);

mutex_lock(&priv->lock);

if (local->flags & MEDIA_PAD_FL_SINK) {
if (!is_media_entity_v4l2_subdev(remote->entity)) {
ret = -EINVAL;
goto out;
}

remote_sd = media_entity_to_v4l2_subdev(remote->entity);

if (flags & MEDIA_LNK_FL_ENABLED) {
if (priv->src_sd) {
ret = -EBUSY;
goto out;
}
priv->src_sd = remote_sd;
} else {
priv->src_sd = NULL;
}

goto out;
}

/* this is a source pad */

if (flags & MEDIA_LNK_FL_ENABLED) {
if (priv->sink) {
ret = -EBUSY;
goto out;
}
} else {
v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
v4l2_ctrl_handler_init(&priv->ctrl_hdlr, 0);
priv->sink = NULL;
/* do not apply IC burst alignment in csi_try_crop */
priv->active_output_pad = CSI_SRC_PAD_IDMAC;
goto out;
}

/* record which output pad is now active */
priv->active_output_pad = local->index;

/* set CSI destination */
if (local->index == CSI_SRC_PAD_IDMAC) {
if (!is_media_entity_v4l2_video_device(remote->entity)) {
ret = -EINVAL;
goto out;
}

if (priv->fim) {
ret = imx_media_fim_add_controls(priv->fim);
if (ret)
goto out;
}

priv->dest = IPU_CSI_DEST_IDMAC;
} else {
if (!is_media_entity_v4l2_subdev(remote->entity)) {
ret = -EINVAL;
goto out;
}

remote_sd = media_entity_to_v4l2_subdev(remote->entity);
switch (remote_sd->grp_id) {
case IMX_MEDIA_GRP_ID_IPU_VDIC:
priv->dest = IPU_CSI_DEST_VDIC;
break;
case IMX_MEDIA_GRP_ID_IPU_IC_PRP:
priv->dest = IPU_CSI_DEST_IC;
break;
default:
ret = -EINVAL;
goto out;
}
}

priv->sink = remote->entity;
out:
//return 0; //The part that we have added.
mutex_unlock(&priv->lock);
return ret;
}

In this structure, we have modified the csi_link_setup just as you did, but there is no change. So, it would be very helpful if you share your driver file, ofc if you still have it or do you have any recommendation to solve this issue by changing exactly which part of the driver file.

Thanks again, have a nice day.

Re: imx477 sensor driver for i.MX8M-Plus

malik_cisse — Wed, 29 Jun 2022 07:21:35 GMT

Hi,
Unfortunately I cannot share the code because I am under NDA.

I do understand your frustration though since the debug logs often doe not indicate actual root cause.