ov5640 v4l2-ctrl in imx8m Ultra lite

[dbt]

Hello everyone.

I'm facing difficulties trying to make the ov5640 working in a preexisting system which should go with Imx8M ultra lite.

I've managed to configure the dts file and the driver ov5640_mipi_v2.c in order to get the camera working with gst-launch with the following command:

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=2592,height=1944, framerate=15/1 ! avimux ! filesink location=./output.avi

And I've correctly managed the driver to support the resolutions framerates i'm interested in.

However problems starts trying to integrate the camera in a software we need to use which rely on various xioctl which seems to not be supported by the driver. Thoose xioctl are VIDIOC_REQBUF, VIDIOC_QUERYBUF, VIDIOC_QUERYCTRL, VIDIOC_G_FMT and for shure some other.

Am I missing some part of the v4l2 infrastructure?
I've tried to face up the VIDIOC_QUERYCTRL missing response to the queryctrl.
According to https://www.kernel.org/doc/html/v4.9/media/kapi/v4l2-controls.html#c.v4l2_query_ext_ctrl I have my ctrl_handler defined inside the ov5640_ctrls of the ov5640 struct (sensor)

const struct v4l2_ctrl_ops *ops = &ov5640_ctrl_ops;

struct ov5640_ctrls *ctrls = &sensor->ctrls;

struct v4l2_ctrl_handler *hdl = &ctrls->handler;

v4l2_ctrl_handler_init(hdl, 32);

after adding some sort of controls like:
ctrls->brightness = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_BRIGHTNESS, -0xff, 0xff, 1, 0);
sensor->subdev.ctrl_handler = hdl;

inside my ov5640_ctrl_ops the s_ctrl function is mapped to my ov5640_s_ctrl function in which I control all the controls request.

Despite all of the implementation seems done to me I think some parts are missing because after a xioctl with VIDIOC_QUERYCTRL nothing of my controls are reported, the call is handled by v4l2_queryctrl in v4l2-ctrls.c

Despite all my ctrls inserted in ov5640_mipi_v2, v4l2-ctl --device=/dev/video0 -l only output this:

User Controls

horizontal_flip 0x00980914 (bool) : default=0 value=0
vertical_flip 0x00980915 (bool) : default=0 value=0
alpha_component 0x00980929 (int) : min=0 max=255 step=1 default=0 value=0

Can I have some guidelines in order to integrate my v4l2 subdev ov5640 driver to the whole system in order to support all the xioctl my application is requesting?

[dbt]

No problem, thanks for the reply! Unfortunately I already have v4l2_device_register_subdev_nodes in subdev_notifier_complete.

I have updates since I'm still working on this: I can save the v4l2_ctrl_handler* pointer that is created with v4l2_ctrl_handler_init inside my ov5640 driver in a global pointer, then, inside the function findRef of v4l2-ctrls.c I can override the v4l2_ctrl_handler pointer in this way:

struct v4l2_ctrl_handler* manually_registered_ctrl_handler;

EXPORT_SYMBOL(manually_registered_ctrl_handler); //<--- populated by ov5640 driver

/* Find a control with the given ID. */

static struct v4l2_ctrl_ref *find_ref(struct v4l2_ctrl_handler *hdl, u32 id)

{

  struct v4l2_ctrl_ref *ref;

  int bucket;

  if(manually_registered_ctrl_handler)

  {

    printk("OV5640 OVERRIDE: change ctrl_handler\n");

    hdl = manually_registered_ctrl_handler;

  }

....rest of find_ref function

In this way I obtain all the controls value that I set in the driver.
Honestly I'm missing which part of the driver are not "automatically integrated with v4l2 system of the kernel" since with gst-launch I can perfectly launch a video stream...

[dbt]

Greetings, I'm still blocked here, hopeing someone can drive me the correct way.

When I make a xioctl VIDIOC_QUERYCTRL, the call is handled in the kernel space by

find_ref (in v4l2-ctrl.c) which only finds 1 bucket with 4 controls id: =9963817(Alpha Component), 9963797(Vertical Flip) , 9963796(Horizontal Flip) and 9961473(User Controls).

Is in this point where I should find all the controls that have been placed inside the ctrl_handler of my ov5640_mipi_v2 driver?

[joanxie]

sorry for my late response,  you can refer to the link as belowhttps://community.nxp.com/t5/i-MX-Processors/v4l2-ctl-no-control-available-no-brightness-no-exposure-and-so/m-p/1578106

[dbt]

I have to clarify that thoose 4 controls (vertical flip, horizontal flip, alpha component and user controls) are listed even if I skip the whole process of adding controls inside my driver. So is seems the v4l2 infrastructure is in some way queryng available controls from a wrong position.

[joanxie]

it seems that you use all of your own v4l2 source code, the link I sent to you, the customer can use other controls like brightness, maybe  you can refer to the nxp v4l2 source code, this is document and source code

https://github.com/nxp-imx/linux-imx/blob/lf-6.6.y/Documentation/userspace-api/media/v4l/control.rst

https://github.com/nxp-imx/linux-imx/blob/lf-6.6.y/drivers/media/v4l2-core/v4l2-ctrls-core.c

[dbt]

More precisely:
Why despite having in ov5640_mipi_v2.c the following code

v4l2_ctrl_handler_init(hdl, 32);

v4l2_ctrl_new_std(hdl, ops, V4L2_CID_BRIGHTNESS, -0xff, 0xff, 1, 0 );

sensor->subdev.ctrl_handler = hdl;

v4l2_async_register_subdev_sensor_common(&sensor->subdev);

If I try to make a userspace xioctl(pcontext->device.fd, VIDIOC_QUERYCTRL, &queryctrl) where 

queryctrl.id = V4L2_CID_BRIGHTNESS, at the kernel level My call is handled in

v4l2_ctrls.c -> v4l2_queryctrl and then v4l2_query_ext_ctrl is call which returns -1 to my xioctl and no one of the value I put in ctrl inside ov5640_mipi_v2 are returned to userspace?

Thanks for supporting my problem

[dbt]

Actually I'm trying to understand why the driver I'm using doesn't seems to have V4L2_CAP_VIDEO_CAPTURE but only V4L2_CAP_VIDEO_CAPTURE_MPLANE as video capability, is all the functions of VIDEO_CAPTURE included in a device that has VIDEO_CAPTURE_MPLANE capability?

Is actually mxc_isi_cap_querycap that is flagging

cap->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE_MPLANE;

to VIDIOC_QUERYCAP requert, is this the correct behaviour?

In the documentation you proposed me there is no reference to VIDIOC_QUERYCTRL, what is the way I should follow to implement this function? shouldn't be "automatically implemented" once the driver has done  v4l2_ctrl_new_std...?

Thanks in advance

[joanxie]

you can refer to the chapter 6.1.7 V4L2 Capture of enclosed file