Imx8mpevk board is not detecting mipi csi camera in the chromium browser

[Rajesh_]

I have used meta-browser to add chromium support to the image (imx-image-full).When I am trying to check the webcam by the https://webcamtests.com/ and https://webrtc.github.io/samples/  , usb camera is detected by the chromium in the above mentioned two websites but not my mipi camera .I am using the chromium version 101.0.4951.53 . Why my mipi csi camera is not detected by the chromium which was connected to  CSI1 MIPI port to the imx8mpevk board. Can anyone help me resolve this ? Thanks in advance .

[Rajesh_]

Hi @Dhruvit ,
I hope you are doing well.
I have added the MPLANE support patches into Chromium to enable Multiplanar device support and then I am able to stream the camera in
the browser. By default we have enabled eglfs in custom platform, and able to see streaming but with some artifacts like below images
even with youtube as well please check the attached images and videos. Streaming the camera in the browser works fine on Wayland(attached 
 the image image_no_artifacts from wayland)without any artifacts. However, when I captured the same images with artifacts(from eglfs) and viewed them in my pc they were fine(without any artifacts).Can you please help us to tackle this problem by suggesting the possibilities.

Thanks & Regards
Rajesh

[riteshkrmehta]

@Dhruvit,

Hope you are going well.

Are you able to crack down this problem why camera not enumerated in chromium?

as per previous comment from Rajesh it look like issue is on platform end.
Please help on this asap.

Thanks,

Ritesh

[Dhruvit]

Hi @Rajesh_,

I hope you are doing well.

I'm afraid that WebRTC supports MIPI CSI Camera.

One can try using gstreamer to test & stream the mipi csi camera.

Please refer to 7.3.8 Camera preview and 7.3.10 Web camera in i.MX Linux User's Guide

Thanks & Regards,
Dhruvit Vasavada

[Rajesh_]

Hi @Dhruvit 

I'm doing well, thank you! .Thank you for your response . I am able to stream the camera with the the following command 

gst-launch-1.0 v4l2src device=/dev/video3 ! fpsdisplaysink video-sink=waylandsink sync=false -v

But when I am trying to access it from the chromium browser and qtwebengine applications I am unable to access the mipi csi camera . Can you help me in this on where can I check regarding this . When I test the camera by using https://webcamtests.com/ I am  unable to detect the mipi camera what might be the reason.  Thank you for your help.

Thanks & regards 
RAJESH

[Dhruvit]

Hi @Rajesh_,

https://webcamtests.com/ is a third-party website for camera testing. One can put a review regarding the camera not detecting on their site.

Thanks & Regards,
Dhruvit Vasavada

[Rajesh_]

Hi @Dhruvit ,
Even when I try to test the WebRTC samples from the https://webrtc.github.io/samples/  by the QtWebEngine example browsers (Ex:simple browser) and Chromium, I'm facing issues with detecting the MIPI camera on my board. However, USB camera is detected and I can stream it. I suspect the reason for this issue is that the browsers are unable to locate the MIPI camera when it searches for video input devices on the board. The logs indicate that no devices are found during this process. You can see in the below log

UMP::SetupVideoInput. request_id=0, video constraints=
[1048:1059:1019/091825.186667:VERBOSE1:media_stream_manager.cc(1112)] MDM::DoEnumerateDevices({type=VIDEO_INPUT})
[1048:1059:1019/091825.186708:VERBOSE1:media_stream_manager.cc(1112)] video capture: VideoCaptureManager::EnumerateDevices
[1048:1059:1019/091825.186729:VERBOSE1:media_stream_manager.cc(1112)] video capture: ServiceVideoCaptureProvider::GetDeviceInfosAsync
Detected locale "C" with character encoding "ANSI_X3.4-1968", which is not UTF-8.
Qt depends on a UTF-8 locale, and has switched to "C.UTF-8" instead.
If this causes problems, reconfigure your locale. See the locale(1) manual
for more information.
[1048:1059:1019/091825.355046:VERBOSE1:media_stream_manager.cc(1112)] video capture: VideoCaptureManager::OnDeviceInfosReceived: Recevied 0 device infos.
[1048:1059:1019/091825.355136:VERBOSE1:media_stream_manager.cc(1112)] MDM::DevicesEnumerated({type=VIDEO_INPUT}, {labels])
[1134:1134:1019/091825.355427:VERBOSE1:webrtc_logging.cc(35)] UMP::SelectVideoDeviceSettings. request_id=0.
[1134:1134:1019/091825.355653:VERBOSE1:webrtc_logging.cc(35)] UMP::GetUserMediaRequestFailed({request_id=0})
[1134:1134:1019/091825.356001:VERBOSE1:webrtc_logging.cc(35)] UMP::DelayedGetUserMediaRequestFailed({request_id=0}, {result=NO_HARDWARE})

I assume that this is the platform related issue because mipi camera is treated as output device and chromium is checking for input device for streaming. Correct if I am wrong .

This is the camera information

root@imx8mpevk:~# v4l2-ctl --all -d /dev/video3
Driver Info:
        Driver name      : mxc-isi-cap
        Card type        : mxc-isi-cap
        Bus info         : platform:32e00000.isi:cap_devic
        Driver version   : 6.1.36
        Capabilities     : 0x84201000
                Video Capture Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04201000
                Video Capture Multiplanar
                Streaming
                Extended Pix Format
Priority: 2
Format Video Capture Multiplanar:
        Width/Height      : 640/480
        Pixel Format      : 'RGBP' (16-bit RGB 5-6-5)
        Field             : None
        Number of planes  : 1
        Flags             :
        Colorspace        : sRGB
        Transfer Function : Default
        YCbCr/HSV Encoding: ITU-R 601
        Quantization      : Full Range
        Plane 0           :
           Bytes per Line : 1280
           Size Image     : 614400
Crop Capability Video Output:
        Bounds      : Left 0, Top 0, Width 640, Height 480
        Default     : Left 0, Top 0, Width 640, Height 480
        Pixel Aspect: 1/1
Selection Video Output: compose, Left 0, Top 0, Width 640, Height 480, Flags:
Selection Video Output: compose_default, Left 0, Top 0, Width 640, Height 480, Flags:
Selection Video Output: compose_bounds, Left 0, Top 0, Width 640, Height 480, Flags:
Streaming Parameters Video Capture:
        Capabilities     : timeperframe
        Frames per second: 30.000 (30/1)
        Read buffers     : 0User Controls                horizontal_flip 0x00980914 (bool)   : default=0 value=0
                  vertical_flip 0x00980915 (bool)   : default=0 value=0
                alpha_component 0x00980929 (int)    : min=0 max=255 step=1 default=0 value=0

[Rajesh_]

Any update on the bug resolution ?

[Dhruvit]

Hi @Rajesh_,

I hope you are doing well.

We are working on it.

I will respond to you once I have any update.

I have shared you few comments over email.

Thanks & Regards,
Dhruvit Vasavada

[Rajesh_]

Hi @Dhruvit ,
I have updated the details . Can you please update me on this bug resolution as it has very high importance and  Let me know if any further information is required for resolving this .

Thanks,
Rajesh

[Dhruvit]

Hi @Rajesh_,

I think the issue is related to "Video Capture" vs "Video Capture Multiplanar".

Multi-planar API introduces new multi-planar formats. Those formats use a separate set of FourCC codes. It is important to distinguish between the multi-planar API and a multi-planar format. Multi-planar API calls can handle all single-planar formats as well (as long as they are passed in multi-planar API structures), while the single-planar API cannot handle multi-planar formats. (For more details, please check https://www.kernel.org/doc/html/v4.9/media/uapi/v4l/planar-apis.html)

You can see that your camera device caps is "Video Capture Multiplanar", but I guess the website you are using to test is using the single-planar API. That's why it cannot see the camera device.

Thanks & Regards,
Dhruvit Vasavada

[Rajesh_]

Hi @Dhruvit ,

I think it is not related to the websites as I have used multiple websites to test this but still getting the same error like below and I have also used multiple browsers to test this.

[1518:1518:1103/085030.163350:VERBOSE1:webrtc_logging.cc(32)] UMP::DelayedGetUserMediaRequestFailed({request_id=0}, {result=NO_HARDWARE})
js: Error accessing the camera: [object DOMException]

I tried it even by changing the multiplanar to single planar in the source code file by changing  V4L2_CAP_VIDEO_CAPTURE_MPLANE to V4L2_CAP_VIDEO_CAPTURE but still it doesn't worked.
you can see the modified camera details. Still it is giving the same error NO_HARDWARE.You can see the v4l2-ctl details below

Driver name : mxc-isi-cap
Card type : mxc-isi-cap
Bus info : platform:32e00000.isi:cap_devic
Driver version : 5.15.120
Capabilities : 0x84200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Extended Pix Format
Media Driver Info:
Driver name : mxc-md
Model : FSL Capture Media Device
Serial :
Bus info :
Media version : 5.15.120
Hardware revision: 0x00000000 (0)
Driver version : 5.15.120
Interface Info:
ID : 0x03000014
Type : V4L Video
Entity Info:
ID : 0x00000012 (18)
Name : mxc_isi.0.capture
Function : V4L2 I/O
Pad 0x01000013 : 0: Sink
Link 0x02000041: from remote pad 0x100000e of entity 'mxc_isi.0' (Video Pixel Formatter): Data, Enabled
Priority: 2
Crop Capability Video Output:
Bounds : Left 0, Top 0, Width 640, Height 480
Default : Left 0, Top 0, Width 640, Height 480
Pixel Aspect: 1/1
Selection Video Output: compose, Left 0, Top 0, Width 640, Height 480, Flags:
Selection Video Output: compose_default, Left 0, Top 0, Width 640, Height 480, Flags:
Selection Video Output: compose_bounds, Left 0, Top 0, Width 640, Height 480, Flags:
Streaming Parameters Video Capture:
Capabilities : timeperframe
Frames per second: 30.000 (30/1)
Read buffers : 0

User Controls

horizontal_flip 0x00980914 (bool) : default=0 value=0
vertical_flip 0x00980915 (bool) : default=0 value=0
alpha_component 0x00980929 (int) : min=0 max=255 step=1 default=0 value=0

 
Thanks & Regards 
Rajesh 

[Dhruvit]

Hi @Rajesh_,

I hope you are doing well.

We are working on it. I will let you know once we find a solution.

Thanks & Regards,
Dhruvit Vasavada

[Rajesh_]

Hi @Dhruvit ,

we have tried to debug more at our end and found some interesting stuff.

I have build yocto for imx6 sabresd board with same yocto src which can build for imx8. So, It will use the same chromium version 101.

I have tested with https://webrtc.github.io/samples/ in imx6 and it's work well as MIPI CSI camera got detected properly.

So, Now I feel this could be the issue for mxc-isi-cap core driver as we use mxc_v4l_capture core drivers in imx6. 

More detail from IMX6:

Here you can see "Video input" is available which is not present in imx8.

root@imx6qsabresd:~/.config/chromium# v4l2-ctl --all -d /dev/video0
Driver Info:
Driver name : mxc_v4l2
Card type :
Bus info :
Driver version : 0.1.11
Capabilities : 0x05000005
Video Capture
Video Overlay
Read/Write
Streaming
Video input : 0 (CSI IC MEM: no power)
Video output: 0 (DISP3 BG)
Video Standard = 0x00000000

I think this is the actual problem, Please help us to solve this asap.

Please, Let me know if more details needed for this

Thanks,

Rajesh

[Dhruvit]

Hi @Rajesh_,

After some search in the Chromium source code, it seems it indeed doesn't support "V4L2_CAP_VIDEO_CAPTURE_MPLANE". You can search it in the code and will not find any match.

https://chromium.googlesource.com/chromium/src/+/refs/tags/101.0.4951.54/media/capture/video/linux/v...

Also, it seems previously someone tried to add "V4L2_CAP_VIDEO_CAPTURE_MPLANE" support in Chromium, but didn't get merged into the mainline.

https://chromium-review.googlesource.com/c/chromium/src/+/742845

Therefore, I think this issue is not related to our driver but Chromium. Please check with the Chromium community for more details.

Thanks & Regards,
Dhruvit Vasavada

[Rajesh_]

Hi @Dhruvit ,

Thanks for the update but this looks major drawbacks as user can't use MIPI CSI camera which is connected in imx8 NXP platform. 

As I also seen that CAPTURE_MPLANE is only in imx8 platform but not in imx6 based platform so, this is working their. 

Is it not possible to make imx8 mxc-isi driver as a SINGLE plane or add the both SINGLE & MPLANE support to work chromium?

Please help us to suggest the possibilities as we are stuck with this problem.

Thanks,

Rajesh

[Dhruvit]

Hi @Rajesh_,

I've checked with the driver owner, the IP on i.MX8MP is quite different from the one on i.MX6, and thus the driver is quite different. Comparing with i.MX6 is not very meaningful in this case.

On i.MX8MP, the MIPI CSI driver only support the "Video Capture Multiplanar" API. Usually it's the application that needs to follow the API that the device support, not the other way around. And from what I shared in the previous discussion, I think chromium has the ablility to support "Video Capture Multiplanar" API. Therefore, I think ask chromium community to add "Video Capture Multiplanar" support in chromium is the more reasonable solution.

Thanks & Regards,
Dhruvit Vasavada

[Rajesh_]

Hi @Dhruvit ,

Is there any update on this bug resolution?

Thanks and Regards
Rajesh Mallempati

[simon_ko]

Dear Dhruvit.

Regarding this case, we also have the same problem and need to resolve it.
What is causing this problem?
Can you give me a schedule for when it will be resolved?

Thanks.