Hi NXP Team,
We came across ISP in i.MX8MPlus, which is used in conjunction with camera. We found the device tree node of the same. However, can anyone help on how to use ISP, when to use and flow of ISP?
Thanks in advance !
Regards,
Kunal
Hi jgsandom,
Thank you. This is 100% clear.
Actually one does not really need the debug build of isp-imx since release already has all needed features when activating traces and exporting the logs with export ISP_LOG_LEVEL=8.
I am still stuck in making the ISP work though. I think its due to the fact that I let yocto build with older isp-imx-4.2.2.6.0 which I then partly overwrite with separately cross-compiled isp-imx-4.2.2.11.0 using:
scp -r build_output_release_Full/opt/imx8-isp target@target-ip:/opt
scp -r build_output_release_Full/usr/lib/* target@target-ip:/usr/lib/
I have the feeling that some version 6 related stuff is still to be overwritten besides the two lines above.
my run.sh script cannot properly run (open fails).
I think I need to incorporate isp-imx-4.2.2.11.0 to the yocto too.
I tried putting isp-imx_4.2.2.11.0.bb next to isp-imx_4.2.2.6.0.bb and select version 11 in local.conf like this:
IMAGE_INSTALL_append = " \
kernel-module-isp-vvcam \
isp-imx \
"
PREFERRED_VERSION_isp-imx = "4.2.2.11.0"
PREFERRED_VERSION_kernel-module-isp-vvcam = "4.2.2.11.0"
I also tried to setup separate meta-layers but it did not help either.
I have hard time with this step and keep on getting license mismatch errors.
How did you manage to update vvcam and isp user space code from 6 to 11? I am unfortunatelly not a yocto expert.
Thank you,
Thank you so much jgsandom,
This is exactly the type of info that would help moving on with the ISP.
NXP: Please provide a one page tutorial on how to integrate the ISP with custom Sensors not using Basler at all.
Thank you
we have released the user guide for sensor porting:
Thank you so much jgsandom.
I am going to investigate your suggestions. This is highly appreciated since discovering all this without help is extremly tedious.
Hi Joanxie,
isp-imx-4.2.2.6.0.bin seems indeed to contain all kernel + user space resources needed for ISP.
I would like to buy an EVK + Camera to demonstrate exactly this. What EVK+Camera should I use. I would like to start my own ISP development from a working system.
The Basler BCON cam (official cam of the NXP imx8mp evk) is not using isp-imx-4.2.2.6.0.bin because Basler uses their own user space software (Pylon).
Thank you
we bought this for imx8mp, you can refer to it
Hi @joanxie,
Thank you for your response.
Where would I find the document of the bin/source file you shared? Also, where in yocto this source code will be?
We are planning to use Sony IMX219 camera sensor which is RAW sensor. Can you please help what changes are required to use ISP with some example?
Regards,
Kunal
Hi @kunal_003
Is it possible to know which all are the basic registers that need to be configured for IMX219 sensor?
I have configured register sets which are mentioned in the open source IMX219 driver.(https://github.com/torvalds/linux/blob/master/drivers/media/i2c/imx219.c) .
How can we verify whether all the registers are configured properly. When i tried with i2cdump getting the following output/
root@imx8mpevk:~# i2cdump -f -y 2 0x10 w
0,8 1,9 2,a 3,b 4,c 5,d 6,e 7,f
00: 0000 0000 0000 0004 0000 0000 0000 0000
08: 0000 0000 0000 ff03 0000 0000 0000 0000
10: 0000 0000 0000 0077 0000 0000 0000 0000
18: 0000 0000 0000 0000 0000 0000 0000 0000
20: 0000 0000 0000 0000 0000 0000 0000 0000
28: 0000 0000 0000 0000 0000 0000 0000 0000
30: 0000 0000 0000 02a8 0000 ec7b 7558 0000
38: 0000 0000 0000 0000 0000 0000 0000 0000
40: 0000 0000 0000 00ff 0000 0000 0000 0000
48: 0000 0000 0000 0000 0000 0000 0000 0000
50: 0000 0000 0000 0f00 0000 0000 0000 0000
58: 0000 0000 0000 0000 0000 0000 0000 0000
60: 0000 0000 0000 0000 0000 0000 0000 0000
68: 0000 0000 0000 0000 0000 0000 0000 0000
70: 0000 0000 0000 0000 0000 aa1b bb2d 0000
78: 0000 0000 0000 0000 0000 0000 0000 0100
80: 0000 0000 0000 0000 0000 0000 0000 0000
88: 0000 0000 0000 0000 0000 0000 0000 0300
90: 0008 0000 0000 0000 0000 0000 0000 0000
98: 0000 0000 0000 0000 0000 0000 0000 0000
a0: 0000 0000 0000 0000 0000 0000 0000 0000
a8: 0000 0000 0000 0000 0000 0000 0000 0000
b0: 0000 0000 0000 0000 0000 06ee 7b4e 0000
b8: 0000 0000 0000 0000 0000 0000 0000 0000
c0: 0000 0000 0000 019f 0000 0000 0000 0000
c8: 0000 0000 0000 0000 0000 0000 0000 0300
d0: 0000 0000 0000 0000 0000 0000 0000 0000
d8: 0000 0000 0000 0000 0000 0000 0000 0000
e0: 0000 0000 0000 0000 0000 0000 0000 0000
e8: 0000 0000 0000 0000 0000 0000 0000 0000
f0: 0000 0000 0000 0001 a38b b769 89a0 0000
f8: 0000 0000 0000 0000 0000 0000 0000 0000
root@imx8mpevk:~#
It is not matching with the values in the driver code. Is there any other ways to verify this.
Can you please help here.
Thanks in advance
Best regards
Charlie.
Hi @kunal_003
I'm also using IMX219 sensor with IMX8MP. But facing some issues during integration.
Is your porting IMX219 with IMX8MP is completely successful?
Regards
Charles
Hi @Amal_Antony3331,
Can you share specific details on the issues you are facing with the integration?
Thanks,
Joe
Hi @jgsandom
I'm taking the OV2775 sensor reference for porting the IMX219.
Now able to get the video nodes on v4l2-ctl command.
I'm using the following command to get the stream and attaching the output log.
root@imx8mpevk:/opt/imx8-isp/bin# ./video_test -w 1920 -h 1080 -f RAW10 -t file -m0 -d2
[ 506.302106] Enter mipi_csis_clk_enable ...
[ 506.306609] Enter mipi_csis_clk_enable ...
[ 506.311173]
[ 506.311173] enter imx219_priv_ioctl ((( (cmd = -2140645888))))
[ 506.318604] Command : VIDIOC_QUERYCAP
[ 506.322901] enter (( imx219_ioc_qcap ))
[ 506.327076] enter dwe_runtime_resume ....
[ 506.331451] enter dwe_enable_clocks ....
[ 506.335747] enter dwe_open ....
[ 506.349623] enter dwe_close ....
[ 506.365599] enter dwe_runtime_suspend ....
[ 506.378250] Enter mipi_csis_clk_enable ...
[ 506.382773] Enter mipi_csis_clk_enable ...
[ 506.387275]
[ 506.387275] enter imx219_priv_ioctl ((( (cmd = -2140645888))))
[ 506.394742] Command : VIDIOC_QUERYCAP
[ 506.399052] enter (( imx219_ioc_qcap ))
[ 506.403154] Enter mipi_csis_clk_enable ...
loading [IMX219.drv]...
[ 506.427968] Enter mipi_csis_clk_enable ...
[ 506.434057] Enter mipi_csis_clk_enable ...
[ 506.438550]
[ 506.438550] enter imx219_priv_ioctl ((( (cmd = -2140645888))))
[ 506.445963] Command : VIDIOC_QUERYCAP
[ 506.450243] enter (( imx219_ioc_qcap ))
[ 506.454310] enter dwe_runtime_resume ....
[ 506.458677] enter dwe_enable_clocks ....
[ 506.462967] enter dwe_open ....
IMX219_IsiQuerySensorSupportIss (enter)
WARN : [DEWARP_HW] dwe type /dev/v4l-subdev0 0
WARN : [DEWARP_HW] dwe type /dev/v4l-subdev1 1
WARN : [DEWARP_HW] dwe type /dev/v4l-subdev2 2
WARN : [DEWARP_HW] dwe type /dev/v4l-subdev3 3
loading [IMX219.drv]...
IMX219_IsiQuerySensorSupportIss (enter)
Video Test:caps supports:{
count = 1
{
index = 0
width = 1920
height = 1080
fps = 30
hdr_mode = 0
}
}
[ 506.516725] enter imx219_priv_ioctl ((( (cmd = 290))))
[ 506.525407] Command : VVSENSORIOC_G_SENSOR_MODE
[ 506.530563] enter (( imx219_g_mode ))
[ 506.534435] sensor->cur_mode.index = 0 pmode->index = 0 sensor->cur_mode.width =1920 sensor->cur_mode.height =1080
loading [IMX219.drv]...
IMX219_IsiQuerySensorSupportIss (enter)
loading [IMX219.drv]...
IMX219_IsiQuerySensorSupportIss (enter)
IMX219_IsiGetSensorIss ((enter))
IMX219_IsiGetSensorIss ((exit))
IMX219_IsiQuerySensorSupportIss (enter)
IMX219_IsiCreateSensorIss (enter)
IMX219_IsiCreateSensorIss (exit)
IMX219_IsiInitSensorIss (enter)
[ 506.584657] enter isp_mi_stop
IMX219_IsiInitSensorIss (exit)
IMX219_IsiGetSensorModeIss (enter)
IMX219_IsiGetCapsIss (enter)
Bus width == 10
IMX219_IsiGetCapsIss (exit)
isp input: 1920x1080 RAW10
isp output: 1920x1080 YUV422I
dewarp input: 1920x1080 YUV422I
dewarp output: 1920x1080 YUV422I
IMX219_IsiSetupSensorIss (enter)
IMX219_IsiSetupSensorIss IMX219 System-Reset executed
[ 506.691954]
[ 506.691954] enter imx219_priv_ioctl ((( (cmd = 283))))
[ 506.698675] Command : VVSENSORIOC_S_HDR_MODE
[ 506.703596] enter (( imx219_s_hdr ))
[ 506.707375] HDR Selected = 0
[ 506.710721]
[ 506.710721] enter imx219_priv_ioctl ((( (cmd = 294))))
[ 506.717444] Command : VVSENSORIOC_S_SENSOR_MODE
[ 506.722692] enter (( imx219_s_sensor_mode ))
[ 506.727150] Mode Selected = 0
[ 506.730309] enter (( imx219_set_fmt ))
[ 506.734241] enter (( imx219_write_reg_arry ))
[ 506.738779] enter imx219_write_reg_arry
IMX219_AecSetModeParameters: (enter)
IMX219_AecSetModeParameters: AecMaxIntegrationTime = 0.033298
IMX219_AecSetModeParameters: (exit)
IMX219_Private_SetFormat: 1920 1080 0
[ 506.805845] enter (( imx219_get_format_code ))
[ 506.810514] Bayer pattern selected == 2
IMX219_IsiSetupSensorIss (exit[ 506.814723]
[ 506.814723] enter imx219_priv_ioctl ((( (cmd = 264))))
[ 506.824382] Command : VVSENSORIOC_WRITE_REG
[ 506.829280] enter (( imx219_write_reg ))
[ 506.833388] Write reg : reg=601, val=3
IMX219_IsiActivateTestPatte[ 506.839040]
[ 506.839040] enter imx219_priv_ioctl ((( (cmd = 285))))
[ 506.847093] Command : VVSENSORIOC_S_FPS
[ 506.851561] enter (( imx219_s_fps ))
[ 506.855339] FPS SET = 30
IMX219_IsiRegisterWriteIss (enter)
[ 506.858425]
[ 506.858425] enter imx219_priv_ioctl ((( (cmd = 290))))
[ 506.867980] Command : VVSENSORIOC_G_SENSOR_MODE
[ 506.873133] enter (( imx219_g_mode ))
[ 506.876981] sensor->cur_mode.index = 0 pmode->index = 0 sensor->cur_mode.width =1920 sensor->cur_mode.height =1080
IMX219_IsiRe[ 506.887801]
[ 506.887801] enter imx219_priv_ioctl ((( (cmd = 286))))
[ 506.897368] Command : VVSENSORIOC_G_FPS
[ 506.901829] enter (( imx219_g_fps ))
[ 506.905586] GET FPS = 30
[ 506.908345] enter imx219_priv_ioctl ((( (cmd = 276))))
[ 506.915058] Command : VVSENSORIOC_S_EXP
[ 506.919515] enter (( imx219_s_exp ))
[ 506.923274] enter (( imx219_write_reg ))
[ 506.927383] Write reg : reg=15a, val=ff
[ 506.932849] (exit) result = 0
[ 506.932849] enter imx219_priv_ioctl ((( (cmd = 279))))
[ 506.941179] Command : VVSENSORIOC_S_GAIN
[ 506.945736] enter (( imx219_s_gain ))
[ 506.949581] enter (( imx219_write_reg ))
[ 506.953688] Write reg : reg=157, val=c8
IMX219_IsiActivateTestPattern[ 506.959259] .. Enter mipi_csis_clk_enable ...
[ 506.965207] Enter csis_s_fmt ...
[ 506.968727] Enter find_csis_format ...
[ 506.972751] Enter disp_mix_gasket_config ...
[ 506.977297] Enter mipi_csis_set_params ...
[ 506.981662] Enter __mipi_csis_set_format ...
[ 506.986201] fmt: 0x2008, 1920 x 1080
[ 506.990305] Enter mipi_csis_set_hsync_settle ...
[ 506.995247] Enter mipi_csis_clk_enable ...
[ 506.999780] enter isp_s_comp
[ 507.002682] enter isp_s_comp
[ 507.005564] enter isp_s_comp
[ 507.008514] Enter mipi_csis_clk_enable ...
[ 507.012949] Enter mipi_csis_s_stream ...
[ 507.017157]
[ 507.017157] Enable = 1 state = 0x0
[ 507.022137] Enter mipi_csis_sw_reset ...
[ 507.026354] Enter mipi_csis_start_stream ...
[ 507.030895] Enter disp_mix_gasket_config ...
[ 507.035440] Enter mipi_csis_set_params ...
[ 507.039810] Enter __mipi_csis_set_format ...
[ 507.044350] fmt: 0x2008, 1920 x 1080
[ 507.048460] Enter mipi_csis_set_hsync_settle ...
[ 507.053363] Enter mipi_csis_system_enable ...
[ 507.057997] Enter disp_mix_gasket_enable ...
[ 507.062543]
[ 507.062543] Enter mipi_csis_clean_irq ...
[ 507.068123]
[ 507.068123] Enter mipi_csis_enable_interrupts ...
IMX219_IsiSetSensorFpsIss (enter)
IMX219_IsiSetSensorFpsIss: set fps(0) out of range, correct to 0 (0, 0)
IMX219_IsiSetSensorFpsIss set sensor fps = 0
IMX219_IsiSetSensorFpsIss (exit)
IMX219_IsiGetSensorModeIss (enter)
[ 507.094680] enter wdr3_hw_init
IMX219_IsiGetGainLimitsIss (enter)
[ 507.098550] wdr3 res: 1920 1080
Min Gain = 1.000000 Max Gain = 21.000000
[ 507.105342] enter isp_mi_start
[ 507.111282]
[ 507.111282] enter imx219_priv_ioctl ((( (cmd = 274))))
[ 507.120137] >>> Command : VVSENSORIOC_S_STREAM
IMX219_IsiGetGainLimitsIss (exit)
[ 507.124865] enter (( imx219_s_stream )) control value = 1
IMX219_IsiGetIntegrationTimeLimitsIss (enter)
[ 507.133292] enter (( imx219_write_reg ))
[ 507.140257] Write reg : reg=100, val=1
Min INtegration Time = 0.000030 Max Int time = 0.033298
IMX219_IsiGetIntegrationTimeLimitsIss (exit)
IMX219_IsiGetSensorFpsIss (enter)
IMX219_IsiGetSensorFpsIss (exit)
IMX219_IsiExposureControlIss (enter)
IMX219_IsiExposureControlIss: g=1.000000, Ti=0.000030
==>> HDR Mode = 0
IMX219_IsiSetIntegrationTimeIss (enter)
IMX219_IsiSetIntegrationTimeIss: set AEC_PK_EXPO=0x00001
IMX219_IsiSetIntegrationTimeIss Ti=0.000030
IMX219_IsiExposureControlIss: set: NewGain=1.000000, hcgTi=0.000030, hcgskip=1
IMX219_IsiSetGainIss (enter)
IMX219_IsiSetGainIss g=1.000000
IMX219_IsiExposureControlIss: set: NewGain=1.000000, hcgTi=0.000030, hcgskip=1
IMX219_IsiExposureControlIss: set: vsg=1.000000, vsTi=0.000030, vsskip=1
IMX219_IsiExposureControlIss (exit)
#### CsiSetFormat 1920 1080 808534599
IMX219_IsiGetSensorModeIss (enter)
IMX219_IsiGetSensorModeIss (enter)
CamEngineEnableCproc: set val 1 1 1 1.10 -15 1.00 0.00
IMX219_IsiSensorSetStreamingIss (enter)
IMX219_IsiSensorSetStreamingIss (exit)
As I can see, till Start Streaming API called and proper value is written to the register.
But there is no streaming.
./video_test -w 1920 -h 1080 -f RAW10 -t file -m0 -d2 is the command used.
I'm using the Ov2775 xml file for IMX219. Will this cause an issue?
Also can you point out the area where i can look into?
Thanks and Regards
Charles
this is internal material, I couldn't share here(public community), but I copy the steps here
•Sample code to port new camera Module.
Hi @joanxie,
I saw the link(isp-vvacm) provided by you, however I still have below query/confusion. Please help me in this case:
Regards,
Kunal
current bsp doesn't support SONY IMX219 directly, you need porting by yourself, current bsp includes ISP source code directly, you can find them under path after you building the bsp
"imx-yocto-bsp/build-xwayland/tmp/work/imx8mpevk-poky-linux/kernel-module-isp-vvcam/4.2.2.6.1-r0/git/vvcam"
you also can find the source code from:
"https://source.codeaurora.org/external/imx/isp-vvcam/tree/vvcam?h=imx_5.4.70_2.3.0"