imx8mp - clear HDR support for imx662

[jp4]

Hello experts,

Currently I am enabling HDR for imx662 in imx8mp.

Sensor IMX662 : This sensor support clear HDR mode, in which two frames(high gain and low gain) will be interleaved without any compression as shown below.

I have updated all the necessary register to support clear HDR in sensor side, I have enabled virtual channel mode in clearHDR so that vc-0 will be containing high gain frame, and vc-1 will be containing low gain frame.

Platform IMX8MP:
1. I have updated vvcam structure to match with sensor output as below
.hdr_mode = SENSOR_MODE_HDR_STITCH,
.stitching_mode = SENSOR_STITCHING_DUAL_DCG_NOWAIT,

2. As per the AN1020420-AN13326 ISP HDR Modes and Sensor Types(2.0).pdf, I have updated drivers/staging/media/imx/imx8-mipi-csi2-sam.c file.

Issue: After above mentioned changes, I have started stream, I could not see any output without any error logs.

Observations: 
1. There is no error reported in mxc-mipi-csi2.0 below is the screenshot of status registers.

2. There is no frame entered in vsiisp, below is the screenshot of /proc/vsisisp0 file.

3. I have enabled debug prints in isp-imx source, During HDR stream the below mentioned poll is always getting timeout. There is no POLLIN is happening. 

(units/fpgs/fpga/source/altera_fpga.c)

 4. below are the dumped registers for isp_imsc, isp_ris, isp_mis, isp_icr, isp_isr from vvcam/isp/mrv_all_regs.h 

 5. I have attached dmesg logs.

Kindly let me, is there anything else i should take care?

Thanks in advance.

[jp4]

dmesg :
3815.388377] mxc-mipi-csi2.0: mipi_csis_imx8mp_phy_reset: bus fmt is 12 bit !
[ 3815.388438] mxc-mipi-csi2.0: fmt: 0x3012, 1920 x 1080
[ 3815.388663] imx662 2-0042: entering th func:imx662_enum_mbus_code
[ 3815.394854] mxc-mipi-csi2.0: mipi_csis_imx8mp_phy_reset: bus fmt is 12 bit !
[ 3815.394901] mxc-mipi-csi2.0: csis_s_hdr: 1
[ 3815.395041] enter isp_s_input
[ 3815.398078] enter isp_s_input
[ 3815.401127] enter isp_s_input
[ 3815.404126] enter isp_s_input
[ 3815.407129] enter isp_s_mcm
[ 3815.409934] enter isp_s_demosaic
[ 3815.413183] enter isp_enable_gamma_out
[ 3815.416957] enter isp_enable_wb
[ 3815.420116] enter isp_s_hist
[ 3815.423003] enter isp_s_hist
[ 3815.425901] enter isp_s_exp
[ 3815.428721] enter isp_s_exp
[ 3815.431535] enter isp_s_exp
[ 3815.434338] enter isp_disable_awb
[ 3815.437675] enter isp_enable_awb
[ 3815.440928] enter isp_s_afm
[ 3815.443743] enter isp_s_comp
[ 3815.446636] enter isp_s_comp
[ 3815.449607] enter isp_s_comp
[ 3815.452657] enter isp_s_cac
[ 3815.455570] enter isp_s_cac
[ 3815.458393] enter isp_s_cac
[ 3815.461245] enter isp_disable_lsc
[ 3815.464608] enter isp_enable_lsc
[ 3815.468223] imx662 2-0042: entering th func:imx662_enum_mbus_code
[ 3815.474377] mxc-mipi-csi2.0: mipi_csis_imx8mp_phy_reset: bus fmt is 12 bit !
[ 3815.474393] mxc-mipi-csi2.0: mipi_csis_s_stream: 1, state: 0x0
[ 3815.474426] mxc-mipi-csi2.0: fmt: 0x3012, 1920 x 1080
[ 3815.487110] mxc-mipi-csi2.0: --- mipi_csis_s_stream ---
[ 3815.487127] mxc-mipi-csi2.0: CSIS_VERSION[0]: 0x03060301
[ 3815.487134] mxc-mipi-csi2.0: CSIS_CMN_CTRL[4]: 0x00004f05
[ 3815.487139] mxc-mipi-csi2.0: CSIS_CLK_CTRL[8]: 0x000f0000
[ 3815.487144] mxc-mipi-csi2.0: CSIS_INTMSK[10]: 0x0fffff1f
[ 3815.487149] mxc-mipi-csi2.0: CSIS_INTSRC[14]: 0x00000000
[ 3815.487154] mxc-mipi-csi2.0: CSIS_DPHYSTATUS[20]: 0x00000022
[ 3815.487159] mxc-mipi-csi2.0: CSIS_DPHYCTRL[24]: 0x1480001f
[ 3815.487165] mxc-mipi-csi2.0: CSIS_DPHYBCTRL_L[30]: 0x000001f4
[ 3815.487170] mxc-mipi-csi2.0: CSIS_DPHYBCTRL_H[34]: 0x00000000
[ 3815.487175] mxc-mipi-csi2.0: CSIS_DPHYSCTRL_L[38]: 0x00000000
[ 3815.487180] mxc-mipi-csi2.0: CSIS_DPHYSCTRL_H[3c]: 0x00000000
[ 3815.487185] mxc-mipi-csi2.0: CSIS_ISPCONFIG_CH0[40]: 0x000000b0
[ 3815.487190] mxc-mipi-csi2.0: CSIS_ISPCONFIG_CH1[50]: 0x000000b1
[ 3815.487195] mxc-mipi-csi2.0: CSIS_ISPCONFIG_CH2[60]: 0x000000b2
[ 3815.487201] mxc-mipi-csi2.0: CSIS_ISPCONFIG_CH3[70]: 0x000000b3
[ 3815.487206] mxc-mipi-csi2.0: CSIS_ISPRESOL_CH0[44]: 0x04380780
[ 3815.487211] mxc-mipi-csi2.0: CSIS_ISPRESOL_CH1[54]: 0x04380780
[ 3815.487216] mxc-mipi-csi2.0: CSIS_ISPRESOL_CH2[64]: 0x04380780
[ 3815.487221] mxc-mipi-csi2.0: CSIS_ISPRESOL_CH3[74]: 0x04380780
[ 3815.487226] mxc-mipi-csi2.0: CSIS_ISPSYNC_CH0[48]: 0x00000000
[ 3815.487231] mxc-mipi-csi2.0: CSIS_ISPSYNC_CH1[58]: 0x00000000
[ 3815.487236] mxc-mipi-csi2.0: CSIS_ISPSYNC_CH2[68]: 0x00000000
[ 3815.487241] mxc-mipi-csi2.0: CSIS_ISPSYNC_CH3[78]: 0x00000000
[ 3815.487246] mxc-mipi-csi2.0: --- mipi_csis_s_stream ---
[ 3815.487252] mxc-mipi-csi2.0: GPR_GASKET_0_CTRL[60]: 0x00000000
[ 3815.487257] mxc-mipi-csi2.0: GPR_GASKET_0_HSIZE[64]: 0x00000000
[ 3815.487262] mxc-mipi-csi2.0: GPR_GASKET_0_VSIZE[68]: 0x00000000
[ 3815.487296] enter isp_s_input
[ 3815.490295] enter isp_s_mux
[ 3815.493939] imx662 2-0042: entering th func:imx662_priv_ioctl
[ 3815.499738] imx662 2-0042: In imx662_priv_ioctl, Received IOCTL VVSENSORIOC_S_EXP
[ 3815.507252] imx662 2-0042: entering th func:imx662_s_exposure
[ 3815.513022] imx662 2-0042: mcu_set_ctrl called in imx662_s_exposure, exposure line = 2250 exposure time = 19998000 ns
[ 3815.523692] imx662 2-0042: entering th func:mcu_set_ctrl
[ 3815.530428] imx662 2-0042: entering th func:mcu_get_cmd_status
[ 3815.556653] imx662 2-0042: entering th func:mcu_get_cmd_status
[ 3815.580825] enter isp_disable_lsc
[ 3815.584191] enter isp_enable_lsc
[ 3815.587640] enter isp_s_dpcc
[ 3815.591045] enter isp_enable_lsc
[ 3815.594581] enter wdr3_hw_init
[ 3815.597920] wdr3 res: 1920 1080
[ 3815.601408] enter isp_s_demosaic
[ 3815.604865] enter isp_s_demosaic
[ 3815.608350] enter isp_s_cac
[ 3815.611288] enter isp_s_cnr
[ 3815.614101] enter isp_s_cnr
[ 3815.616930] enter isp_s_cnr
[ 3815.619749] enter isp_s_cnr
[ 3815.622674] enter isp_enable_gamma_out
[ 3815.626557] enter isp_s_cc
[ 3815.629403] enter isp_s_cc
[ 3815.672779] enter isp_set_stream 1
[ 3815.686941] enter isp_s_hist
[ 3815.689932] enter isp_s_hist
[ 3815.692989] enter isp_s_exp
[ 3815.695826] enter isp_s_exp
[ 3815.698742] enter isp_enable_awb
[ 3815.702029] enter isp_s_afm
[ 3815.705318] enter isp_mi_start
[ 3815.709168] enter isp_start_stream
[ 3815.712935] imx662 2-0042: entering th func:imx662_priv_ioctl
[ 3815.718741] imx662 2-0042: In imx662_priv_ioctl, Received IOCTL VVSENSORIOC_S_STREAM
[ 3815.726578] imx662 2-0042: entering th func:imx662_s_stream
[ 3815.732210] imx662 2-0042: entering th func:imx662_start_streaming
[ 3815.738427] imx662 2-0042: entering th func:mcu_cam_stream_on
[ 3815.827128] imx662 2-0042: entering th func:mcu_get_cmd_status
[ 3815.879135] imx662 2-0042: entering th func:mcu_get_cmd_status
[ 3815.931150] imx662 2-0042: entering th func:mcu_get_cmd_status
[ 3815.943419] mxc-mipi-csi2.0: Frame Start: 1
[ 3815.943439] mxc-mipi-csi2.0: status: 01000000
[ 3815.943447] mxc-mipi-csi2.0: status: 02000000
[ 3815.943455] mxc-mipi-csi2.0: status: 00000008
[ 3815.943465] mxc-mipi-csi2.0: status: 00000008
[ 3815.943475] mxc-mipi-csi2.0: status: 00000008
[ 3815.943486] mxc-mipi-csi2.0: status: 00000008
[ 3815.943496] mxc-mipi-csi2.0: status: 00000008
[ 3815.943513] mxc-mipi-csi2.0: status: 00000008
[ 3815.943527] mxc-mipi-csi2.0: status: 00000008
[ 3815.943540] mxc-mipi-csi2.0: status: 00000008
[ 3815.943548] mxc-mipi-csi2.0: status: 00000008
[ 3815.943559] mxc-mipi-csi2.0: status: 00000008
[ 3815.943569] mxc-mipi-csi2.0: status: 00000008
[ 3815.943579] mxc-mipi-csi2.0: status: 00000008
[ 3815.943596] mxc-mipi-csi2.0: status: 00000008
[ 3815.943610] mxc-mipi-csi2.0: status: 00000008
[ 3815.943620] mxc-mipi-csi2.0: status: 00000008
[ 3815.943631] mxc-mipi-csi2.0: status: 00000008
[ 3815.943641] mxc-mipi-csi2.0: status: 00000008
[ 3815.943652] mxc-mipi-csi2.0: status: 00000008
[ 3815.943664] mxc-mipi-csi2.0: status: 00000008
[ 3815.943672] mxc-mipi-csi2.0: status: 00000008
[ 3815.943685] mxc-mipi-csi2.0: status: 00000008
[ 3815.943693] mxc-mipi-csi2.0: status: 00000008
[ 3815.943703] mxc-mipi-csi2.0: status: 00000008
[ 3815.943717] mxc-mipi-csi2.0: status: 00000008
[ 3815.943725] mxc-mipi-csi2.0: status: 00000008
[ 3815.943735] mxc-mipi-csi2.0: status: 00000008
[ 3815.943745] mxc-mipi-csi2.0: status: 00000008
[ 3815.943758] mxc-mipi-csi2.0: status: 00000008
[ 3815.943767] mxc-mipi-csi2.0: status: 00000008
[ 3815.943776] mxc-mipi-csi2.0: status: 00000008
[ 3815.943787] mxc-mipi-csi2.0: status: 00000008
[ 3815.943800] mxc-mipi-csi2.0: status: 00000008
[ 3815.976726] mxc-mipi-csi2.0: Frame End: 1
[ 3815.976748] mxc-mipi-csi2.0: status: 00300000
[ 3815.976756] mxc-mipi-csi2.0: Frame Start: 2
[ 3815.976762] mxc-mipi-csi2.0: status: 03000000
[ 3815.976789] mxc-mipi-csi2.0: status: 00000008
[ 3815.976799] mxc-mipi-csi2.0: status: 00000008
[ 3815.976809] mxc-mipi-csi2.0: status: 00000008
[ 3815.976820] mxc-mipi-csi2.0: status: 00000008
[ 3815.976830] mxc-mipi-csi2.0: status: 00000008
[ 3815.976840] mxc-mipi-csi2.0: status: 00000008
[ 3815.976851] mxc-mipi-csi2.0: status: 00000008
[ 3815.976861] mxc-mipi-csi2.0: status: 00000008
[ 3815.976872] mxc-mipi-csi2.0: status: 00000008
[ 3815.976882] mxc-mipi-csi2.0: status: 00000008
[ 3815.976892] mxc-mipi-csi2.0: status: 00000008
[ 3815.976903] mxc-mipi-csi2.0: status: 00000008
[ 3815.976913] mxc-mipi-csi2.0: status: 00000008
[ 3815.976923] mxc-mipi-csi2.0: status: 00000008
[ 3815.976934] mxc-mipi-csi2.0: status: 00000008
[ 3815.976944] mxc-mipi-csi2.0: status: 00000008
[ 3815.976955] mxc-mipi-csi2.0: status: 00000008
[ 3815.976965] mxc-mipi-csi2.0: status: 00000008
[ 3815.976975] mxc-mipi-csi2.0: status: 00000008
[ 3815.976986] mxc-mipi-csi2.0: status: 00000008
[ 3815.976996] mxc-mipi-csi2.0: status: 00000008
[ 3815.977007] mxc-mipi-csi2.0: status: 00000008
[ 3815.977017] mxc-mipi-csi2.0: status: 00000008
[ 3815.977037] mxc-mipi-csi2.0: status: 00000008
[ 3815.977048] mxc-mipi-csi2.0: status: 00000008
[ 3815.977058] mxc-mipi-csi2.0: status: 00000008
[ 3815.977069] mxc-mipi-csi2.0: status: 00000008
[ 3815.977080] mxc-mipi-csi2.0: status: 00000008
[ 3815.977089] mxc-mipi-csi2.0: status: 00000008
[ 3815.977100] mxc-mipi-csi2.0: status: 00000008
[ 3815.977113] mxc-mipi-csi2.0: status: 00000008
[ 3815.977121] mxc-mipi-csi2.0: status: 00000008
[ 3815.977133] mxc-mipi-csi2.0: status: 00000008
[ 3815.987197] imx662 2-0042: entering th func:mcu_get_cmd_status
[ 3816.010060] mxc-mipi-csi2.0: Frame End: 2
[ 3816.010082] mxc-mipi-csi2.0: status: 00300000
[ 3816.010091] mxc-mipi-csi2.0: Frame Start: 3
[ 3816.010097] mxc-mipi-csi2.0: status: 03000000
[ 3816.010125] mxc-mipi-csi2.0: status: 00000008
[ 3816.010133] mxc-mipi-csi2.0: status: 00000008
[ 3816.010143] mxc-mipi-csi2.0: status: 00000008
[ 3816.010153] mxc-mipi-csi2.0: status: 00000008
[ 3816.010170] mxc-mipi-csi2.0: status: 00000008
[ 3816.010184] mxc-mipi-csi2.0: status: 00000008
//////////////////////////////////////////////////
[ 3817.576778] mxc-mipi-csi2.0: Frame Start: 50
[ 3817.576784] mxc-mipi-csi2.0: status: 01000000
[ 3817.576791] mxc-mipi-csi2.0: status: 02000000
[ 3817.576814] mxc-mipi-csi2.0: status: 00000008
[ 3817.576824] mxc-mipi-csi2.0: status: 00000008
[ 3817.576834] mxc-mipi-csi2.0: status: 00000008
[ 3817.576845] mxc-mipi-csi2.0: status: 00000008
[ 3817.576855] mxc-mipi-csi2.0: status: 00000008
[ 3817.576865] mxc-mipi-csi2.0: status: 00000008
[ 3817.576876] mxc-mipi-csi2.0: status: 00000008
[ 3817.576886] mxc-mipi-csi2.0: status: 00000008
[ 3817.576897] mxc-mipi-csi2.0: status: 00000008
[ 3817.576907] mxc-mipi-csi2.0: status: 00000008
[ 3817.576917] mxc-mipi-csi2.0: status: 00000008
[ 3817.576928] mxc-mipi-csi2.0: status: 00000008
[ 3817.576938] mxc-mipi-csi2.0: status: 00000008
[ 3817.576948] mxc-mipi-csi2.0: status: 00000008
[ 3817.576959] mxc-mipi-csi2.0: status: 00000008
[ 3817.576969] mxc-mipi-csi2.0: status: 00000008
[ 3817.576980] mxc-mipi-csi2.0: status: 00000008
[ 3817.576990] mxc-mipi-csi2.0: status: 00000008
[ 3817.577000] mxc-mipi-csi2.0: status: 00000008
[ 3817.577011] mxc-mipi-csi2.0: status: 00000008
[ 3817.577021] mxc-mipi-csi2.0: status: 00000008
[ 3817.577032] mxc-mipi-csi2.0: status: 00000008
[ 3817.577042] mxc-mipi-csi2.0: status: 00000008
[ 3817.577052] mxc-mipi-csi2.0: status: 00000008
[ 3817.577063] mxc-mipi-csi2.0: status: 00000008
[ 3817.577073] mxc-mipi-csi2.0: status: 00000008
[ 3817.577083] mxc-mipi-csi2.0: status: 00000008
[ 3817.577094] mxc-mipi-csi2.0: status: 00000008
[ 3817.577104] mxc-mipi-csi2.0: status: 00000008
[ 3817.577115] mxc-mipi-csi2.0: status: 00000008
[ 3817.577125] mxc-mipi-csi2.0: status: 00000008
[ 3817.577135] mxc-mipi-csi2.0: status: 00000008
[ 3817.577146] mxc-mipi-csi2.0: status: 00000008
[ 3817.577156] mxc-mipi-csi2.0: status: 00000008
[ 3817.578091] mxc-mipi-csi2.0: Frame End: 50
[ 3817.578097] mxc-mipi-csi2.0: status: 00300000
[ 3817.603187] imx662 2-0042: entering th func:mcu_get_cmd_status
[ 3817.628757] imx662 2-0042: mcu_cam_stream_off 2807 MCU Stream Off Success !!
[ 3817.646275] enter isp_s_demosaic
[ 3817.649617] enter isp_disable_lsc
[ 3817.653206] enter isp_s_dpcc
[ 3817.656490] mxc-mipi-csi2.0: mipi_csis_s_stream: 0, state: 0x0
[ 3817.656541] mxc-mipi-csi2.0: Frame End events: 50
[ 3817.661259] mxc-mipi-csi2.0: Frame Start events: 50
[ 3817.666142] mxc-mipi-csi2.0: Non-image data after odd frame events: 0
[ 3817.672584] mxc-mipi-csi2.0: Non-image data before odd frame events: 0
[ 3817.679113] mxc-mipi-csi2.0: Non-image data after even frame events: 0
[ 3817.685642] mxc-mipi-csi2.0: Non-image data before even frame events: 0
[ 3817.692257] mxc-mipi-csi2.0: Unknown Error events: 0
[ 3817.697223] mxc-mipi-csi2.0: CRC Error events: 0
[ 3817.701841] mxc-mipi-csi2.0: ECC Error events: 0
[ 3817.706460] mxc-mipi-csi2.0: FIFO Overflow Error events: 0
[ 3817.711946] mxc-mipi-csi2.0: Lost Frame End Error events: 0
[ 3817.717520] mxc-mipi-csi2.0: Lost Frame Start Error events: 0
[ 3817.723267] mxc-mipi-csi2.0: SOT Error events: 0
[ 3817.728309] enter isp_s_vsm
[ 3817.731318] enter isp_s_afm
[ 3817.734172] enter isp_disable_awb
[ 3817.737510] enter isp_s_exp
[ 3817.740398] enter isp_s_hist
[ 3817.743881] enter isp_s_dpcc
[ 3817.791458] imx662 2-0042: enteri

[jp4]

I would like to add few more observations.
6. I have confirmed mipi data out from sensor whenever I am streaming on, by probing mipi lanes.
7. I have configured my sensor with clearHDR, but I randomly configured .hdr_mode = SENSOR_MODE_HDR_NATIVE/LINEAR(ideally that is not correct mode) still I could get frames. I could not get frames only when I keep HDR_STITCH mode.

[jp4]

I could see similar kind of issue already discussed in below post.
https://community.nxp.com/t5/i-MX-Processors/i-MX8MP-ISP-adding-a-new-Sony-1080p-sensor-with-ClearHD...

Kindly provide your thoughts @joanxie   @artsiomstaliaro 

[artsiomstaliaro]

As I understand you are able to get video with HDR?

[jp4]

No, I could not get any output if I keep HDR_STITCH mode.

[joanxie]

we haven't tested this mode, but as I known, imx8mp ISP can support it, could you confirm your output data from camera is “2x12-bit dual DCG without waiting" mode?  and did you test other mode? I had another customer who can work line mode and dual HDR mode on imx662, did you try other mode? only this mode doesn't work?

[jp4]

1. Yes, data out from sensor is “2x12-bit dual DCG without waiting" mode

2. I am not getting any stream for all stitiching mode (sensor_stitching_mode_e).

3. I have tried DOL as well (changed sensor and driver configuration to DOL). but I could not get any stream.

4. As a conclusion, I am not getting any stream when I select hdr_mode=HDR_STITCH.

[joanxie]

how about setting the mode as below?

.hdr_mode = SENSOR_MODE_HDR_STITCH,
.stitching_mode = SENSOR_STITCHING_L_AND_S,

[prabhu_k]

I have already mentioned that, I tried with all the modes in sensor_stitching_mode_e, including below, But I could not get any stream.

.hdr_mode = SENSOR_MODE_HDR_STITCH,
.stitching_mode = SENSOR_STITCHING_L_AND_S,

[joanxie]

sent to you a patch, pls check it, this isn't official version, just for reference, so couldn't share on the public community

[artsiomstaliaro]

@joanxie Can you share patch also with me please?

[jp4]

I did not receive, could you please sent to "jp4" user

[joanxie]

I send the mail to the jeyaprakash.ck@e-consystems.com, I couldn't send other mail address, if you need receive mail via other mail address, pls use that mail address to submit ticket here

https://www.nxp.com/support/support:SUPPORTHOME

[artsiomstaliaro]

Can you send me also, please. Thank you