The image parsed by mipi raw12 data will be divided into 4 similar parts

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

The image parsed by mipi raw12 data will be divided into 4 similar parts

2,521 Views
pan_
Contributor I

We used linux5.4.3 environment on imx8m mini soc to collect the data of TOF chip in RAW12 format.  Now we have successfully obtained the data and parsed it into grayscale images, but the displayed images are abnormal. The whole picture is divided into four parts, and each part shows similar pictures. See the attachment for specific effects. 

In order to achieve the imx8m mini support for RAW12 I have modified the "Linux/drivers/media/platform/MXC/capture/mx6s_capture.c, Linux/drivers/media/platform/MXC/capture/mxc_mipi_csi.c "two files and add tof camera driver, modify specific content see attachment.  At the same time, using the test mode of tof camera, we found that soc processed the data returned by tof, that is, soc moved the 12 bits of data returned by tof 2 bits to the left, and then took the lower 8 bits of data, and the higher 8 bits of data would be equal to the lower 8 bits of data. such as tof: 0x266 -->soc:0x9999. May I ask how I should deal with this problem? Looking forward to your reply. 

Labels (1)
0 Kudos
Reply
12 Replies

2,500 Views
joanxie
NXP TechSupport
NXP TechSupport

imx8mm can capture raw data, but couldn't handle it directly, how did  you transfer raw data to the display?

0 Kudos
Reply

2,497 Views
pan_
Contributor I
I used the V4l2 framework, then added the RAW12 format to the relevant driver file, and finally read the data away at the application layer.  Since the raw data we are using the tof chip is RAW12, we can convert it into a grayscale image at the application level based on this data. 
0 Kudos
Reply

2,464 Views
joanxie
NXP TechSupport
NXP TechSupport

is it the interlaced data from your camera? could you dump the mipi csi and csi registers with me? and send the logfile to me when you test the camera

0 Kudos
Reply

2,452 Views
pan_
Contributor I

I solved the problem of the screen being divided into 4 parts, but now we have found a new problem, normally raw12 storage takes 2 bytes and the high 4 bits are 0, but the data [12:15] we get at the application layer is not 0 bit and the data is changing.  I checked the csi RXFIFO content and found that he was correct. 

  • csi configuration

  • [ 154.875734] addr=0x32e20000 : 0x011b0903
    [ 154.875737] addr=0x32e20004 : 0xc0000000
    [ 154.875740] addr=0x32e20008 : 0x000010a0
    [ 154.875744] addr=0x32e2000c : 0x00000000
    [ 154.875746] addr=0x32e20010 : 0x00000000
    [ 154.875749] addr=0x32e20014 : 0x00009600
    [ 154.875752] addr=0x32e20018 : 0x80004000
    [ 154.875756] addr=0x32e2001c : 0x00000000
    [ 154.875758] addr=0x32e20020 : 0x00000000
    [ 154.875761] addr=0x32e20024 : 0x00000000
    [ 154.875764] addr=0x32e20028 : 0x80100000
    [ 154.875768] addr=0x32e2002c : 0x80200000
    [ 154.875770] addr=0x32e20030 : 0x00000000
    [ 154.875773] addr=0x32e20034 : 0x028001e0
    [ 154.875776] addr=0x32e20038 : 0x00000270
    [ 154.875780] addr=0x32e2003c : 0x00000000
    [ 154.875782] addr=0x32e20040 : 0x00000000
    [ 154.875785] addr=0x32e20044 : 0x00000000
    [ 154.875789] addr=0x32e20048 : 0xd84ad000
  • mipi csi configuration:
  • [ 154.737558] mxc_mipi-csi.0: addr=0x32e30004 : 0x00004105
    [ 154.737560] mxc_mipi-csi.0: addr=0x32e30008 : 0x000f0000
    [ 154.737563] mxc_mipi-csi.0: addr=0x32e3000c : 0xdeadcafe
    [ 154.737565] mxc_mipi-csi.0: addr=0x32e30010 : 0xffffffff
    [ 154.737568] mxc_mipi-csi.0: addr=0x32e30014 : 0x00000000
    [ 154.737570] mxc_mipi-csi.0: addr=0x32e30018 : 0x00000000
    [ 154.737573] mxc_mipi-csi.0: addr=0x32e3001c : 0x00000000
    [ 154.737576] mxc_mipi-csi.0: addr=0x32e30020 : 0x000000c0
    [ 154.737578] mxc_mipi-csi.0: addr=0x32e30024 : 0x1f800007
    [ 154.737581] mxc_mipi-csi.0: addr=0x32e30028 : 0xdeadcafe
    [ 154.737588] mxc_mipi-csi.0: addr=0x32e3002c : 0xdeadcafe
    [ 154.737590] mxc_mipi-csi.0: addr=0x32e30030 : 0x000001f4
    [ 154.737593] mxc_mipi-csi.0: addr=0x32e30034 : 0x00000000
    [ 154.737596] mxc_mipi-csi.0: addr=0x32e30038 : 0x00000000
    [ 154.737598] mxc_mipi-csi.0: addr=0x32e3003c : 0x00000000
    [ 154.737601] mxc_mipi-csi.0: addr=0x32e30040 : 0x000000b0
    [ 154.737603] mxc_mipi-csi.0: addr=0x32e30044 : 0x01e00280
    [ 154.737606] mxc_mipi-csi.0: addr=0x32e30048 : 0x00000000
    [ 154.737609] mxc_mipi-csi.0: addr=0x32e3004c : 0xdeadcafe
    [ 154.737611] mxc_mipi-csi.0: addr=0x32e30050 : 0x000008fd
    [ 154.737614] mxc_mipi-csi.0: addr=0x32e30054 : 0x80008000
    [ 154.737616] mxc_mipi-csi.0: addr=0x32e30058 : 0x00000000
    [ 154.737619] mxc_mipi-csi.0: addr=0x32e3005c : 0xdeadcafe
    [ 154.737622] mxc_mipi-csi.0: addr=0x32e30060 : 0x000008fe
    [ 154.737624] mxc_mipi-csi.0: addr=0x32e30064 : 0x80008000
    [ 154.737627] mxc_mipi-csi.0: addr=0x32e30068 : 0x00000000
    [ 154.737630] mxc_mipi-csi.0: addr=0x32e3006c : 0xdeadcafe
    [ 154.737632] mxc_mipi-csi.0: addr=0x32e30070 : 0x000008ff
    [ 154.737635] mxc_mipi-csi.0: addr=0x32e30074 : 0x80008000
    [ 154.737638] mxc_mipi-csi.0: addr=0x32e30078 : 0x00000000
    [ 154.737640] mxc_mipi-csi.0: addr=0x32e3007c : 0xdeadcafe
    [ 154.737643] mxc_mipi-csi.0: addr=0x32e30080 : 0x000000b0
    [ 154.737646] mxc_mipi-csi.0: addr=0x32e30084 : 0x01e00280
    [ 154.737648] mxc_mipi-csi.0: addr=0x32e30088 : 0x00000000
    [ 154.737651] mxc_mipi-csi.0: addr=0x32e3008c : 0xdeadcafe
    [ 154.737653] mxc_mipi-csi.0: addr=0x32e30090 : 0x00000000
    [ 154.737656] mxc_mipi-csi.0: addr=0x32e30094 : 0x00000000
    [ 154.737659] mxc_mipi-csi.0: addr=0x32e30098 : 0x00000000
    [ 154.737661] mxc_mipi-csi.0: addr=0x32e3009c : 0xdeadcafe
    [ 154.737664] mxc_mipi-csi.0: addr=0x32e300a0 : 0x00000000
    [ 154.737666] mxc_mipi-csi.0: addr=0x32e300a4 : 0x00000000
    [ 154.737669] mxc_mipi-csi.0: addr=0x32e300a8 : 0x00000000
    [ 154.737672] mxc_mipi-csi.0: addr=0x32e300ac : 0xdeadcafe
    [ 154.737674] mxc_mipi-csi.0: addr=0x32e300b0 : 0x00000000
    [ 154.737677] mxc_mipi-csi.0: addr=0x32e300b4 : 0x00000000
    [ 154.737679] mxc_mipi-csi.0: addr=0x32e300b8 : 0x00000000
    [ 154.737682] mxc_mipi-csi.0: addr=0x32e300bc : 0xdeadcafe
    [ 154.737686] mxc_mipi-csi.0: addr=0x32e300c0 : 0x00000000
    [ 154.737689] mxc_mipi-csi.0: addr=0x32e300c4 : 0x00000000
    [ 154.737691] mxc_mipi-csi.0: addr=0x32e300c8 : 0x40000001
    [ 154.737694] mxc_mipi-csi.0: addr=0x32e300cc : 0x000000e4
    [ 154.737697] mxc_mipi-csi.0: addr=0x32e300d0 : 0x00000000
    [ 154.737699] mxc_mipi-csi.0: addr=0x32e300d4 : 0x00000000
    [ 154.737702] mxc_mipi-csi.0: addr=0x32e300d8 : 0x00000000
    [ 154.737704] mxc_mipi-csi.0: addr=0x32e300dc : 0xdeadcafe
    [ 154.737707] mxc_mipi-csi.0: addr=0x32e300e0 : 0xdeadcafe
    [ 154.737710] mxc_mipi-csi.0: addr=0x32e300e4 : 0xdeadcafe
    [ 154.737712] mxc_mipi-csi.0: addr=0x32e300e8 : 0xdeadcafe
    [ 154.737715] mxc_mipi-csi.0: addr=0x32e300ec : 0xdeadcafe
    [ 154.737718] mxc_mipi-csi.0: addr=0x32e300f0 : 0xdeadcafe
    [ 154.737720] mxc_mipi-csi.0: addr=0x32e300f4 : 0xdeadcafe
    [ 154.737723] mxc_mipi-csi.0: addr=0x32e300f8 : 0xdeadcafe
    [ 154.737726] mxc_mipi-csi.0: addr=0x32e300fc : 0xdeadcafe
    [ 154.737728] mxc_mipi-csi.0: addr=0x32e30100 : 0x00000001
    [ 154.737731] mxc_mipi-csi.0: addr=0x32e30104 : 0x00000000
    [ 154.737733] mxc_mipi-csi.0: addr=0x32e30108 : 0x00000000
    [ 154.737736] mxc_mipi-csi.0: addr=0x32e3010c : 0x00000000
    [ 154.737739] mxc_mipi-csi.0: addr=0x32e30110 : 0x00000000
    [ 154.737741] mxc_mipi-csi.0: addr=0x32e30114 : 0x00000000
    [ 154.737744] mxc_mipi-csi.0: addr=0x32e30118 : 0x00000000
    [ 154.737747] mxc_mipi-csi.0: addr=0x32e3011c : 0x00000000
  • CSI_CSIRXFIFO Value such as 0x00b2008f,Application layer values such as 0xb6c290c2 
    Looking forward to your reply.
0 Kudos
Reply

2,436 Views
joanxie
NXP TechSupport
NXP TechSupport

how did you fix your previous issue about 4 parts?

I checked your CSICR3 register, try to enable bit3 SENSOR_16BITS

0 Kudos
Reply

2,393 Views
pan_
Contributor I
  • For the Part 4 problem, I solved it by setting the PIXEL BIT

pan__0-1684746500565.png

  • I set SENSOR_16BITS, but the final result didn't change .

  • Note that I received the correct data on csi RXFIFO, while I added some print information to the mx6s_csi_frame_done function and it showed the correct result, but ended up with the wrong data on the application layer. 

 

static void mx6s_csi_frame_done(struct mx6s_csi_dev *csi_dev,
		int bufnum, bool err)
{
	struct mx6s_buf_internal *ibuf;
	struct mx6s_buffer *buf;
	struct vb2_buffer *vb;
	unsigned long phys;
	unsigned int phys_fb1;
	unsigned int phys_fb2;
	uint8_t *pmen;

	ibuf = list_first_entry(&csi_dev->active_bufs, struct mx6s_buf_internal,
			       queue);

	if (ibuf->discard) {
		/*
		 * Discard buffer must not be returned to user space.
		 * Just return it to the discard queue.
		 */
		list_move_tail(csi_dev->active_bufs.next, &csi_dev->discard);
	} else {
		buf = mx6s_ibuf_to_buf(ibuf);

		vb = &buf->vb.vb2_buf;
		phys = vb2_dma_contig_plane_dma_addr(vb, 0);
		if (bufnum == 1) {
			phys_fb2 = csi_read(csi_dev, CSI_CSIDMASA_FB2);
			if (phys_fb2 != (u32)phys) {
				dev_err(csi_dev->dev, "%lx != %x\n", phys,
					phys_fb2);
			}
		} else {
			phys_fb1 = csi_read(csi_dev, CSI_CSIDMASA_FB1);
			if (phys_fb1 != (u32)phys) {
				dev_err(csi_dev->dev, "%lx != %x\n", phys,
					phys_fb1);
			}
		}
		dev_dbg(csi_dev->dev, "%s (vb=0x%p) 0x%p %lu\n", __func__, vb,
				vb2_plane_vaddr(vb, 0),
				vb2_get_plane_payload(vb, 0));

		pmen=vb2_plane_vaddr(vb, 0);	
		dev_dbg(csi_dev->dev, "addr=0x%p 0x%08x\n",pmen ,__raw_readl(pmen));
		dev_dbg(csi_dev->dev, "phys1 addr=0x%p  \n",phys);

		list_del_init(&buf->internal.queue);
		vb->timestamp =ktime_get_ns();
		to_vb2_v4l2_buffer(vb)->sequence = csi_dev->frame_count;
		if (err)
			vb2_buffer_done(vb, VB2_BUF_STATE_ERROR);
		else
			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
	}

	csi_dev->frame_count++;
	csi_dev->nextfb = (bufnum == 0 ? 1 : 0);

	/* Config discard buffer to active_bufs */
	if (list_empty(&csi_dev->capture)) {
		if (list_empty(&csi_dev->discard)) {
			dev_warn(csi_dev->dev,
					"%s: trying to access empty discard list\n",
					__func__);
			return;
		}

		ibuf = list_first_entry(&csi_dev->discard,
					struct mx6s_buf_internal, queue);
		ibuf->bufnum = bufnum;

		list_move_tail(csi_dev->discard.next, &csi_dev->active_bufs);

		mx6s_update_csi_buf(csi_dev,
					csi_dev->discard_buffer_dma, bufnum);
		return;
	}

	buf = list_first_entry(&csi_dev->capture, struct mx6s_buffer,
			       internal.queue);

	buf->internal.bufnum = bufnum;

	list_move_tail(csi_dev->capture.next, &csi_dev->active_bufs);

	vb = &buf->vb.vb2_buf;
	vb->state = VB2_BUF_STATE_ACTIVE;

	phys = vb2_dma_contig_plane_dma_addr(vb, 0);
	mx6s_update_csi_buf(csi_dev, phys, bufnum);
}​


pan__1-1684746942041.pngpan__3-1684747142713.png

 

 

  • At the same time I did some other checks that the data received at the application layer was correct when the tof sensor was in test mode (the transmission data was fixed). However, if it is switched to the normal mode, there will be anomalies in the data received at the application layer. Specific anomalies are as I described before. I think the possibility of error of tof sensor can be eliminated, because the data received in RXFIFO is correct. I initially thought there might be a problem with synchronization after the DMA transfer, but now I can't figure out the reason. I hope you can give us a good method.

0 Kudos
Reply

2,361 Views
joanxie
NXP TechSupport
NXP TechSupport

let me remind, bit SENSOR_16BITS in RM, in driver, the name is TWO_8BIT_SENSOR

0 Kudos
Reply

2,368 Views
joanxie
NXP TechSupport
NXP TechSupport

I checked your mx6s_capture.c, you should used gated clock mode, but refer to the register, you set GCLK_MODE to 0, so your dump registers are all correct or did you changed the mx6s_capture after you send the patch?

0 Kudos
Reply

2,370 Views
joanxie
NXP TechSupport
NXP TechSupport

I checked your mx6s_capture.c, you should used gated clock mode, but refer to the register, you set GCLK_MODE to 0, so your dump registers are all correct or did you changed the mx6s_capture after you send the patch?

0 Kudos
Reply

2,373 Views
joanxie
NXP TechSupport
NXP TechSupport

test mode (the transmission data was fixed)

> what about test mode? do you mean that each frame has the same data?

as you mentioned before, higher 4bits doesn't correct, are they random or shift data? I mean could you know where are the higher 4bits from?

I checked your status register, why didn't your board detect SOF or EOF? after you enable SENSOR_16BITS, nothing different? if this is sync issue, you shouldn't get correct data from CSI, I checked the sync related register, it seems they are ok, did you check your application?

0 Kudos
Reply

2,353 Views
pan_
Contributor I

I have checked why the high 4 bits are not 0. The reason is that there is a problem in the analysis of our software. After changing the analysis software, the data is normal. But now we have a new problem, which is that each PIXEL is 12 bits, but I set the PIXEL BIT to 1 to solve the problem of splitting the image into 4 parts, so we only get 10 bits of data. I would like to know how to configure csi to receive 12bit RAW12 data.

0 Kudos
Reply

2,335 Views
joanxie
NXP TechSupport
NXP TechSupport

how about change BIT_PIXEL_BIT to 0 and enable SENSOR_16BITS?

0 Kudos
Reply