topic Re: i.MX53 IPU Deinterlacer in i.MX Processors

i.MX53 IPU Deinterlacer

CraigPetku — Wed, 05 Oct 2011 23:38:13 GMT

Has anyone come across a white paper or a Linux driver which implements the HW deinterlacer found in the i.MX53? After getting an ADV7180 running on CSI1, I now have odd/even fields that need to be combined (preferably using the IPU). For a quick demo using the unit_tests it's easy just to crop and scale, but I'd really like to see the IPU do the heavy lifting.

Re: i.MX53 IPU Deinterlacer

DualDisplayiss1 — Sun, 16 Oct 2011 04:29:04 GMT

Do you mind use i.mx51CSI1 interface+ADV7180?

Eric

Re: i.MX53 IPU Deinterlacer

CraigPetku — Thu, 20 Oct 2011 02:31:03 GMT

I haven't found good documentation on this yet, but while trying to get a ADV7180 up in Android I discovered the CSI->MEM path appears to be deinterlacing video on the fly. Unfortunately it's not aligning the two frames properly and I have VSYNC issues. The partial de-interlacing results in no chroma where the frames don't align. Changing the ADV register settings per their DS results in better algnment (defaults have every other fame really deinterlaced poorly), but still not correct (and still not v-synced)

Askng FSL for detailed information on the CSI and IPU configuration but it's slow getting a reply. The CCIR registers in the CSI block are not documented for bit order or why there are two blanking fields per frame. This makes verifying that they are setup correctly almost impossible, but the reference code seems reasonable ofter compiling several variations of CCIR code bit orders and testing them to see which ones are potential fits. The naming convention for fields in the CCIR registers does not appear to be be imply correct ordering for the H, F and V bits. The only thing keeping me sane on all the compiles is having a fast quad core laptop.

Placing printk's everwhere I don't see where the CSI is linking to the VDIC and the SMFC doesn't appear to be throwing away frames and rescaling.

Also of interest is the PRP path (tested with the mxc_v4l2_overlay unit test) works fine (good hsync) but the CSI_ENC route fails to sync. I've tried going to progressive mode hoping to just see interleaved data, but that fails to VSYNC as well (picture rolls). Attempts to change the mx5x/libcamera routines to use the PRP path (changing the input from 1 to 0) fail with no video captured.

I would even consider trying gated mode to get rid of the sync issue, but the RM and UG have no verified timing diagrams to demonstrate signal polarities and porch requirements. It's also unclear if the CSI would discard the embedded codes based upon an external hsync. I'm guessing this would require a logic analyzer to verify the h-v sync timing is properly aligned with the SAV and EAV codes unless FSL can state that it would treat this as YUV data (even though the values would be out of range).

Has anyone sucessfully integrated an adv7180 with Android yet? I'm starting to wonder if CSI1 on the i.MX53 has an issue with AV codes...

Re: i.MX53 IPU Deinterlacer

iWave — Fri, 21 Oct 2011 06:00:11 GMT

Eric,

We need de-interlace support in i.MX51 CSI1 interface & we are struggling to bring-up.

Can you share the de-interlacing support on i.mx51CSI1 interface+ADV7180?

Re: i.MX53 IPU Deinterlacer

CraigPetku — Sat, 22 Oct 2011 01:55:30 GMT

So hunting through the code today I found the csi_enc routines seem to do their de-interlacing using the idmac in ipu_common.c. Still looking for my cause of video rolling which is starting to look like the idmac My updates are still too rough to post. Hopefully I'll have this figured out before Thursday. If so I'll send a note to Zafeer.

Re: i.MX53 IPU Deinterlacer

DualDisplayiss1 — Sat, 22 Oct 2011 08:49:55 GMT

iWave Systems Technologies said:

Eric,

We need de-interlace support in i.MX51 CSI1 interface & we are struggling to bring-up.

Can you share the de-interlacing support on i.mx51CSI1 interface+ADV7180?

Hi~~

Are you porting in Wince ?

we 're just debugging now since HW ready in these day.As i know ADV7180 is use I2C command to control this chip.

can you read chip-set staus?

the otherwise,do you have record software (API).

Eric

Re: i.MX53 IPU Deinterlacer

CraigPetku — Sat, 22 Oct 2011 17:05:54 GMT

Ok, I got the ADV7180 working with the Android camera app. I'll have several action items for my local rep (when I see him) e.g., explain the synchronization strategy between the CSI port and the IDMAC in CSI->MEM.

It looks like in the CSI->MEM path the CSI free runs and the IDMAC generates the camera call back every time it completes capturing one frame of data without syncronizing SOF to the CSI port SAV code timing. If so, there's potential for these to get out of sync.

In the CSI->IC path syncronization appears to be done appropriately resulting in a stable image always.

Of course this could just be the current state of Linux drivers and not a design limitation,.

Once you get the frames synced properly and adjust offsets to account for the porches, then the de-interlacer in the IDMAC will start to behave as well.

Re: i.MX53 IPU Deinterlacer

LautaroCarmona1 — Thu, 27 Oct 2011 06:33:52 GMT

Hi Craig,

While we were doing some tests with TVP5147 and i.MX51's CSI, we realized that our CSI is not performing deinterlacing either and we are receiving the two fields of every frame as two separated image of about 240 lines each, and that they are being drawn together (I will upload a photo of this sceneario to explain myself better asap)... i found in this discussion that you had a similar situation with ADV7180... can you explain me how did you get the frames synced properly?

Thanks,
Lautaro

Re: i.MX53 IPU Deinterlacer

CraigPetku — Fri, 28 Oct 2011 00:04:42 GMT

To start with I am working with the 10.2 version of FSL android. However, looking at an older ltib release from 2010 it looks like ths has been out for awhile.

Look at ./drivers/mxc/ipu3/ipu_common.c. You should find something like...

if (_ipu_chan_is_interlaced(channel)) {
_ipu_ch_param_set_interlaced_scan(dma_chan);
}

In my case this handles the CSI-MEM deinterlacing.

So Where does the interlace parameter come from?...

Look at mxc_v4l_open in ./drivers/media/video/mxc/capture

Older distro's have something like

csi_param.pixclk_pol = ifparm.u.bt656.latch_clk_inv;

        /* Once we handle multiple inputs this will need to change. */
        csi_param.csi = 0;

        if (ifparm.u.bt656.mode
                == V4L2_IF_TYPE_BT656_MODE_NOBT_8BIT)
            csi_param.data_width = IPU_CSI_DATA_WIDTH_8;
        else if (ifparm.u.bt656.mode
                == V4L2_IF_TYPE_BT656_MODE_NOBT_10BIT)
            csi_param.data_width = IPU_CSI_DATA_WIDTH_10;

Newer ones include the magic parameter based upon a clock setting returned by the sensor driver. Sorry I can't post the actual code tonight, so I'll try from work tomorrow. Worst case, pull down the latest version of the linux OS and look in the mxc_capture around where I posted, then update your sensor driver to return a 0 (I think) in the parameter that results in interlaced video being identified.

Re: i.MX53 IPU Deinterlacer

charleschang — Fri, 04 Nov 2011 03:12:42 GMT

I use imx51, android 10.3 with ADV7180.

the unit test program : mxc_v4l2_tvin runs well , it output de-interlaced frame on screen.

the de-interlace seems be done on the output path, not the capturing path.

Re: i.MX53 IPU Deinterlacer

LautaroCarmona1 — Wed, 09 Nov 2011 02:41:47 GMT

Dear Craig,

Hi! Thanks for your advice! I could not test it until today!

As you suggested us, we looked at drivers/media/video/mxc/capture/mxc_v4l2_capture.c and we found this piece of code:

/* This may not work on other platforms. Check when adding a new one.*/

pr_debug(" clock_curr=mclk=%d\n", ifparm.u.bt656.clock_curr);
if (ifparm.u.bt656.clock_curr == 0) {
csi_param.clk_mode = IPU_CSI_CLK_MODE_CCIR656_INTERLACED;
} else {
csi_param.clk_mode = IPU_CSI_CLK_MODE_GATED_CLK;
}
csi_param.pixclk_pol = ifparm.u.bt656.latch_clk_inv;

if (ifparm.u.bt656.mode == V4L2_IF_TYPE_BT656_MODE_NOBT_8BIT) {
csi_param.data_width = IPU_CSI_DATA_WIDTH_8;
} else if (ifparm.u.bt656.mode== V4L2_IF_TYPE_BT656_MODE_NOBT_10BIT) {
csi_param.data_width = IPU_CSI_DATA_WIDTH_10;
} else {
csi_param.data_width = IPU_CSI_DATA_WIDTH_8;
}

So, we assumed that the clock setting you were talking about was ifparm.u.bt656.clock_curr .

Then, we modified our driver in order to set this value to return a 0, like this:

static int ioctl_g_ifparm(struct v4l2_int_device *s, struct v4l2_ifparm *p)

{
#ifdef DEBUG
printk("In tvp5147:ioctl_g_ifparm\n");
#endif

if (s == NULL) {
printk("<3>"" ERROR!! no slave device set!\n");
return -1;
}

/* Initialize structure to 0s then set any non-0 values. */
memset(p, 0, sizeof(*p));
p->if_type = V4L2_IF_TYPE_BT656; /* This is the only possibility.*/
p->u.bt656.mode = V4L2_IF_TYPE_BT656_MODE_BT_10BIT;
//p->u.bt656.nobt_hs_inv = 1;
p->u.bt656.bt_sync_correct = 1;

/* tvp5147 has a dedicated clock so no clock settings needed. */

p->u.bt656.clock_curr = 0;

return 0;
}

And then we also add some printk's in mxc_v4l2_capture.c :

if (ifparm.u.bt656.clock_curr == 0) {

csi_param.clk_mode = IPU_CSI_CLK_MODE_CCIR656_INTERLACED;

printk("INTERLACED!!\n");
} else {
csi_param.clk_mode = IPU_CSI_CLK_MODE_GATED_CLK;

printk("NOT INTERLACED!!\n");
}

Finally, we run the mxc_v4l2_overlay.out and mxc_v4l2_capture.out unit tests, but in both cases we saw the same results: two separated image of about 288 lines each vertically aligned (instead of one unique image of 576 lines), and in the serial console we could see the printk output "INTERLACED!!".

Any ideas?

Thanks in advance,

Lautaro

P.S.: We still have to try to capture using the csi->mem path

Craig Petku said:

To start with I am working with the 10.2 version of FSL android. However, looking at an older ltib release from 2010 it looks like ths has been out for awhile.

Look at ./drivers/mxc/ipu3/ipu_common.c. You should find something like...

if (_ipu_chan_is_interlaced(channel)) {
        _ipu_ch_param_set_interlaced_scan(dma_chan);
    }

In my case this handles the CSI-MEM deinterlacing.

So Where does the interlace parameter come from?...

Look at mxc_v4l_open in ./drivers/media/video/mxc/capture

Older distro's have something like

csi_param.pixclk_pol = ifparm.u.bt656.latch_clk_inv;

        /* Once we handle multiple inputs this will need to change. */
        csi_param.csi = 0;

        if (ifparm.u.bt656.mode
                == V4L2_IF_TYPE_BT656_MODE_NOBT_8BIT)
            csi_param.data_width = IPU_CSI_DATA_WIDTH_8;
        else if (ifparm.u.bt656.mode
                == V4L2_IF_TYPE_BT656_MODE_NOBT_10BIT)
            csi_param.data_width = IPU_CSI_DATA_WIDTH_10;

Newer ones include the magic parameter based upon a clock setting returned by the sensor driver. Sorry I can't post the actual code tonight, so I'll try from work tomorrow. Worst case, pull down the latest version of the linux OS and look in the mxc_capture around where I posted, then update your sensor driver to return a 0 (I think) in the parameter that results in interlaced video being identified.

Re: i.MX53 IPU Deinterlacer

charleschang — Wed, 09 Nov 2011 05:50:54 GMT

I use ADV7180 + imx51. android 10.3.1

the unit test program mxc_v4l2_tvin runs well, it output de-interlaced frame on overlay. and no need to memcpy each frame.

-- the mxv_v4l2_tvin code is different fromt imx53. though they have the same filename.

just a little bug , the default video standard is PAL, it sync fail when we input the NTSC video.

and

sometime it hangs when video source is disappeared for a short period.

iWavesystems said:

Eric,

We need de-interlace support in i.MX51 CSI1 interface & we are struggling to bring-up.

Can you share the de-interlacing support on i.mx51CSI1 interface+ADV7180?

Re: i.MX53 IPU Deinterlacer

LautaroCarmona1 — Sun, 13 Nov 2011 03:39:21 GMT

Here is a picture of what we are capturing with TVP5147 and i.MX51's CSI.

It seems that the CSI is not performing deinterlacing and here it can be seen that we are receiving the two fields of every frame as two separated image of 288 lines each, and that they are being drawn together.

To capture this raw video I executed this command:

/unit_tests/mxc_v4l2_capture.out -iw 720 -ih 576 -ow 720 -oh 576 -f UYVY -fr 15 test.yuv

Then we played it with this other command :

mplayer -rawvideo w=720:h=576:fps=15mat=uyvy -demuxer rawvideo test.yuv

P.S.: we also tried with both possible values (0 and 1) to the "-i" argument in order to try csi->prp->mem and csi->mem path with the same results! :(

Re: i.MX53 IPU Deinterlacer

Ananth — Thu, 24 Nov 2011 13:01:32 GMT

I use TVP5151 + imx53

I use mxv_v4l2_tvin test app and in the output the image seen in the monitor "rolls".

I tried adding

p->u.bt656.clock_curr = 0;

nothing works :-(

Any fix/workaround for this problem ?

Re: i.MX53 IPU Deinterlacer

Ananth — Fri, 06 Jan 2012 11:40:24 GMT

I was able to fix this problem by programming drivers/media/video/mxc/capture/mxc_v4l2_capture.c .active_top =13 to .active_top = 3. And i was able to view the video in a good form using

mxc_v4l2_tvin.out -ow 720 -oh 480 -ol 20 -f UYVY

i was able to see a video interlaced form of video as told by Lautaro Carmona by using mxc_v4l2_capture.out file.

Queries

1. I am looking for any method/patches in IMX53 to avoid frame rolling issue (other than mentioned fix).

2. From my De-coder(tvp5147) i was able to get 720x480 but in my start of line and end of line i have some blank data getting appended.

So i wish to crop it from 720 to 700 by programming 20 pixel skip from left. so i tried "-l" option in mxc_v4l2_capture.out

but it is not working ... Any suggestion for this problem ?

3. i wish to take even lines frame and odd lines frame out of image. So i used -t as 240 and output dimension as 720x240 in mxc_v4l2_capture.out. My method it is not working ... Any suggestion for this problem ?

Re: i.MX53 IPU Deinterlacer

WillSheppard — Tue, 21 Feb 2012 16:03:07 GMT

Hello,

I was unable to get the VDI module working but as for the display issue we managed to fix that.

We had the issue where Field A and B were displayed one above the other.

We now see interlaced video whereby the two fields are put into memory by interleaving their repsective lines. Of course you will see "combing artefacts" but that is becuase the video needs to be de-interlaced; however, it has been sufficient for us to use it this way.

The relevant entry is in the file ipu_common.c as mentioned earlier. If you ensure your IPU logical channel is calling the set_interlaced function, the memory writes will put the frames together as interleaved data not separate fields.

Re: i.MX53 IPU Deinterlacer

JackyHuang — Thu, 08 Mar 2012 14:47:47 GMT

Hi charles

I use imx51, android 10.2 with ADV7180. but i found the screen rolling endless.

how did you porting the adv7180 driver to android 10.2

Is it need to modify the camera HAL?

after I porting the adv7180 driver which is run normally on android 9.4 to android 10.2,

i got a rolling screen.

Could you give me some advice?

thanks

charles chang said:

I use imx51, android 10.3 with ADV7180.

the unit test program : mxc_v4l2_tvin runs well , it output de-interlaced frame on screen.

the de-interlace seems be done on the output path, not the capturing path.

Re: i.MX53 IPU Deinterlacer

CraigPetku — Fri, 09 Mar 2012 00:11:50 GMT

It's been several months since I got into the guts of the camera HAL and ADV7180, but here's some general comments.

The Gingerbread Camera HAL really wants a 4:3 aspect ratio. The only format that I found would work was to have the 7180 report 720x480 back to V4L2 and ignore the fact that the frame is actually 525 lines. This results in rolling video since the DMA buffer is configured for only 480 lines and the image is 525. It also has terrible frame combining in the IPU routines. One solution is to program the registers in the 7180 to only output 480 lines. This can be done by carefully adjusting where the AV codes occur in the stream. Once the blanked HS lines are discarded (on both fields) the IPU will do field combining which may result in combing artifacts but is usable in development. The down side to this appears that iPOD video has slightly different timing and the first few lines get clipped. A more robust approach would be to use the v4l2_capture routines to pull in 525 lines and de-interlace the video before passing the 480 lines of interest up to the android camera routines. I have not had time to implement this yet. I also reversed the field1 / field0 codes in the IPU routines since they seemed to be backwards, but this is not confirmed and the RM is of little use in interpreting the bitfields for the AV codes.

Re: i.MX53 IPU Deinterlacer

JackyHuang — Fri, 09 Mar 2012 11:04:25 GMT

Actually, I stop the screen rolling through modify the adv7180 reg value, the reg is about the NTSC V bit and F bit ,

the reg addr is 0xe5, 0xe6, 0xe7.

however, I got some sync issue that make the screen appear some antialiasing.

Are you resolve the issue through modify the adv7180 reg value?

and, Could you explain how to passing the 480 lines of interest data to android camera routines?

actually, I can't get the point about the robust solution!

Hope you can tell me the detail of you solution.

Thanks advance!
Craig Petku said:

It's been several months since I got into the guts of the camera HAL and ADV7180, but here's some general comments.

The Gingerbread Camera HAL really wants a 4:3 aspect ratio. The only format that I found would work was to have the 7180 report 720x480 back to V4L2 and ignore the fact that the frame is actually 525 lines. This results in rolling video since the DMA buffer is configured for only 480 lines and the image is 525. It also has terrible frame combining in the IPU routines. One solution is to program the registers in the 7180 to only output 480 lines. This can be done by carefully adjusting where the AV codes occur in the stream. Once the blanked HS lines are discarded (on both fields) the IPU will do field combining which may result in combing artifacts but is usable in development. The down side to this appears that iPOD video has slightly different timing and the first few lines get clipped. A more robust approach would be to use the v4l2_capture routines to pull in 525 lines and de-interlace the video before passing the 480 lines of interest up to the android camera routines. I have not had time to implement this yet. I also reversed the field1 / field0 codes in the IPU routines since they seemed to be backwards, but this is not confirmed and the RM is of little use in interpreting the bitfields for the AV codes.

Re: i.MX53 IPU Deinterlacer

mithunk — Wed, 18 Apr 2012 05:09:57 GMT

I am using TV-in application.it is working fine for PAL but for NTSC,screen rolls :(

I am working on tvp5150 decoder driver and what registers have to set/modify to stop the the screen rolling for NTSC.

thanks,