Trouble with deinterlacing NTCS/PAL video with an IMX8MP
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Deinterlacing does not work on Yocto Linux, based on kernel 5.10.35.
We have adv748x connected to i.MX8MP using MIPI-CSI interface.
adv748x receive NTSC signal and transmit it over CSI interface to i.MX8MP.
here is output of the media-ctl -p command
Media controller API version 5.10.35
Media device information
------------------------
driver mxc-md
model FSL Capture Media Device
serial
bus info
hw revision 0x0
driver version 5.10.35
Device topology
- entity 1: mxc_isi.0 (16 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
pad0: Sink
<- "mxc-mipi-csi2.0":4 [ENABLED]
pad1: Sink
pad2: Sink
pad3: Sink
pad4: Sink
pad5: Sink
pad6: Sink
pad7: Sink
pad8: Sink
pad9: Sink
pad10: Sink
pad11: Sink
pad12: Source
-> "mxc_isi.0.capture":0 [ENABLED]
pad13: Source
pad14: Source
pad15: Sink
- entity 18: mxc_isi.0.capture (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video3
pad0: Sink
<- "mxc_isi.0":12 [ENABLED]
- entity 22: mxc-mipi-csi2.0 (8 pads, 2 links)
type Node subtype V4L flags 0
device node name /dev/v4l-subdev0
pad0: Sink
<- "adv748x 2-0070 txa":1 [ENABLED,IMMUTABLE]
pad1: Sink
pad2: Sink
pad3: Sink
pad4: Source
-> "mxc_isi.0":0 [ENABLED]
pad5: Source
pad6: Source
pad7: Source
- entity 31: adv748x 2-0070 txa (2 pads, 3 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev3
pad0: Sink
[fmt:UYVY8_2X8/720x240 field:alternate colorspace:smpte170m]
<- "adv748x 2-0070 afe":8 [ENABLED]
<- "adv748x 2-0070 hdmi":1 []
pad1: Source
[fmt:UYVY8_2X8/720x240 field:alternate colorspace:smpte170m]
-> "mxc-mipi-csi2.0":0 [ENABLED,IMMUTABLE]
- entity 34: adv748x 2-0070 afe (9 pads, 1 link)
type V4L2 subdev subtype Decoder flags 0
device node name /dev/v4l-subdev1
pad0: Sink
pad1: Sink
pad2: Sink
pad3: Sink
pad4: Sink
pad5: Sink
pad6: Sink
pad7: Sink
pad8: Source
[fmt:UYVY8_2X8/720x240 field:alternate colorspace:smpte170m]
-> "adv748x 2-0070 txa":0 [ENABLED]
I'm able to capture with command
gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw,width=720,height=240 ! queue ! kmssink
but this command capture only half of image - frame with only odd or only even lines
(720x240 instead 720x480).
When i try to enable deinterlacing it fails:
# gst-launch-1.0 --gst-debug=v4l2src:5 v4l2src device=/dev/video3 ! video/x-raw,width=720,height=240,interlace-mode=interlaced ! queue ! kmssink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
0:00:00.745451625 776 0xaaab147314c0 DEBUG v4l2src gstv4l2src.c:515:gst_v4l2src_negotiate:<v4l2src0> caps of src: video/x-raw(format:Interlaced), format=(string)YUY2, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string)alternate; video/x-raw, format=(string)YUY2, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string){ progressive, interleaved }; video/x-raw(format:Interlaced), format=(string)YUY2, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string)alternate; video/x-raw(format:Interlaced), format=(string)BGRA, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string)alternate; video/x-raw, format=(string)BGRA, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string){ progressive, interleaved }; video/x-raw(format:Interlaced), format=(string)BGRA, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string)alternate; video/x-raw(format:Interlaced), format=(string)BGRx, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string)alternate; video/x-raw, format=(string)BGRx, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string){ progressive, interleaved }; video/x-raw(format:Interlaced), format=(string)BGRx, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string)alternate; video/x-raw(format:Interlaced), format=(string)BGR, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string)alternate; video/x-raw, format=(string)BGR, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string){ progressive, interleaved }; video/x-raw(format:Interlaced), format=(string)BGR, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string)alternate; video/x-raw(format:Interlaced), format=(string)RGB, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string)alternate; video/x-raw, format=(string)RGB, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string){ progressive, interleaved }; video/x-raw(format:Interlaced), format=(string)RGB, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string)alternate; video/x-raw(format:Interlaced), format=(string)NV12, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string)alternate; video/x-raw, format=(string)NV12, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string){ progressive, interleaved }; video/x-raw(format:Interlaced), format=(string)NV12, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string)alternate; video/x-raw(format:Interlaced), format=(string)RGB16, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string)alternate; video/x-raw, format=(string)RGB16, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string){ progressive, interleaved }; video/x-raw(format:Interlaced), format=(string)RGB16, framerate=(fraction)[ 0/1, 2147483647/1 ], width=(int)[ 1, 4096 ], height=(int)[ 1, 8192 ], interlace-mode=(string)alternate
New clock: GstSystemClock
0:00:00.745651375 776 0xaaab147314c0 DEBUG v4l2src gstv4l2src.c:523:gst_v4l2src_negotiate:<v4l2src0> caps of peer: video/x-raw, width=(int)720, height=(int)240, interlace-mode=(string)interlaced, format=(string){ BGRx, BGRA, RGBx, RGBA, NV12_10LE }, framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:01.078356375 776 0xaaab147314c0 DEBUG v4l2src gstv4l2src.c:529:gst_v4l2src_negotiate:<v4l2src0> intersect: EMPTY
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.333395500
Setting pipeline to NULL ...
Freeing pipeline ...
I investigated the imx8-isi-hw.c and didn't find deinterlacing implementation.
There are 2 functions mxc_isi_channel_deinterlace_init and mxc_isi_channel_set_deinterlace but first of them is empty and second is never used.
Maybe I use wrong pipeline?
Is there other way to get interlaced image from the Capture Media Device?
Is there implementation of the ISI deinterlacing?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Did anyone ever manage to get odd/even (aka top/bottom) indications from v4l2_buffer via i.MX 8 MIPI CSI-2 (imx8-isi, mxc_isi)?
I'm managing to dequeue interlaced frames (50 fps, 720 x 288) from ADV7280A-M.
But the members of the v4l2_buffer are not changing to indicate odd/even.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi amberworth,
Were you able to get deinterlacing NTCS/PAL to work with the mx8mplus?
Regards,
Mario
igorpadykov
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Amber
one can try to port below patch for i.MX8MP:
Camera porting documents can be found on
Best regards
igor
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi igor
So far I have no success to deinterlace frames of the adv7280a-m with the i.MX8MP. Even with the patch applied to kernel 5.4
Basically frames can be received via mipi csi-2 on the i.MX8MP. Only as soon as i.MX8MP deinterlacing is activated, nothing happens.
Here the register dump for PAL (frame 720x576, field 720x288), Deinterlace method is weaving, no blending, no ntsc_top.
[ 130.526648] ISI CHNLC register dump, isi0
[ 130.530928] CHNL_CTRL[0x00]: e0000000
[ 130.535661] CHNL_IMG_CTRL[0x04]: 20002001
[ 130.540383] CHNL_OUT_BUF_CTRL[0x08]: 7c707
[ 130.544850] CHNL_IMG_CFG[0x0c]: 12002d0
[ 130.549488] CHNL_IER[0x10]: 3cfc0000
[ 130.554231] CHNL_STS[0x14]: 100
[ 130.558520] CHNL_SCALE_FACTOR[0x18]: 10001000
[ 130.563254] CHNL_SCALE_OFFSET[0x1c]: 00
[ 130.567467] CHNL_CROP_ULC[0x20]: 00
[ 130.571681] CHNL_CROP_LRC[0x24]: 00
[ 130.575884] CHNL_CSC_COEFF0[0x28]: 00
[ 130.580090] CHNL_CSC_COEFF1[0x2c]: 00
[ 130.584291] CHNL_CSC_COEFF2[0x30]: 00
[ 130.588502] CHNL_CSC_COEFF3[0x34]: 00
[ 130.592709] CHNL_CSC_COEFF4[0x38]: 00
[ 130.596920] CHNL_CSC_COEFF5[0x3c]: 00
[ 130.601128] CHNL_ROI_0_ALPHA[0x40]: 00
[ 130.605342] CHNL_ROI_0_ULC[0x44]: 00
[ 130.609555] CHNL_ROI_0_LRC[0x48]: 00
[ 130.613762] CHNL_ROI_1_ALPHA[0x4c]: 00
[ 130.617969] CHNL_ROI_1_ULC[0x50]: 00
[ 130.622180] CHNL_ROI_1_LRC[0x54]: 00
[ 130.626385] CHNL_ROI_2_ALPHA[0x58]: 00
[ 130.630591] CHNL_ROI_2_ULC[0x5c]: 00
[ 130.634800] CHNL_ROI_2_LRC[0x60]: 00
[ 130.639002] CHNL_ROI_3_ALPHA[0x64]: 00
[ 130.643220] CHNL_ROI_3_ULC[0x68]: 00
[ 130.647431] CHNL_ROI_3_LRC[0x6c]: 00
[ 130.651637] CHNL_OUT_BUF1_ADDR_Y[0x70]: 6a400000
[ 130.656361] CHNL_OUT_BUF1_ADDR_U[0x74]: 00
[ 130.660570] CHNL_OUT_BUF1_ADDR_V[0x78]: 00
[ 130.664774] CHNL_OUT_BUF_PITCH[0x7c]: 5a0
[ 130.669069] CHNL_IN_BUF_ADDR[0x80]: 00
[ 130.673277] CHNL_IN_BUF_PITCH[0x84]: 00
[ 130.677480] CHNL_MEM_RD_CTRL[0x88]: 00
[ 130.681688] CHNL_OUT_BUF2_ADDR_Y[0x8c]: 6a100000
[ 130.686414] CHNL_OUT_BUF2_ADDR_U[0x90]: 00
[ 130.690623] CHNL_OUT_BUF2_ADDR_V[0x94]: 00
[ 130.694831] CHNL_SCL_IMG_CFG[0x98]: 12002d0
[ 130.699469] CHNL_FLOW_CTRL[0x9c]: 00Are the registers configured correctly?
Any idea what the problem could be?
Thanks in advance.
Mario
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Just in case anyone would be interested:
I had exactly the same problem (IMX8QM): MIPI CSI2 block receives frames OK, but as soon as deinterlace is turned on in ISI, no frames are getting through ISI.
This could be "fixed" by setting VC_INTERLACED (0x30) MIPI CSI CSR register to appropriate value (to me, VC0 is interlaced). I had to modify drivers/staging/media/imx/imx8-mipi-csi2.c to do this.
Frames are getting through after that change.
However, the result was ... a bit disappointing. "Need more work".
1. There is 50% chance that ISI weaving deinterlacing will result in a corrupted video.
It seems that odd/even flags are lost somewhere between CSI and ISI and ISI uses wrong field order. As a result, lines order is wrong and instead of getting smooth picture, you get it very "minecraft"-styled.
In CSI protocol, there is "F-bit", which indicates even/odd field, but it is unclear to me how it passes CSI -> Pixel Link -> ISI chain. Ref.Manual says "VC Interlaced - to drive the interlace port of the Pixel Link Master, via the HSYNC/VSYNC generator."
If that is simply counts up VSYNCs, it will never work, of course (right as observed, 50/50). If so, the only working deinterlace method would be blending (?).
2. Color space conversion stopped to work.
I'm unable to tell whether this is intended behaviour, because to my understanding, only "blending" requires CSC resources and weaving only requires different RAM layout at the output block.
Again, documentation is unclear here. See CHNL_BYPASS of ISI CHNL_CTRL CSR in Ref.Manual.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
It seems we have the same problem over here on the i.MX8QM…
Were you able to get the odd/even issue solved?
We’re also wondering how the odd/even is getting transferred over the pixel link between the MIPI-CSI and the ISI? The MIPI signals described in the reference manual do not contain any F-signal too…
We have also 50% chance that the odd/even fields are ordered/deinterlaced in the right way. This for both NTSC and PAL signals arriving (decoded) digitally on the MIPI input.
And if you would be curious, the blending mode has exactly the same problem. The problem is already visible in the intermediate YUYV image generated by ISI channel 1, which then enters ISI channel 0.
Any help or feedback would be nice.
Thanks!
Regards.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I was able to find out the following:
The interrupt callback mipi_csis_irq_handler of drivers/staging/media/imx/imx8-mipi-csi2-sam.c is executed 50 times per second. And the error bit status is always set to 00001000. The bit 12 means "Indication of lost of Frame Start packet, CH0."
But if deinterlacing is disabled, frames are received and error bit 12 is also set.
Can "lost of Frame Start packet" have influence on deinterlacing?
And without de-interlacing the frames are passed through?
