4 video capture using MIPI-CSI2 port

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

4 video capture using MIPI-CSI2 port

Jump to solution
57,244 Views
kanoumamoru
Contributor III

We 'd like to realize real time souround view camera system by using 4 camera.

But i.MX6Q don't have 4 parallel video capture ports. Max 2 only. (=paralle1 and parallel2)

So Can we capture 4 camera signals simultaneously via MIPI-CSI2(4lane) ?

According to the reference manual,  it seem to accept  interleaved 4 streaming video data.

i.MX6Q SABRE-AI have MIPI connector.  We want to make 4 camera extension board use this port.

kanou

Labels (1)
1 Solution
20,803 Views
astraub
NXP Employee
NXP Employee

Hi Kanou,

yes, that should be possible. The MIPI-CSI-2 port knows up to 4 virtual channels. Those can be routed to the two IPU blocks (2 each). You need to make sure that each stream uses a different virtual channel. The IPU has to be set to accept two virtual channels each.

Attached a diagram I drew during a discussion with FIL about this topic. Hope this clarifies the isue. Note that the IPU will take care of converting the incoming format to either RGB8888 or YUV4:4.4 before writing the data into memory.

Best Regards

Andreas

View solution in original post

0 Kudos
Reply
37 Replies
16,384 Views
tamilarasane
Contributor III

Hi

Hope this may help some level !

You can look at the deserializer chip avaialble with TI. it converts the 4 cameras in to single MIPI CSI2 packets with configurable virtual channel ID . but is requires companion serilizer chip at  each camera end to convert  parallel  data  to LVDS ( FPD link III ) conversion.

http://www.ti.com/lit/ds/symlink/ds90ub964-q1.pdf 

For MIPI CSI2 cameras you need to convert  MIPI CSI2 to Parallel ( toshiba TC358746AXBG) and then Parallel to FPD link III using TI serilizer chip (DS90UB913A-Q1). 

 

Thanks,

Tamilarasan

16,386 Views
fedoradarichev
Contributor I

So, if I was using two separate data streams, would the configuration of chips be set up in this way: 2 x TC358 -> 2 x DS90UB913AQ -> 1 x DS90UB964-Q1 ? And, the FPD-LINK III interface between the DS90uB913AQ and DS90UB964-Q1, is this accomplished ONLY with coax or STP cables? Say if I wanted this all to be PCB-mounted, can I not just connect these two chips together with traces?

0 Kudos
Reply
16,385 Views
Richard1z
Contributor II

Is this the solution we've all been waiting for ?

http://www.latticesemi.com/en/Products/DesignSoftwareAndIP/IntellectualProperty/IPCore/IPCores04/2In...

Does anyone know of a Processor SoC that easily connects to 2 MIPI CSI2 cameras, without this kind of glue logic ?

(i.e. a SoC with two independent CSI2 ports - surely there must be plenty of ADAS applications ?)

Richard.

0 Kudos
Reply
16,387 Views
daviadtan
Contributor I

Hi Andreas and others,

we also have same application, we need to connect two cameras which all MIPI interface. Please help to check whether below application is correct?

You know there are two IPU in IMX6Q, so if the each camera only include two differential line,we can connect as below:

  • MIPI CSI Lanes 0,1 ==> Camera 0
  • MIPI CSI Lanes 2,3 ==> Camera 1
  • MIPI CLK lane ==> Zero delay clock buffer ==> to both sensors
  • MIPI I2C : Connect together, different I2C address

Please help to double check whether this design is correct for IMX6Q? Does we can use this way to connect two cameras?

Thanks!

pastedImage_0.png

0 Kudos
Reply
16,387 Views
astraub
NXP Employee
NXP Employee

well, there appears to be a device from Omnivision coming (OV690?) - that would allow to combine two cameras into one stream.

Greetings

Andreas

0 Kudos
Reply
16,387 Views
yanirhirshberg
Contributor I

Do have more info on the part (link?). Googling has not turned up much nor their website.

0 Kudos
Reply
16,387 Views
astraub
NXP Employee
NXP Employee

Hi Yanir,

you need an interface FPGA with 4 MIPI CSI Inputs and one output. The FPGA would then use each of the input streams

assign them to a specific virtual channel (in the packet headers) and combine then all of the 4 streams into one output

stream. See the MIPI communication like an IP stream of packets - only that instead of IP addresses, virtual channel

numbers are being used.

Greetings

Andreas

16,387 Views
yanirhirshberg
Contributor I

Thanks, that makes sense, but not the answer I was hoping for.

0 Kudos
Reply
16,387 Views
Selea
Senior Contributor I

The porblem is that is not available any "MIPI multiplexer" you ha to do by yourself with an FPGA... we had the same problem.

mar

0 Kudos
Reply
16,398 Views
Selea
Senior Contributor I

In our application we would like to acquire video streams from 4 analog cameras (a 4 channel video server).

I can understand how to handle the 4 stream from the mipi interface. But my question is:

How to have those 4 channels in the same mipi flux?

I mean i cannot find any video decoder with mipi output, ok i can solve it using a converter from parallel to mipi (several available on the market), but still i need something to multiplex the 4 single mipi stream to a single multiplexed mipi.

Do anybody know a chip to do that?

Thanks

Omar

0 Kudos
Reply
16,398 Views
astraub
NXP Employee
NXP Employee

Hi Omar,

you will need soem kind of FPGA in between the cameras and the MIPI CSI-2 port, as it will be also required to put the 4 cameras into 4 separate logical streams. I am not aware of any ready built solution for this. All my customers do this themselves.

Greetings

Andreas

16,397 Views
jrychter
Contributor V

I have an application where I will need to process input from 4 CSI-2 cameras (encoding to H.264 later). I am looking at i.MX processors and found this discussion. I am unclear on the following:

  • which i.MX processors will be capable of processing 4 CSI-2 connections?
  • how to put the cameras into separate logical streams?
  • how to actually route the streams to IPUs?

I looked at the PDF diagram, but it isn't easy to understand for someone without a lot of experience in the field.

Since there is clearly a lot of interest in this solution, perhaps an application note could be prepared?

0 Kudos
Reply
16,387 Views
astraub
NXP Employee
NXP Employee

Hi,

all i.MX6 variants should be able to do this. They differ however regarding the maximum bitrate that can be supported

as the Solo/DualLite variants only support two hw lanes.

The MIPI CSI-2 protocol is a packet based protocol (somehow similar to IP). In the headers of the packets is the virtual

channel number (1-4). This has to be inserted by hardware – for example in a FPGA.

The IPU setup certainly is not easy to understand. I attach a slide that might increase the understanding of the basic

principles.

Mit freundlichen Grüßen / With best regards

Andreas Straub

Senior Field Application Engineer

Freescale Halbleiter Deutschland GmbH

Schatzbogen 7

81829 Muenchen / GERMANY

www.freescale.com<;http://www.freescale.com/>

Tel: +49 (0) 89 92103 - 544

Fax: +49 (0) 89 92103 - 101

Mobile: +49 (0) 171 4665886

All types of technical support (Schematic review, layout review, software review, hardware board and software) provided by Freescale Field application team are subject to Freescale's general Terms and Conditions unless superseded by a direct contract.

-

USt.-ID-Nr./VAT-ID-No. DE813898243

Steuernummer/Tax No. 143/138/30552

Sitz der Gesellschaft/Registered Office: München

Registergericht/Registered Court: Amtsgericht München HR B 151590

Geschäftsführer/General Manager: Jürgen Weyer, Daryl Raiford, Marie-Pascale Diez, John Holmes

This e-mail, and any associated attachments have been classified as:

Public

Freescale Semiconductor Internal Use Only

Freescale Semiconductor Confidential Proprietary

0 Kudos
Reply
16,385 Views
davemcmordie
Contributor III

Hi Andreas and others,

The information in this thread has been a valuable starting point for me-- thanks for sharing your knowledge and ideas.

The OV5647 sensor from OmniVision provides the ability to control the virtual channel number for the sensor when using the MIPI interface.  Do you think that it would be possible to capture from two of these sensors using the following configuration:

  • MIPI CSI Lanes 0,1 ==> Sensor0
  • MIPI CSI Lanes 2,3 ==> Sensor1
  • MIPI CLK lane ==> Zero delay clock buffer ==> to both sensors
  • MIPI I2C : use analog switch to control which sensor we are talking to (one at a time)

If all the FPGA would be doing is inserting the virtual channel ID in the MIPI packet header, does it make sense that this would work?

Best,


Dave

0 Kudos
Reply
16,385 Views
astraub
NXP Employee
NXP Employee

Hi Dave,

I do not believe this would work. The number of lanes is used solely as a bandwidth multiplier and will not allow

to split it into two ”groups” for a specific MIPI virtual channel. The received or transmitted data stream will be just

(de-)multiplexed using the number of lines available. Just layer 1 processing.

Greetings

Andreas

0 Kudos
Reply
16,385 Views
yanirhirshberg
Contributor I

It's still unclear how to connect 4 cameras to the csi-2 interface. For example if I want 4 CSI-2 one lane cameras  connected to the i.mx6 csi interface do they all share lane 1? If I want two 4 lane cameras connected to they all share the entire bus?

This would create stubs in routing if mulitple cameras had they're own connectors. I agree that an app note is in order for this.

0 Kudos
Reply
16,386 Views
rob1234
Contributor I

Toshiba make a Parallel (24-Bit Bus, RGB565/666/888, RAW8/10/12/14, YUV422, YUV444) up to 154MHz to 1Gbps CSI-2 Bridge (bi-directional) IC, part number TC358746. This could partially solve the issue of getting the parallel Video data into the serial port. Therefore an FPGA is not needed.

http://www.toshiba.com/taec/components/ProdBrief/11J02_TC358746_ProdBrief.pdf

0 Kudos
Reply
16,386 Views
jrychter
Contributor V

From what I understand, this Toshiba chip can bridge a parallel interface to CSI-2 (either way), but I can't see how it can be useful in reassigning virtual channels in a CSI-2 serial stream.

Also, having re-read the entire thread, I am still unclear on what is the actual limit on the number of cameras. From what I understood, four is the upper limit that the IPU units can handle. It would be great to get a confirmation on that, as there are applications where more cameras might be needed.

0 Kudos
Reply
16,386 Views
rob1234
Contributor I

There are devices that can format the video from multiple sources, however they would need to connect to the iMX6 over it's PCIe x 1 port. This presents more issues with getting the data into the IPU.

Companies such as Conexant, & Pericom, etc make multichannel analogue/Digital Video encoders to PCIe/BT.656 bridges.

0 Kudos
Reply
16,386 Views
rob1234
Contributor I

There is also IP for FPGAs (which maybe in the design anyhow) that can take in Composite video (external decoder) and then Mosaic this into a 1080p format so that the four images are lined up in the top left, right, bottom left and right of the single frame. This could be sent to the iMX6 which then performs a single 1080p/30 encode.

A UK company called Bitec developed such IP for Altera some years ago.

http://www.bitec.ltd.uk/hsmc_qvideo_mosaic_1080p_v81.pdf

0 Kudos
Reply