iMX6 TW686x Surround view patch

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

iMX6 TW686x Surround view patch

No ratings

iMX6 TW686x Surround view patch

We use PCIe to connect Intersil TW6865 chip for the surround view solution.

This is the connection of PCIe to iMX6Q SabreSD board.5171_5171.JPG1.JPG

 

This is the block diagram of the connection:

5172_5172.JPG3.JPG

This is the 4 camera surround view:5173_5173.JPG4.JPG

 

Code base is L3.0.35_12.10.02 release. You can merge the patch file to the latest Freescale release.

Please check the attach file for the patch code.

 

Note:  It is only a test version.

The last code for L3.0.35 BSP: L3.0.35_GA4.1.0 Patches.7z

The last code for L3.10.53 BSP: L3.10.53_TW686x_patch.7z

Patch for L4.1.15 1.1.0 GA BSP: TW6865 driver for Linux L4.1.15_1.1.0-ga.7z

Labels (4)
Attachments
Comments

What's the CPU occupancy when run the application? Thanks.

Tested on my local side:

Test with Top  command:

Mem: 109772K used, 787076K free, 0K shrd, 3520K buff, 24616K cached

CPU: 12.2% usr 15.3% sys  0.0% nic 63.7% idle  0.0% io  0.0% irq  8.6% sirq

Load average: 7.02 5.16 2.56 4/94 3535

Test with MMDC tool:

root@freescale ~$ ./mmdc_prof

MMDC new Profiling results:

***********************

Total cycles count: 528091007

Busy cycles count: 385370313

Read accesses count: 9834967

Write accesses count: 10137409

Read bytes count: 466175272

Write bytes count: 379222896

  1. Read burst size: 47
  2. Write burst size: 37

Read: 444.58 MB/s /  Write: 361.66 MB/s  Total: 806.23 MB/s

Utilization: 13%

Bus Load: 72%

Bytes Access: 42

Hi JiangJustin

I am doing the same thing as you did above.

But my tw6865 card can't be identified. Bellow is what i have done.

1 .  I add the patch file to the kernel, version is L3.0.35_4.1.0_130816

2.   Change the kernel configure within make menuconfig to enable tw6865.

3.   Build the kernel and write it to my SD card. I am using i.mx6q sabreSDB board.

4.   Attach the mini-pcie=>PCIe adaptor and attch TW6865 card to adaptor's pcie slot.

5.   Boot the board, when system is up. use "lspci" to display the device. But there is nothing.

6.   Check the kernel boot message with "dmesg", there are errors as bellow:

iMX6 PCIe PCIe RC mode imx_pcie_pltfm_probe entering.

PCIE: imx_pcie_pltfm_probe start link up.

link up failed, DB_R0:0x004abc43, DB_R1:0x08000000!

IMX PCIe port: link down!

It seems like that link up was failed.

could you give me some advices to slove this?

Thank you very much.

Could you please share you .config file to us? We can check config file firstly. And please make sure the pcie card have been tightly inserted. Thanks.

Thank you very much.

i send my config file

and the pictures of my device i am using.

Hi,

Did you get my configure file?

I found that there is nothing on this page.

How can i send you the attach file? e-mail is ok?

No, I have not seen your config file. Maybe it will need some time for community administer’ s examination to attachment.

Please make sure you config file about pcie aligned to the following:

#

# MX6 Options:

#

CONFIG_IMX_PCIE=y

# CONFIG_IMX_PCIE_EP_MODE_IN_EP_RC_SYS is not set

# CONFIG_IMX_PCIE_RC_MODE_IN_EP_RC_SYS is not set

My configuration is same with yours.

Where you get the pcie switch to connect tw6865. It seems that not all of pcie switch can work  friendly. You can try some other pcie device to test. Actually, now we have mini-pcie interface daughter card for tw6865.

I buied the mini-pcie->pciex1 switch card from www.taobao.com.

and confirm it works with an intel gigabit ethernet card .

ethernet card can be recongnized by the board.

By the way,where can i buy the tw6865 daughter board you said?

Thank you

I am not sure if there is one compliance issue involved in you side. I would suggest you can try one more switch about it. 

I checked about the daughter card, it is just for internal use. 

Hi

Thank you, could you confirm the pcie clock's enable mentioned at bellow message in another discussion.

>Hongxing, thank you for your response.

>I use L3.0.35_4.1.0_130816_source.tar.gz downloaded from freescale's IMX6QSD platform resource website.

>Now, the problem has been solved. The PCIe clock output was not enabled in the code when IMX6 worked as RC.

PCIE link up failed when booting the kernel on IMX6QSD board

And when you did this things at begin, before use the daughter card,

which switch card you used? Can you give me an url of it where i can buy it from.

Thank you.

Or can i borrow the daughter card,

or borrow(buy) your old environment (mini-pcie2pcie switch card & tw6865 pcie card)

Thank you very much. 

If we enable CONFIG_IMX_PCIE_EP_MODE_IN_EP_RC_SYS item, the pice reference clock will not output by default , and it will work in separate clock mode. So we checked the .config file firstly.

If you can recognize other pcie device,  I think your issue  is not same with this. 

Thank you.

I tried another case.

I put the tw6865 card and the minipcie2pcie switch to the notepc's mini-pcie slot,

And it can be linked up.

If there is no other way, i plan to create the daughter board use your board data.

but it will take a long time.

If that is ok, may i borrow your daughter board? or i wish to buy one.

Thank you very much.

I checked the information we bought from the following link:

1. http://item.taobao.com/item.htm?spm=a1z10.1.w4004-5299295193.32.YwybCE&id=22108840583

2. The daughter card num is limited, I have not found one more available.

Did you mean that, the switch board you buied from bellow link is tested

ok with the tw6865 pci board?

http://item.taobao.com/item.htm?spm=a1z10.1.w4004-5299295193.32.YwybCE&id=22108840583

Thank you very much.

The information from hardware team, we bought at the beginning period.

But I am not sure if all of their devices can work, as you know ~~.

OK, I got it.

Thank you very much.

Hello! Thanks a lot for the post.

What was the maximum resolution you got on each of the cameras (relative to the CPU performance required to manage those image streams).

Thanks!

Bruno

For resolution of eache cameras, you can check tw68xx spec, I think.

We have tried with 6 cameras of D1, it can work well on imx6 platform.

Hello JiangJustin ,

Now I use Yocto project "fsl-yocto-3.10.17_1.0.0" & "fsl-community-bsp" with Sabre SD board and,I want to add "TW68" Patch file to linux-imx kernel(3.10.17-r0).

But I only have the patch file "pcie_cap_patch.patch.zip" from you, It's can't support Yocto linux kernal.

Would you please help me to make Patch file for Yocto project on linux kernel 3.10.17.

Thank you very much !! :smileyhappy:

Hi  simon yeh,

We should have created the patches for 3.10.17, so  I think you can contact your FAE to get it.

Hi b47504,

I got TW68xx driver form my FAE. Working properly. Thanks you !

Hi our Query is for Intersil TW6865

Data sheet says 4 realtime D1 video. Now  Real time video is ~ 30fps.

No where in datasheet it speaks about frame rate of simultaneous 4 channel with D1 resolution.

Please some body help us to know what will be the frame rate of simultaneous (fps) of each channel, when  4 channel is running simultaneously.

Thank You in Advance!

Based on my previous experience, 4 cameras can work together, and the application's preview speed of each camera can meet 30fps.   

Dear Mr. Zhao;

:smileyhappy:

Thanks for your info.

Regards;

Subrata

Hi,

As I know before, the daughter card should be just for internal use. I do not know the flow how the customer to get it. I think you can contact your FAE for the information. Thanks!

Hi,  everyone

     Now, we are designing the surrounding view hardware base on I.MX6 and  TW68xx. This hardware platform is Integrated into one PCB and support 6 camera most. Now, we are developing the software and Image processing algorithm。 So anyone want to Cooperate with us? we can share this PCB platform, and you can share me your software and Image processing algorithm? If possible, please contact me. email: lisim51@126.com

HI Yuan Zhao,

    do you support this demo now? could you send me your email address to my email lisim51@126.com?  I have some question for you.thanks.

hi :

    I used the same devices with you,and the devices can not be recongnized by the board(iMX6Q SabreSD board).

    Have you fixed the problem?if so,how did you do it ? or can you send the images to my email lxl2988@163.com.

Thank you very much.

Hi

Sorry, I didn’t fixed this problem.

I finally changed my way to create my own board.

Best regards..

I am doing the same thing! I use the tw6865 and  I can read the picture from the cameras.what confuse me is that I can not change the resolution of the caption.

I use V4L2 lib to config the camera,when it is configed with 720*576, i will get a full picture,just like this:

IMG_20151113_180740.jpg

when it is configed with other resolution,such as 360*288,640*480 and 320*240, i will get a Incomplete picture ,just like this:

IMG_20151113_175032.jpg

I use a definition to define the resolution:

#define  IMAGEWIDTH    720
#define  IMAGEHEIGHT   576

here is the configuration code:

int Camera_type ::init_v4l2()
{
    if ((camera.fd = open(camera.device_add, O_RDWR| O_NONBLOCK,0)) == -1)
    {
        printf("Error opening V4L interface\n");
        return (false);
    }
    //query cap
    if (ioctl(camera.fd, VIDIOC_QUERYCAP, &camera.cap) == -1)/* 获取设备支持的操作 */
    {
        printf("Error opening device %s: unable to query device.\n",camera.device_add);
        return (false);
    }
    else
    {
         printf("driver:\t\t%s\n",camera.cap.driver);
         printf("card:\t\t%s\n",camera.cap.card);
         printf("bus_info:\t%s\n",camera.cap.bus_info);
         printf("version:\t%d\n",camera.cap.version);
         printf("capabilities:\t%x\n",camera.cap.capabilities);
         if ((camera.cap.capabilities & V4L2_CAP_VIDEO_CAPTURE) == V4L2_CAP_VIDEO_CAPTURE)
         {
            printf("Device %s: supports capture.\n",camera.device_add);
        }
        if ((camera.cap.capabilities & V4L2_CAP_STREAMING) == V4L2_CAP_STREAMING)
        {
            printf("Device %s: supports streaming.\n",camera.device_add);
        }
    }
    //emu all support fmt
    camera.fmtdesc.index=0;
    camera.fmtdesc.type=V4L2_BUF_TYPE_VIDEO_CAPTURE;
    printf("Support format:\n");
    while(ioctl(camera.fd,VIDIOC_ENUM_FMT,&camera.fmtdesc)!=-1) /* 获取设置支持的视频格式 */
    {
        printf("\t%d.%s\n",camera.fmtdesc.index+1,camera.fmtdesc.description);
        camera.fmtdesc.index++;
    }
    //set fmt
    camera.fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    camera.fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;
    camera.fmt.fmt.pix.height = IMAGEHEIGHT;
    camera.fmt.fmt.pix.width = IMAGEWIDTH;
    camera.fmt.fmt.pix.field = V4L2_FIELD_INTERLACED;
    if(ioctl(camera.fd, VIDIOC_S_FMT, &camera.fmt) == -1)/* 设置捕获视频的格式 */
    {
        printf("Unable to set format\n");
        return false;
    }
    if(ioctl(camera.fd, VIDIOC_G_FMT, &camera.fmt) == -1)/* 获取设置的视频格式 */
    {
        printf("Unable to get format\n");
        return false;
    }
    {
         printf("fmt.type:\t\t%d\n",camera.fmt.type);
         printf("pix.pixelformat:\t%c%c%c%c\n",camera.fmt.fmt.pix.pixelformat & 0xFF, (camera.fmt.fmt.pix.pixelformat >> 8) & 0xFF,(camera.fmt.fmt.pix.pixelformat >> 16) & 0xFF, (camera.fmt.fmt.pix.pixelformat >> 24) & 0xFF);
         printf("pix.height:\t\t%d\n",camera.fmt.fmt.pix.height);
         printf("pix.width:\t\t%d\n",camera.fmt.fmt.pix.width);
         printf("pix.field:\t\t%d\n",camera.fmt.fmt.pix.field);
    }
    //set fps
    printf("init %s \t[OK]\n",camera.device_add);
    return true;
}

could you please tell me where i am wrong?

Hi!@

I am doing the same thing! I use the tw6865 and  I can read the picture from the cameras.what confuse me is that I can not change the resolution of the caption.

I use V4L2 lib to config the camera,when it is configed with 720*576, i will get a full picture,just like this:

IMG_20151113_180740.jpg

when it is configed with other resolution,such as 360*288,640*480 and 320*240, i will get a Incomplete picture ,just like this:

IMG_20151113_175032.jpg

I use a definition to define the resolution:

#define  IMAGEWIDTH    720
#define  IMAGEHEIGHT   576

here is the configuration code:

int Camera_type ::init_v4l2()
{
    if ((camera.fd = open(camera.device_add, O_RDWR| O_NONBLOCK,0)) == -1)
    {
        printf("Error opening V4L interface\n");
        return (false);
    }
    //query cap
    if (ioctl(camera.fd, VIDIOC_QUERYCAP, &camera.cap) == -1)/* 获取设备支持的操作 */
    {
        printf("Error opening device %s: unable to query device.\n",camera.device_add);
        return (false);
    }
    else
    {
         printf("driver:\t\t%s\n",camera.cap.driver);
         printf("card:\t\t%s\n",camera.cap.card);
         printf("bus_info:\t%s\n",camera.cap.bus_info);
         printf("version:\t%d\n",camera.cap.version);
         printf("capabilities:\t%x\n",camera.cap.capabilities);
         if ((camera.cap.capabilities & V4L2_CAP_VIDEO_CAPTURE) == V4L2_CAP_VIDEO_CAPTURE)
         {
            printf("Device %s: supports capture.\n",camera.device_add);
        }
        if ((camera.cap.capabilities & V4L2_CAP_STREAMING) == V4L2_CAP_STREAMING)
        {
            printf("Device %s: supports streaming.\n",camera.device_add);
        }
    }
    //emu all support fmt
    camera.fmtdesc.index=0;
    camera.fmtdesc.type=V4L2_BUF_TYPE_VIDEO_CAPTURE;
    printf("Support format:\n");
    while(ioctl(camera.fd,VIDIOC_ENUM_FMT,&camera.fmtdesc)!=-1) /* 获取设置支持的视频格式 */
    {
        printf("\t%d.%s\n",camera.fmtdesc.index+1,camera.fmtdesc.description);
        camera.fmtdesc.index++;
    }
    //set fmt
    camera.fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    camera.fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;
    camera.fmt.fmt.pix.height = IMAGEHEIGHT;
    camera.fmt.fmt.pix.width = IMAGEWIDTH;
    camera.fmt.fmt.pix.field = V4L2_FIELD_INTERLACED;
    if(ioctl(camera.fd, VIDIOC_S_FMT, &camera.fmt) == -1)/* 设置捕获视频的格式 */
    {
        printf("Unable to set format\n");
        return false;
    }
    if(ioctl(camera.fd, VIDIOC_G_FMT, &camera.fmt) == -1)/* 获取设置的视频格式 */
    {
        printf("Unable to get format\n");
        return false;
    }
    {
         printf("fmt.type:\t\t%d\n",camera.fmt.type);
         printf("pix.pixelformat:\t%c%c%c%c\n",camera.fmt.fmt.pix.pixelformat & 0xFF, (camera.fmt.fmt.pix.pixelformat >> 8) & 0xFF,(camera.fmt.fmt.pix.pixelformat >> 16) & 0xFF, (camera.fmt.fmt.pix.pixelformat >> 24) & 0xFF);
         printf("pix.height:\t\t%d\n",camera.fmt.fmt.pix.height);
         printf("pix.width:\t\t%d\n",camera.fmt.fmt.pix.width);
         printf("pix.field:\t\t%d\n",camera.fmt.fmt.pix.field);
    }
    //set fps
    printf("init %s \t[OK]\n",camera.device_add);
    return true;
}

could you please tell me where i am wrong?

Hi~Dear

     Now I encounted some problems when i use the patch in IMX6 Community.

     My Applitcatin is mxc_v4l2_tvin.out. But It can't use V4L2_MEMORY_USERPTR mode to capture video stream,and it often displayed that "USERPTR is currently not supported ,buffer_prepare  OOPS".And it burn down.But when i use V4L2_MEMORY_MMAPED,it can normally capture video stream.

    

The command is :

hdmi

./mxc_v4l2_tvin.out -x 0 -ot 0 -ol 0 -ow 960 -oh 540 -m 2 &

./mxc_v4l2_tvin.out -x 1 -ot 0 -ol 960 -ow 960 -oh 540 -m 2 &

./mxc_v4l2_tvin.out -x 2 -ot 540 -ol 0 -ow 960 -oh 540 -m 2 &

./mxc_v4l2_tvin.out -x 3 -ot 540 -ol 960 -ow 960 -oh 540 -m 2 &

hardware platform: IMX6Q-SABRESED + PCIE + TW6865

LTIB: L3.0.35_4.1.0_130816_source.tar.gz

Hi,

Could you teach me that how to save the data captured by the lens?

Have a good day!

Hi hu bing,

    What about your problem of capture ?

     It is kind that help me a demo about V4L2 code.There is my email 1058477863@qq.com.

     Thanks!

Hi, Qingrong Jiang!

I'm a FAE from STC, one of the NXP distributors in Korea.
We made TW6865 test board, and we can see 4ch image on HDMI with MX6Q-SABRESDP.
But we can see afterimage when I move my hand fast in front of Camera. (Please see attached video clip.)

(BSP version : L3.0.35_4.1.0)
I met intersil FAE, and he said it is the problem of deinterlacing.
Do you have any guide or solution to solve this issue?

SVM.jpg

Dear Qingrong Jiang​:

Hi

When I use this methord to get the data of images by the v4l2 capture.c,

I meet like this:

tw.png

i.MX6Q +PCIE+TW68 驱动显示等问题-恩智浦MCU论坛论坛 - 21ic电子技术论坛

can you check with our Korea FAE. They should have some patches for it.

We made TW6865 test board, and we can see 4ch image on HDMI with MX6Q-SABRESDP.
But we can see raster images when I move my hand fast in front of Camera.

Dear JiangJustin ,

We have developed a camera board using Intersil TW6865 chip (As same as Freescale TW6865 miniPCIe Daughter Board) which can input two analog cameras to iMAX6Q Sabre Lite Development board using PCIe interface. 

I need to clarify following >>

In iMAX6Q Sabre Lite for PCIe port there are 7 pin connections as follows. +3.3V, PCIE_RXM, PCIE_RXP, PCIE_TXM, PCIE_TXP, GND, GND.

pastedImage_1.png

Intersil TW6865 chip has REFCLK-, REFCLK+ pin connections for differential reference clock inputs.

pastedImage_5.png

Don't we need to connect this pins with iMAX6Q board if we want to interface with PCIe interface ?

Thank You,

Peter Amond.imx6_pcie_msi_handler

Hi, Justin Jiang .

Is it possible to use opencv to make surround view come true?

Our i.MX6 doesn't support OpenCV HW acceleration.OpenGL ES, OpenVG, OpengCL are supported.

We have less experience about the surround view stitching implementation. I think you can try to use it, but it rely on CPU's compute capability. I'm not sure if the performance is good enough.

Hi, Justin Jiang .

My board has a lvds lcd, now it is able to display the qt demo, I have also attached the tw68 to the board. The "lspci" command output is below:

00:00.0 PCI bridge: Synopsys, Inc. Device abcd (rev 01)
01:00.0 Non-VGA unclassified device: Intersil Techwell Device 6869 (rev 01)

So, is the command below able to display the video on the lvds lcd? If not, how to change it.

gst-launch v4l2src device=/dev/video0  ! 'video/x-raw-yuv, format=(fourcc)UYVY' ! mfw_isink disp-width=960 disp-height=540 &

gst-launch v4l2src device=/dev/video1  ! 'video/x-raw-yuv, format=(fourcc)UYVY' ! mfw_isink disp-width=960 disp-height=540 axis-left=960 &

gst-launch v4l2src device=/dev/video2  ! 'video/x-raw-yuv, format=(fourcc)UYVY' ! mfw_isink disp-width=960 disp-height=540 axis-top=540  &

gst-launch v4l2src device=/dev/video3  ! 'video/x-raw-yuv, format=(fourcc)UYVY' ! mfw_isink disp-width=960 disp-height=540 axis-top=540 axis-left=960 &

BTW, my kernel version is 4.1.15.

Thanks!

I don't know your display resolution, you need change it according your LVDS display resolution. Like 1024x768 in our SDP board, change it to below:

gst-launch v4l2src device=/dev/video0  ! 'video/x-raw-yuv, format=(fourcc)UYVY' ! mfw_isink disp-width=512 disp-height=384 &

gst-launch v4l2src device=/dev/video1  ! 'video/x-raw-yuv, format=(fourcc)UYVY' ! mfw_isink disp-width=512 disp-height=384 axis-left=512 &

gst-launch v4l2src device=/dev/video2  ! 'video/x-raw-yuv, format=(fourcc)UYVY' ! mfw_isink disp-width=512 disp-height=384 axis-top=384 &

gst-launch v4l2src device=/dev/video3  ! 'video/x-raw-yuv, format=(fourcc)UYVY' ! mfw_isink disp-width=512 disp-height=384 axis-top=384 axis-left=512 &

Thank you for your reply!

There are "tw68-core.c" "tw68-video.c"... files in the linux 4.1.15 source tree, I'm not sure whether they are working for tw6865, I have enabled the tw68 in kernel, but there is no corresponding /dev/videoX for the tw6865. So I want confirm that is this the driver for tw6865.

The source code is under "/drivers/media/pci/tw68/" directory in the linux 4.1.15 source tree.

--------------------------------------------------

I ask the tw68 driver author, it's not for the tw6865.

Tw6865 driver is here:

GitHub - sasamy/tw6869: tw6869/65 media bridge driver 

Hi Lei Ma,

try dmesg | grep tw68 and see whether you have a dma memory allocation error message. If yes, try passing coherent_pool = 128M (or a reasonable memory size) as a kernel parameter. i.e. setenv bootargs = "${bootargs} coherent_pool = 128M" in uboot or 6x_bootscript. This will enumerate your video device if the issue was with memory allocation. The mainline tw686x linux driver should work for your case !

Rgs

Anuradha

HI。

Can it working on Android4.4.2, if it can  ,is there any docs?

thanks!

Hi,

I'm using the tw686x driver, but the gst-lanuch-1.0 doesn't work, I try the

"gst-launch-1.0 imxv4l2src device=/dev/video2" , output:

====== IMXV4L2SRC: 4.1.4 build on Jan 31 2017 20:15:40. ======
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...

(gst-launch-1.0:990): GStreamer-CRITICAL **: gst_mini_object_ref: assertion 'mini_object != NULL' failed

** (gst-launch-1.0:990): CRITICAL **: gst_video_info_from_caps: assertion 'caps != NULL' failed

(gst-launch-1.0:990): GStreamer-CRITICAL **: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed

** (gst-launch-1.0:990): CRITICAL **: gst_video_info_from_caps: assertion 'caps != NULL' failed
ERROR: from element /GstPipeline:pipeline0/GstImxV4l2Src:imxv4l2src0: Internal data flow error.
Additional debug info:
../../../../gstreamer-1.8.1/libs/gst/base/gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline0/GstImxV4l2Src:imxv4l2src0:
streaming task paused, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

"dmesg" outputs:

ERROR: v4l2 capture: slave not found!

If I use "imxv4l2videosrc", it'll output:

ERROR: pipeline could not be constructed: no element "imxv4l2videosrc"

 

version:
gst-launch-1.0 version 1.8.1
GStreamer 1.8.1

=======================================================

If I use the old rootfs for L3.0.5, but with the same kernel used above, 4.1.15, the output is correct, command:

gst-launch v4l2src device=/dev/video0 ! 'video/x-raw-yuv, format=(fourcc)UYVY' ! mfw_isink disp-width=512 disp-height=300 axis-left=0 axis-top=0

gst-launch version:

gst-launch-0.10 version 0.10.35
GStreamer 0.10.35

Version history
Last update:
‎07-17-2013 01:19 AM
Updated by: