i.MAX6 Sabre Lite / PCIE REFCLK / Intersil TW6865

[peteramond]

richard.zhu‌EricNelson‌Leo‌

Hi All,

I'm having trouble getting the PCIE up on the i.MAX6 Quad Sabre Lite Development board (https://boundarydevices.com/product/sabre-lite-imx6-sbc/ ) and interfacing with TW6865 Intersil video chip and I couldn't find the sufficient details on how to connect REFCLK- and REFCLK+ of TW6865 with PCIE port of the i.MAX6Q Sabre Lite as it just have following pin connections. 

1. Can you guide me to interface TW6865 with PCIE of Sabre Lite ? (Presently I'm working 3.14.52 kernel based system)
2. When I check the schematic of Sabre lite board I could find there is no pin connection for CLK1_N and CLK1_P (C7 & D7 pins of processor) for PCIe clock. So it seems that we need to generate reference clock externally as it has done in PCIe Daughter board ( https://boundarydevices.com/product/nit6x_pcie/ ). If I need to build external clock what is the frequency should I generate for PCIE ? (Because I'm not using PCIE daughter board) Can you trll me the frequency of the on board oscillator of PCIE Daughter board ?
3. If I'm using 3.14.52 kernel what are the configurations should I follow ?

Thank You,

Peter.

[tengri]

Hi gary_bisson‌ and @mike chin 

I bought the BD nitx PCIE daughter card and tried to interface my custom made (TW6865) video capture card. The recommended schematic for the card from freescale is attached herewith. I have couple of questions about interfacing signals to the board. 

1. Do  following connections seem correct ?

TWCard          Nitrogen 6x

RXM      -->     TXM

RXP       -->    TXP

TXP       -->     RXP

TXM       -->     RXM

RefClkM -->    RefClkM

RefClkP -->    RefClkP

2. If the test is done between TW card and SabreLite (where the refclock is generated by on-board oscillator of the daughter card) what is the correct configuration of PCIE settings ? should Forcegen 1 be enabled or disabled ? And is RC mode of PCIE enabled ?

3. How the PCIe Reset signal (of TW chip) should be used ? 

Thanks You

[gary_bisson]

Hi Peter,

1 & 3. Our 3.14.52 kernel already includes the TW686x driver in the default configuration, should just work:

linux-imx6/boundary_defconfig at boundary-imx_3.14.52_1.1.0_ga · boundarydevices/linux-imx6 · GitHub 

2. The clock generated on the daughter board is a 100MHz.

Regards,

Gary

[tengri]

Dear Gary, 

We were able to detect the custom made pcie board on nitrogen6_max board without any problem. One more thing, the only available driver in the kernel related to this chip is 'tw686x' as a module (this was set in make menuconfig by default). To get he video up, should I modprobe the module ? I can see it under lspci :

00:00.0 PCI bridge: Synopsys, Inc. Device abcd (rev 01)
01:00.0 Multimedia video controller: Intersil Techwell Device 6869 (rev 01)

but not /dev/video devices found in the system ! I am using 4.0.15 kernel - Ubuntu Xenial

Thanks 

Anuradha

[gary_bisson · Accepted solution]

Hi,

Good news!

There are actually two drivers in our kernel for that chip:

CONFIG_VIDEO_TW686X & CONFIG_VIDEO_TW68

Note that we recommend using the mainline driver (CONFIG_VIDEO_TW686X).

Also, we've just backported a bunch of patches from mainline, so please make sure to use the latest:

Commits · boundarydevices/linux-imx6 · GitHub 

As for modprobing, I would expect the driver to get loaded automatically, if not yes please do try to manually modprobe.

Regards,

Gary

[tengri]

Dear gary_bisson,

I added latest patches and rebuilt kernerl around 4.1.15.2, and still I have the issue of enumerating a video device. And I can see :

[    6.800972] tw686x 0000:01:00.0: dma0: unable to allocate P-buffer

error in dmesg. An analog camera (NTSC) is connected to video input of the chip. Seems like the problem is in dma memory allocation. Can we set this in uboot as a boot arg ?

Thanks

[gary_bisson]

Hi,

How much CMA memory is allocated in the kernel? How much memory are you asking for?

Regards,

Gary

[tengri]

Hi, yes ! by increasing the coherent_pool size in bootargs I was able to solve the dma issue !

But now the video device does not seem to give any video outputs, to the following gst commands, I only see a black imxeglvivsink window. But no pipeline errors shown in the terminal :

gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,width=720,height=480,pixelformat=UYVY ! imxipuvideotransform ! imxeglvivsink

gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,width=720,height=480,pixelformat=UYVY ! queue ! pngenc ! multifilesink location="/home/ubuntu/Pissu/frame%d.png"

Rgs

Anuradha

[gary_bisson]

Hi tengri‌,

I've seen you created another post:

TW6865 PCI Interface on Nitrogen6_Max 

It does not make sense to maintain 2 posts for the same question.

Please close this one and mark it as answered.

Regards,

Gary

[tengri]

Hi, there is a bit of instability in pcie link up and device detection. Sometimes kernel hangs at pcie reset probe stage (before proceeding to link up state). Apart from the hardware connections, what should be in our checklist (U-boot patches to make the link stable) ? pci = nomsi does not affect much...... 

Rgs

Anuradha

[gary_bisson]

Hi,

Since it doesn't work with the SabreLite and is unstable with the Nitrogen6_MAX, I guess there is some electrical issue with your PCIe board.

All the off-the-shelf PCIe modules we have tested are working fine on both boards.

But note that Nitrogen6_MAX is more tolerant than SabreLite towards PCIe signals since it is routed on the PCB direclty whereas SabreLite uses a daughter board through a cable.

Regards,

Gary

[tengri]

Hi,

I was able to make the link stable now. As you stated, the problem was with the wiring. So should this mainline driver enumerate a /dev/video0 ? After device has been detected, I checked all the video devices, but non of them have been attached to the tw68 chip. Am I missing something here ?  I can see following in lsmod :

tw686x 28672 0
videobuf2_vmalloc 16384 1 tw686x
videobuf2_memops 16384 1 videobuf2_vmalloc

Further I can see following in dmesg :

[ 2.597402] pci 0000:01:00.0: Setting PCI class for tw6869 PCIe device
[ 16.060908] tw6869: PCI 0000:01:00.0, IRQ 349, MMIO 0x1100000
[ 16.060977] tw686x 0000:01:00.0: enabling device (0140 -> 0142)
[ 16.062171] tw686x 0000:01:00.0: dma0: unable to allocate P-buffer
[ 16.110888] Modules linked in: tw686x(+) videobuf2_vmalloc videobuf2_memops galcore(O+) gpio_keys
[ 16.449337] [<80857520>] (_raw_spin_lock_irqsave) from [<7f05f470>] (tw686x_free_dma+0x24/0xc0 [tw686x])
[ 16.459893] [<7f05f44c>] (tw686x_free_dma [tw686x]) from [<7f060338>] (tw686x_video_free+0x50/0x70 [tw686x])
[ 16.472891] [<7f0602e8>] (tw686x_video_free [tw686x]) from [<7f060864>] (tw686x_video_init+0x50c/0x6d4 [tw686x])
[ 16.486232] [<7f060358>] (tw686x_video_init [tw686x]) from [<7f05e6c8>] (tw686x_probe+0x36c/0x3fc [tw686x])
[ 16.502566] [<7f05e35c>] (tw686x_probe [tw686x]) from [<8043edd4>] (pci_device_probe+0x7c/0xd0)
[ 16.595548] [<8043e4d0>] (__pci_register_driver) from [<7f066024>] (tw686x_pci_driver_init+0x24/0x1000 [tw686x])
[ 16.604440] [<7f066000>] (tw686x_pci_driver_init [tw686x]) from [<80101760>] (do_one_initcall+0xa4/0x200)

Thanks in Advance

[tengri]

Hello Gary, in most of occasions, I can see following message in the kernel log :

AppArmor: AppArmor sha1 policy hashing enabled
imx6_pcie_probe: reset gp 191
imx6q-pcie 1ffc000.pcie: Link up, Gen=1
imx6q-pcie 1ffc000.pcie: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io 0x1000-0xffff]
pci_bus 0000:00: root bus resource [mem 0x01000000-0x01efffff]
pci_bus 0000:00: root bus resource [bus 00-ff]
PCI: bus0: Fast back to back transfers disabled
PCI: bus1: Fast back to back transfers enabled
pci 0000:00:00.0: BAR 0: assigned [mem 0x01000000-0x010fffff]
pci 0000:00:00.0: BAR 6: assigned [mem 0x01100000-0x0110ffff pref]
pci 0000:00:00.0: PCI bridge to [bus 01]
pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt

and for lspci :

root@nitrogen:~# lspci
00:00.0 PCI bridge: Synopsys, Inc. Device abcd (rev 01)

no sign of techwell card info. Any remedies you think we can use ? If we come to this stage I think the electrical connections seem to be okay right ? Btw I am using main line kernel module as you suggested. lsmod shows -

galcore 225280 10
gpio_keys 20480 0
tw686x 28672 0
videobuf2_vmalloc 16384 1 tw686x
videobuf2_memops 16384 1 videobuf2_vmalloc

Thanks

[tengri]

Hi,

Peter was referring to the same project I am working on. So we have this PCIE link never came up issue in the kernel log and as he stated we are interfacing TW6568 with nitrogen 6q board (which has dedicated clk1N/P from imx6q for the pcie interface). I checked the kernel sources : boundary_defconfig, and .config files, so following configurations are on:

> in defconfig :

CONFIG_PCI=y
CONFIG_PCI_MSI=y
CONFIG_PCI_IMX6=y

CONFIG_MEDIA_PCI_SUPPORT=y
CONFIG_VIDEO_TW686X=m

> the .conf file has been attached herewith 

[Bash] imx6 .conf - Pastebin.com 

> Kernel Log Message :

imx6q-pcie 1ffc000.pcie: phy link never came up
imx6q-pcie 1ffc000.pcie: Failed to bring link up!
imx6q-pcie 1ffc000.pcie: failed to initialize host
imx6q-pcie: probe of 1ffc000.pcie failed with error -22

Does this have to do something with pcie timing ? How the PCIE reset signal should be fed to the chip ? I'll try adding following patch to the kernel, apart from that what should be in our checklist (specifically in dts, uboot) ?

IMX6 PCIe Problem 

Thanks in advance !

[gary_bisson]

Hi,

First of all, can you specify which daughter board you are using? Please do all the testing on our daughter board first, then you can later switch to yours.

Then, I can see 2 things you can try to solve this:

1. Add 'pci=nomsi' to your kernel bootargs, some modules require this

2. If 1. doesn't work, can you try selecting PCIE_FORCE_GEN1 via menuconfig and try again both with and without pci=nomsi?

Regards,

Gary

[tengri]

Hello Gary, 

Hi, for testing purposes now we are using a pcie TPLINK network card. Could you please give us a comment on following settings in our .conf file :

1. I tried nomsi in uboot as you suggested, but in .conf  CONFIG_PCI_MSI is set to one. 

#
# Bus support
#
CONFIG_PCI=y
CONFIG_PCI_DOMAINS=y
CONFIG_PCI_SYSCALL=y
CONFIG_PCI_MSI=y
# CONFIG_PCI_DEBUG is not set
# CONFIG_PCI_REALLOC_ENABLE_AUTO is not set
# CONFIG_PCI_STUB is not set
# CONFIG_PCI_IOV is not set
# CONFIG_PCI_PRI is not set
# CONFIG_PCI_PASID is not set

2. What should be the correct settings for pcie host controller ? For this particular application, the imx6 should be in root complex mode right ? 

#
# PCI host controller drivers
#
CONFIG_PCIE_DW=y
CONFIG_PCI_IMX6=y
# CONFIG_PCIE_FORCE_GEN1 is not set
# CONFIG_EP_MODE_IN_EP_RC_SYS is not set
# CONFIG_RC_MODE_IN_EP_RC_SYS is not set
# CONFIG_PCI_HOST_GENERIC is not set
CONFIG_PCIEPORTBUS=y
CONFIG_PCIEAER=y
# CONFIG_PCIE_ECRC is not set
# CONFIG_PCIEAER_INJECT is not set
CONFIG_PCIEASPM=y
# CONFIG_PCIEASPM_DEBUG is not set
CONFIG_PCIEASPM_DEFAULT=y
# CONFIG_PCIEASPM_POWERSAVE is not set
# CONFIG_PCIEASPM_PERFORMANCE is not set
CONFIG_PCIE_PME=y
# CONFIG_PCCARD is not set

thanks 

[gary_bisson]

Hi,

1. Yes that is the whole point of the bootargs, not having to modify the configuration but still disable MSI. Please revert your configuration change. Also, make sure the bootargs is actually set by looking at /proc/cmdline.

2. As said before, try the GEN1 option.

Regards,

Gary

[qqhhqqhh]

Hi,

I add some delays here ,and everything works fine.

hope for help.

drivers/pci/host/pci-imx6.c

static int imx6_pcie_wait_for_link(struct pcie_port *pp)
{
int count = 20000;
struct imx6_pcie *imx6_pcie = to_imx6_pcie(pp);

while (!dw_pcie_link_up(pp)) {
udelay(200);
if (--count)
continue;

dev_err(pp->dev, "phy link never came up\n");
dev_dbg(pp->dev, "DEBUG_R0: 0x%08x, DEBUG_R1: 0x%08x\n",
readl(pp->dbi_base + PCIE_PHY_DEBUG_R0),
readl(pp->dbi_base + PCIE_PHY_DEBUG_R1));

if (!IS_ENABLED(CONFIG_PCI_IMX6_COMPLIANCE_TEST)) {
clk_disable_unprepare(imx6_pcie->pcie);
clk_disable_unprepare(imx6_pcie->pcie_bus);
clk_disable_unprepare(imx6_pcie->pcie_phy);
if (is_imx6sx_pcie(imx6_pcie))
clk_disable_unprepare(imx6_pcie->pcie_inbound_axi);
release_bus_freq(BUS_FREQ_HIGH);
if (imx6_pcie->pcie_phy_regulator != NULL)
regulator_disable(imx6_pcie->pcie_phy_regulator);
}
return -EINVAL;
}

return 0;
}

[tengri]

Hello Gary,

I tried following steps to check whether the issue remains, unfortunately it does :

1. Try switching pci=nomsi in boot args. The value was confirmed in cat /proc/cmdline. 

cpu=6Q board=nitrogen6x mxc_hdmi.only_cea=1 console=ttymxc1,115200 vmalloc=400M consoleblank=0 rootwait fixrtc pci=nomsi root=/dev/mmcblk0p1 galcore.contiguousSize=67108864 splash quiet plymouth.ignore-serial-consoles

2. Build kernel with different pcie configurations :

#
# PCI host controller drivers
#
CONFIG_PCIE_DW=y
CONFIG_PCI_IMX6=y
CONFIG_PCIE_FORCE_GEN1=y                                    #tried setting and not setting
# CONFIG_EP_MODE_IN_EP_RC_SYS is not set
CONFIG_RC_MODE_IN_EP_RC_SYS=y                        #tried setting and not setting
CONFIG_PCI_IMX_EP_DRV=y
# CONFIG_PCI_HOST_GENERIC is not set
CONFIG_PCIEPORTBUS=y
CONFIG_PCIEAER=y
# CONFIG_PCIE_ECRC is not set
# CONFIG_PCIEAER_INJECT is not set
CONFIG_PCIEASPM=y
# CONFIG_PCIEASPM_DEBUG is not set
CONFIG_PCIEASPM_DEFAULT=y
# CONFIG_PCIEASPM_POWERSAVE is not set
# CONFIG_PCIEASPM_PERFORMANCE is not set
CONFIG_PCIE_PME=y
# CONFIG_PCCARD is not set

3. Added the delays to pcie functions in pci-imx6.c as in :

IMX6 PCIe Problem 

and tried all above attempts. But the following messages seem to be recurrent !

imx6q-pcie 1ffc000.pcie: phy link never came up
imx6q-pcie 1ffc000.pcie: Failed to bring link up!
imx6q-pcie 1ffc000.pcie: failed to initialize host

4. Checked the connections with TP-Link TG-3468 ethernet adapter. Connections are as follows :

TP               Nitrogen 6x

RXM    -->    TXM

RXP    -->    TXP

TXP    -->    RXP

TXM    -->    RXM

RefClkM --> RefClkM

RefClkP --> RefClkP

+ supply rails accordingly.

I suspect one more thing, the dts configuration ! Could you please also share your dts content related to pcie node ? I am currently using the pcie node configurations mentioned above (my 2nd comment).

Thanks

Anuradha

[gary_bisson]

Hi,

I use the device tree that is present in our kernel with no modification and it works for all the devices I've tried. I won't be able much more since there's no hardware available.

Regards,

Gary

[tengri]

Hi, it's fine I'll try repeating steps back and forth. You've been helpful all the time, thanks for it.....