i.MX6 PCIe I/O problem

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

i.MX6 PCIe I/O problem

3,721 Views
lmendes
Contributor III

Hi,

I'm trying to use a WCH WC382 2 Serial + 1 Parallel printer card port on an i.MX6 Quad. The card is enumerated and device driver loads, however device driver is unable to probe the parallel port hardware.

[    4.287438] lp: driver loaded but no devices found
[    4.301532] ppdev: user-space parallel port driver

The serial ports are probed:

[    5.895525] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    5.906105] serial 0000:01:00.0: enabling device (0140 -> 0143)
[    5.914874] 0000:01:00.0: ttyS0 at I/O 0x10c0 (irq = 317, base_baud = 115200) is a XR16850
[    5.915934] 0000:01:00.0: ttyS1 at I/O 0x10c8 (irq = 317, base_baud = 115200) is a XR16850

However I'm unable to send or receive data through them, I/O accesses do not work.

Serial Parport kernel module always read back as 0 for PCIe I/O access with inb() and outb(), while PCIe memory accesses work fine.

How can I access PCIe I/O resources?

 

I've also tried to access the card from /sys/bus/pci/devices/0000:01:00.0/resource2 which corresponds to the I/O ports at 1400 [size=4], but I get the following (mmap fails):

sudo ./pcimem /sys/bus/pci/devices/0000\:01\:00.0/resource2 0x0 b
/sys/bus/pci/devices/0000:01:00.0/resource2 opened.
Target offset is 0x0, page size is 4096 map mask is 0xFFF
mmap(0, 4096, 0x3, 0x1, 3, 0x0)
PCI Memory mapped ERROR.
Error at line 90, file pcimem.c (22) [Invalid argument]

 

Similarly I/O resource0 produces the same outcome.

 

However the memory resource1 Memory at 01100000 (32-bit, prefetchable) [size=32K] again work fine:

sudo ./pcimem /sys/bus/pci/devices/0000\:01\:00.0/resource1 0x0 b
/sys/bus/pci/devices/0000:01:00.0/resource1 opened.
Target offset is 0x0, page size is 4096 map mask is 0xFFF
mmap(0, 4096, 0x3, 0x1, 3, 0x0)
mmap(0, 4096, 0x3, 0x1, 3, 0x0)
PCI Memory mapped 4096 byte region to map_base 0x76f3e000.
PCI Memory mapped access 0x 76F3E000.
Read 0x7F

 

The pcimem program source code follows in attachment.

 

00:00.0 PCI bridge: Synopsys, Inc. Device abcd (rev 01) (prog-if 00 [Normal decode])
    Flags: bus master, fast devsel, latency 0, IRQ 317
    Memory at 01000000 (32-bit, non-prefetchable) [size=1M]
    Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
    I/O behind bridge: 00001000-00001fff
    Prefetchable memory behind bridge: 01100000-011fffff
    [virtual] Expansion ROM at 01200000 [disabled] [size=64K]
    Capabilities: [40] Power Management version 3
    Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
    Capabilities: [70] Express Root Port (Slot-), MSI 00
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [140] Virtual Channel
    Kernel driver in use: pcieport

 

01:00.0 Serial controller: Device 1c00:3250 (rev 10) (prog-if 05 [16850])
    Subsystem: Device 1c00:3250
    Flags: fast devsel, IRQ 317
    I/O ports at 1000 [size=256]
    Memory at 01100000 (32-bit, prefetchable) [size=32K]
    I/O ports at 1400 [size=4]
    [virtual] Expansion ROM at 01108000 [disabled] [size=32K]
    Capabilities: [60] Power Management version 3
    Capabilities: [68] MSI: Enable- Count=1/32 Maskable+ 64bit+
    Capabilities: [80] Express Legacy Endpoint, MSI 00
    Capabilities: [100] Advanced Error Reporting
    Kernel driver in use: parport_serial
    Kernel modules: 8250_pci, parport_serial

 

I've also seen related topic https://community.nxp.com/message/630870?commentID=630870#comment-630870 , but nothing worked.

Original Attachment has been moved to: pcimem.c.zip

Labels (3)
9 Replies

2,392 Views
lmendes
Contributor III

To read PCI IO Resource at BAR2, do:

$ hexdump /sys/bus/pci/devices/0000\:01\:00.0/resource2

0000000 0000 0000

0000004

Still all data reads as 0....

0 Kudos

2,392 Views
Yuri
NXP Employee
NXP Employee

Hello,

  Please check first the PCIe clock configuration.   HW_Design_Checking_List_for_i.MX6DQP6DQ6SDL Rev 3.1

contains useful recommendations about  using (external) PCIe clock :
 "Due to CLKx_P/N is LVDS port and don't match with PCIe reference clock specification. For PCIe Gen1
application, following low cost solution can be used(DC bias and AC impedance should be considered).  
Please refer to "HW_Design_Checking_List_for_i.MX6DQP6DQ6SDL Rev3.1.xlsx", sheet "Schematic",
Ref12 for more info."
   "PCIe reference clock solution which provided by CLKx_N/P of i.MX6 chip can't pass PCIe Gen2
compliance test.  Recommend using external PCIe 2.0/3.0 clock generator with 2 HCSL outputs solution.
One clock channel connect to i.MX6 as a reference input, please click Ref14
("HW_Design_Checking_List_for_i.MX6DQP6DQ6SDL Rev3.1.xlsx") for reference circuit.
Another clock channel should connect to PCIe connector, please contact generator vendor for detailed
design guide."
 
https://community.nxp.com/docs/DOC-93819 

  Also, please refer to  the following IMX6 PCI with external clocks  

Have a great day,
Yuri

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

2,392 Views
lmendes
Contributor III

Hi Yuri,

Thanks for your help.

Sorry for the delayed reply, I've been pretty busy lately.

I was able to partially solve the PCIe I/O problem... that is, I'm now able to get the Parallel port to work, but not the controller Serial port.

I've also tried SATA controller cards like the Promise FastTrak TX2650 and I'm now able to get the drivers to load and detect the card, but it always fails identifying the connected HDD. I will have to check your suggestion, because this may now be caused by PCIe having high error rates due to poor quality clock signal, as you suggested. It sometimes looks like the whole Linux becomes sluggish when the SATA controller card is connected with a hard drive.

There is something else however that was/is causing trouble with the PCIe I/O accesses and I'm not completely sure what it is, because it is quite sensitive to kernel configuration at least.

I've tried Linux kernels 4.1.15 and Vanilla Kernel 4.9.30... the Vanilla kernel is easier to get PCIe I/O to work it seems, however at some point I got both kernels to work with some extra kernel code changes, and then I lost it again.

For Vanilla Kernel 4.9.30 I have PCIe I/O working and I know how to do it.

The changes I've made are the following:

It looks like that the PCIe I/O wont't work at all when the CONFIG_OVERLAYS is enabled (Device drivers->Device Tree and Open Firmware support->Device Tree Overlays->Disable), thus it is essential to disable it.

This changes alone seems to produce this result alone:

[    5.522508] lp: driver loaded but no devices found
[    5.542818] ppdev: user-space parallel port driver

[    8.741997] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    8.754821] serial 0000:01:00.0: enabling device (0140 -> 0143)

[    8.795862] parport0: PC-style at 0x1400, irq 338 [PCSPP,TRISTATE]
[    8.894325] lp0: using parport0 (interrupt-driven).
[    8.959846] 0000:01:00.0: ttyS0 at I/O 0x10c0 (irq = 338, base_baud = 115200) is a XR16850
[    8.980616] 0000:01:00.0: ttyS1 at I/O 0x10c8 (irq = 338, base_baud = 115200) is a XR16850

Then, since I want to use inb() and outb() from user space, I've also enabled (General setup->Configure standard Kernel features (expert users)->Sysctl syscall support->Enable) to enable the syscall system cal.

Then I edited arch/arm/mach-imx/mach-imx6q.c, function __init imx6q_init_late(void) and appended:

   register_isa_ports(0xfee0000, 0x1f80000, 0);

With this I was able to use the controller card Parallel port from user space code.

For kernel 4.1.15 If I do similar changes, the device driver is failing to probe the parallel port and doesn't load the driver.

I'm trying to redo the extra code changes I did for this kernel because at some point it worked and was able to load the driver and work just like Vanilla Kernel 4.9.30.

Luís

2,392 Views
Yuri
NXP Employee
NXP Employee

Hello,

  generally kernel 4.9 (of NXP BSP) is expected relatively soon.

Regards,

Yuri.

2,392 Views
krzysztofdrobi_
Contributor I

Yuri Muhin wrote:

Hello,

 

  generally kernel 4.9 (of NXP BSP) is expected relatively soon.

 

Regards,

Yuri.

YuriMuhin_ng‌ Could you share more / less when linux 4.9 for NXP BPSs is expected? Is this release still going to support i.mx6q?

Regards,

Krzysztof

0 Kudos

2,392 Views
Yuri
NXP Employee
NXP Employee

Hello,

  please create request / ticket regarding NXP BSP schedule.

Support|NXP  

Regards,

Yuri.

0 Kudos

2,392 Views
lmendes
Contributor III

Linux Kernel ARM port does not allow accessing PCI I/O resources in SYSFS (including Kernel 4.1.15), contrary to what is suggested in https://community.nxp.com/message/630870?commentID=630870#comment-630870, because:

/drivers/pci/pci-sysfs.c - function static int pci_mmap_resource(struct kobject *kobj, struct bin_attribute *attr,
                 struct vm_area_struct *vma, int write_combine)

calls pci_mmap_page_range(pdev, vma, mmap_type, write_combine);

with mmap_type=pci_mmap_io when PCI RESOURCE is IO...

which for ARM is defined in /arch/arm/kernel/bios32.c, like this:

int pci_mmap_page_range(struct pci_dev *dev, struct vm_area_struct *vma,
            enum pci_mmap_state mmap_state, int write_combine)
{
    if (mmap_state == pci_mmap_io)
        return -EINVAL;

    /*
     * Mark this as IO
     */
    vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);

    if (remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff,
                 vma->vm_end - vma->vm_start,
                 vma->vm_page_prot))
        return -EAGAIN;

    return 0;
}

0 Kudos

2,392 Views
lmendes
Contributor III

Attempts at accessing PCI I/O space from kernel module...

Attempt1: parport_serial - parport_register(...):

    void __iomem *addrs = pci_iomap(dev, 2, 0);
    printk(KERN_INFO "parport_serial LM addrs=%p\n", addrs);
    char w=0xc;
    iowrite8(0, addrs);
    w = ioread8(addrs+2);
    printk(KERN_INFO "parport_serial LM %x\n", w);
    w=0xc;
    iowrite8(w, addrs+2);
    w = ioread8(addrs+2);
    printk(KERN_INFO "parport_serial LM %x\n", w);

Output1:

No data is written, always reads as 0.

[    6.760175] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    6.799483] serial 0000:01:00.0: enabling device (0140 -> 0143)
[    6.814813] parport_serial LM addrs=fee01400
[    6.814827] parport_serial LM 0
[    6.814834] parport_serial LM 0

Attempt2: parport_pc - parport_SPP_supported(...):

   printk(KERN_INFO "HERE LM CONTROL(pb)=%lx, %p\n", CONTROL(pb), __io(CONTROL(pb)));

   void __iomem *addr=ioremap_nocache(0x1f80000 + CONTROL(pb),1); //ioremap takes the Physical address.
    printk(KERN_INFO "HERE LM Addr=%p\n", addr);
    r = readb(addr);
    printk(KERN_INFO "LM READ 1 r=%x\n", r);

    /* Do a simple read-write test to make sure the port exists. */
    w = 0xc;
    writeb(w, addr);

    /* Is there a control register that we can read from?  Some
     * ports don't allow reads, so read_control just returns a
     * software copy. Some ports _do_ allow reads, so bypass the
     * software copy here.  In addition, some bits aren't
     * writable. */
    r = readb(addr);
    printk(KERN_INFO "LM READ 2 r=%x\n", r);
    if ((r & 0xf) == w) {

      ...

Output2:

The pci_iomap(...) is compatible with __io(...) macro address mapping, but neither work...

[    6.814871] HERE LM CONTROL(pb)=1402, fee01402

ioremap_nocache maps the PCI I/O Physical address to another location... but writes don't produce effect and reads are always 0 again.
[    6.814914] HERE LM Addr=f01a6402
[    6.814922] LM READ 1 r=0
[    6.814928] LM READ 2 r=0

Attempt3: parport_pc - parport_SPP_supported(...):

Since PCI MEM space is at 0x100 0000 physical address and IO at 0x1f8 0000,

PCI_MEM + 0xf8 0000 should be PCI I/O.

    /* Do a simple read-write test to make sure the port exists. */
    w = 0xc;
    outb(w, 0xf80000 + CONTROL(pb));

but it seems that __io(...) macro is already mapping to the correct PCI I/O virtual address except that it doesn't work.

Output3:

[    6.224129] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    6.225026] Galcore version 5.0.11.33433
[    6.255265] serial 0000:01:00.0: enabling device (0140 -> 0143)
[    6.280476] parport_serial LM addrs=fee01400
[    6.280492] parport_serial LM 0
[    6.280498] parport_serial LM 0
[    6.280517] HERE LM CONTROL(pb)=1402, fee01402
[    6.280566] HERE LM Addr=f013e402
[    6.280571] LM READ 1 r=0
->    outb(w, 0xf80000 + CONTROL(pb));
[    6.280586] Unable to handle kernel paging request at virtual address fee81402
[    6.296096] pgd = d8b38000
[    6.299639] [fee81402] *pgd=686ec811, *pte=00000000, *ppte=00000000
[    6.310101] Internal error: Oops: 807 [#1] PREEMPT SMP ARM

0 Kudos

2,392 Views
lmendes
Contributor III

BAR assignments:

i.MX6 PCIe Controller BARs

[    3.408383] pci 0000:00:00.0: BAR 0: assigned [mem 0x01000000-0x010fffff]
[    3.413911] pci 0000:00:00.0: BAR 9: assigned [mem 0x01100000-0x011fffff pref]
[    3.419843] pci 0000:00:00.0: BAR 6: assigned [mem 0x01200000-0x0120ffff pref]
[    3.425789] pci 0000:00:00.0: BAR 7: assigned [io  0x1000-0x1fff]

WCH WC382 PCIe card BARs
[    3.430596] pci 0000:01:00.0: BAR 1: assigned [mem 0x01100000-0x01107fff pref]
[    3.436559] pci 0000:01:00.0: BAR 6: assigned [mem 0x01108000-0x0110ffff pref]
[    3.442489] pci 0000:01:00.0: BAR 0: assigned [io  0x1000-0x10ff]
[    3.447321] pci 0000:01:00.0: BAR 2: assigned [io  0x1400-0x1403]

i.MX6 PCIe controller bridge mapping
[    3.452137] pci 0000:00:00.0: PCI bridge to [bus 01]
[    3.455824] pci 0000:00:00.0:   bridge window [io  0x1000-0x1fff]
[    3.460628] pci 0000:00:00.0:   bridge window [mem 0x01100000-0x011fffff pref]

0 Kudos