AnsweredAssumed Answered

Accessing PCI via UIO on T1020/T1040

Question asked by Bradley Gamble on May 15, 2019
Latest reply on Jun 11, 2019 by Bradley Gamble

Hi,

 

I am currently using a device based on the T10xx reference design (T1022 processor). I have previously been building the Linux kernel using the v1.7 QoriQ SDK (QorIQ-SDK-V1.7-SOURCE-20141218-yocto.iso). however I am looking to upgrade to a recent mainline Linux kernel (In this example, 4.20).

 

I have successfully built and booted the 4.20 mainline kernel on my device and can access most of the basic devices (ie MTD, GPIO, i2C...) without major issue. However, on the mainline kernel the PCI bus is not being exposed via UIO. Previously on the 3.1x kernel provided with v1.7 QoriQ SDK the following related kernel options were enabled:

# Freescale specific options # CONFIG_FSL_DPA=y CONFIG_FSL_ERRATUM_A_008007=y CONFIG_FSL_IFC=y CONFIG_FSL_MPIC_TIMER_WAKEUP=y CONFIG_FSL_USDPAA=y CONFIG_UIO_FSL_DMA=y

These options appear not to exist in the mainline kernel - I assume they have been renamed or modified since merging back. I have instead enabled what I assume are the equivalent options:

# DMA Devices # CONFIG_FSL_DPA=y CONFIG_FSL_EDMA=y CONFIG_DMA_ENGINE=y
CONFIG_DMA_OF=y
CONFIG_UIO=y
CONFIG_UIO_PCI_GENERIC=y

Booting with these options appears to enumerate the devices. Below is a truncated copy of the V1.7 QoriQ SDK kernel:

fsl-of-dma ffe100300.dma: dma channel dma-uio0-0 initialized
fsl-of-dma ffe100300.dma: dma channel dma-uio0-1 initialized
fsl-of-dma ffe100300.dma: dma channel dma-uio0-2 initialized
fsl-of-dma ffe100300.dma: dma channel dma-uio0-3 initialized
fsl-of-dma ffe100300.dma: dma channel dma-uio0-4 initialized
fsl-of-dma ffe100300.dma: dma channel dma-uio0-5 initialized
fsl-of-dma ffe100300.dma: dma channel dma-uio0-6 initialized
fsl-of-dma ffe100300.dma: dma channel dma-uio0-7 initialized
fsl-of-dma ffe101300.dma: dma channel dma-uio1-0 initialized
fsl-of-dma ffe101300.dma: dma channel dma-uio1-1 initialized
fsl-of-dma ffe101300.dma: dma channel dma-uio1-2 initialized
fsl-of-dma ffe101300.dma: dma channel dma-uio1-3 initialized
fsl-of-dma ffe101300.dma: dma channel dma-uio1-4 initialized
fsl-of-dma ffe101300.dma: dma channel dma-uio1-5 initialized
fsl-of-dma ffe101300.dma: dma channel dma-uio1-6 initialized
fsl-of-dma ffe101300.dma: dma channel dma-uio1-7 initialized

And for comparison, this is the equivalent for the mainline kernel:

Freescale Elo series DMA driver
fsl-elo-dma ffe100300.dma: #0 (fsl,eloplus-dma-channel), irq 28
fsl-elo-dma ffe100300.dma: #1 (fsl,eloplus-dma-channel), irq 29
fsl-elo-dma ffe100300.dma: #2 (fsl,eloplus-dma-channel), irq 30
fsl-elo-dma ffe100300.dma: #3 (fsl,eloplus-dma-channel), irq 31
fsl-elo-dma ffe100300.dma: #4 (fsl,eloplus-dma-channel), irq 76
fsl-elo-dma ffe100300.dma: #5 (fsl,eloplus-dma-channel), irq 77
fsl-elo-dma ffe100300.dma: #6 (fsl,eloplus-dma-channel), irq 78
fsl-elo-dma ffe100300.dma: #7 (fsl,eloplus-dma-channel), irq 79
fsl-elo-dma ffe101300.dma: #0 (fsl,eloplus-dma-channel), irq 32
fsl-elo-dma ffe101300.dma: #1 (fsl,eloplus-dma-channel), irq 33
fsl-elo-dma ffe101300.dma: #2 (fsl,eloplus-dma-channel), irq 34
fsl-elo-dma ffe101300.dma: #3 (fsl,eloplus-dma-channel), irq 35
fsl-elo-dma ffe101300.dma: #4 (fsl,eloplus-dma-channel), irq 80
fsl-elo-dma ffe101300.dma: #5 (fsl,eloplus-dma-channel), irq 81
fsl-elo-dma ffe101300.dma: #6 (fsl,eloplus-dma-channel), irq 82
fsl-elo-dma ffe101300.dma: #7 (fsl,eloplus-dma-channel), irq 83

At this point, I can bind the devices to the uio_pci_generic driver and they appear to be exposed correctly:

~ # ls /dev/uio*
ls: /dev/uio*: No such file or directory
~ # echo "1957 082C" > /sys/bus/pci/drivers/uio_pci_generic/new_id
~ # ls /dev/uio*
/dev/uio0

This appears to work correctly, I can check the values in /sys/class/uio/uio0/ (ie "dev", "version", "uevent") which are identical to that of the V1.7 QoriQ SDK kernel, with the exception of "name" which was "dma-uio0-0" on the V1.7 QoriQ SDK kernel and is "uio-pci-generic" on the mainline kernel.

The main issue for me is that the mainline kernel is missing the /sys/class/uio/uio0/maps/ directory. As a comparison, on the V1.7 QoriQ SDK:

~ # ls -R /sys/class/uio/uio0
/sys/class/uio/uio0:
dev        event      name       uevent
device     maps       subsystem  version

/sys/class/uio/uio0/maps:
map0

/sys/class/uio/uio0/maps/map0:
addr    name    offset  size

 

And on the mainline kernel:

~ # ls -R /sys/class/uio/uio0
/sys/class/uio/uio0:
dev        device     event      name       subsystem  uevent     version

 

If I look under /sys/class/dma it appears all the DMa channels are available, but in a different format:

/sys/class/dma # ls -la
total 0
drwxr-xr-x    2 root     root             0 Jan  1  1970 .
drwxr-xr-x   41 root     root             0 Jan  1  1970 ..
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma0chan0 -> ../../devices/platform/ffe000000.soc/ffe100300.dma/dma/dma0chan0
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma0chan1 -> ../../devices/platform/ffe000000.soc/ffe100300.dma/dma/dma0chan1
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma0chan2 -> ../../devices/platform/ffe000000.soc/ffe100300.dma/dma/dma0chan2
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma0chan3 -> ../../devices/platform/ffe000000.soc/ffe100300.dma/dma/dma0chan3
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma0chan4 -> ../../devices/platform/ffe000000.soc/ffe100300.dma/dma/dma0chan4
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma0chan5 -> ../../devices/platform/ffe000000.soc/ffe100300.dma/dma/dma0chan5
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma0chan6 -> ../../devices/platform/ffe000000.soc/ffe100300.dma/dma/dma0chan6
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma0chan7 -> ../../devices/platform/ffe000000.soc/ffe100300.dma/dma/dma0chan7
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma1chan0 -> ../../devices/platform/ffe000000.soc/ffe101300.dma/dma/dma1chan0
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma1chan1 -> ../../devices/platform/ffe000000.soc/ffe101300.dma/dma/dma1chan1
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma1chan2 -> ../../devices/platform/ffe000000.soc/ffe101300.dma/dma/dma1chan2
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma1chan3 -> ../../devices/platform/ffe000000.soc/ffe101300.dma/dma/dma1chan3
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma1chan4 -> ../../devices/platform/ffe000000.soc/ffe101300.dma/dma/dma1chan4
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma1chan5 -> ../../devices/platform/ffe000000.soc/ffe101300.dma/dma/dma1chan5
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma1chan6 -> ../../devices/platform/ffe000000.soc/ffe101300.dma/dma/dma1chan6
lrwxrwxrwx    1 root     root             0 Jan  1  1970 dma1chan7 -> ../../devices/platform/ffe000000.soc/ffe101300.dma/dma/dma1chan7

We currently use the Freescale source file "dma_driver.c" which exposes the following functions:

  • int fsl_dma_chan_init(struct dma_ch **dma_ch, uint8_t dma_id, uint8_t ch_id)
  • int fsl_dma_chan_finish(struct dma_ch *dma_ch)
  • int fsl_dma_chan_basic_direct_init(struct dma_ch *dma_ch)
  • int fsl_dma_chan_bwc(struct dma_ch *dma_ch, uint8_t bwc)
  • int fsl_dma_wait_blocking(struct dma_ch *dma_ch)
  • int fsl_dma_direct_start(struct dma_ch *dma_ch, dma_addr_t src_phys, dma_addr_t dest_phys, uint32_t len)
  • int fsl_dma_chain_link_build(struct dma_link_setup_data *link_data, struct dma_link_dsc *link_dsc, uint64_t link_dsc_phys, uint32_t link_count)
  • int fsl_dma_chain_basic_start(struct dma_ch *dma_ch, struct dma_link_setup_data *link_data, uint64_t link_dsc_phys)

This source file is copyright dated 2011-2012 Freescale Semiconductor, Inc. If there is a newer version of this file, please let me know.

 

 

My main questions from this are:

- Am I using the correct driver to access the PCI device? We have software that accesses PCI via UIO so we would like to maintain this compatibility. I assume this related to the "map" directory not being exposed.

- Can I expose the DMA-UIO in the same format as on the v1.7 QoriQ SDK kernel? I assume this was previously done with the Freescale kernel driver - Is it possible to use this on the mainline kernel?

 

If any more details are required please let me know so that I can provide them.

 

Thanks in advance!

Outcomes