FAQs
English (US) 日本語 | Japanese 中文 | Chinese (Simplified)

ls1043A pcie msi interrupt with msi-controller

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 

ls1043A pcie msi interrupt with msi-controller

‎12-31-2022 10:54 AM
390 Views
draven
Contributor III

Hi forum,

There is an LS1043A and a fpga on my setup. I can communicate between ls1043 and fpga over pcie. LS1043A is root complex and fpga is end-point at my setup.

lspci output on my ls1043a :

0000:01:00.0 Memory controller: Xilinx Corporation Device 7011
Subsystem: Xilinx Corporation Device 0007
Flags: bus master, fast devsel, latency 0
Memory at 4040000000 (32-bit, non-prefetchable) [size=64K]
Memory at 4040010000 (32-bit, non-prefetchable) [size=64K]
Capabilities: [40] Power Management version 3
Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [60] Express Endpoint, MSI 00
Capabilities: [100] Device Serial Number 00-00-00-00-00-00-00-00

lspci on my fpga output : 

00:00.0 Memory controller: Xilinx Corporation Device 7011
        Subsystem: Xilinx Corporation Device 0007
        Flags: bus master, fast devsel, latency 0
        Memory at <ignored> (32-bit, non-prefetchable)
        Memory at <ignored> (32-bit, non-prefetchable)
        Capabilities: [40] Power Management version 3
        Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [60] Express Endpoint, MSI 00
        Capabilities: [100] Device Serial Number 00-00-00-00-00-00-00-00
        Capabilities: [128] Vendor Specific Information: ID=0001 Rev=0 Len=038 <?>

I can communicate between ls1043a and fpga bidirectional with devmem on 0x4040000000 address.

LS1043a :  $devmem 0x4040000000 32  , output : 0x0000000
LS1043A:  $devmem 0x4040000000 32 0xDEAD
LS1043a :  $devmem 0x4040000000 32 , output : 0xDEAD
FPGA : $devmem 0x40000000 32 , output : 0xDEAD

I want to send MSI interrupt from ls1043a to fpga and also vice-versa. I know that there is an driver to handle msi interrupt on ls1043 as "irq-ls-scfg-msi.c".

My devicetree related with msi :

		msi1: msi-controller1@1571000 {
			compatible = "fsl,ls1043a-msi";
			reg = <0x0 0x1571000 0x0 0x8>;
			msi-controller;
			interrupts = <0 116 0x4>;
		};

		msi2: msi-controller2@1572000 {
			compatible = "fsl,ls1043a-msi";
			reg = <0x0 0x1572000 0x0 0x8>;
			msi-controller;
			interrupts = <0 126 0x4>;
		};

		msi3: msi-controller3@1573000 {
			compatible = "fsl,ls1043a-msi";
			reg = <0x0 0x1573000 0x0 0x8>;
			msi-controller;
			interrupts = <0 160 0x4>;
		};

And also I added printk to "irq-ls-scfg-msi.c" driver to check if probe and setup_irq methods calling and working. So driver is probed and setup all msi interrupts.

My "irq-ls-scfg-msi.c" driver's debug outputs are on dmesg :  (all output's last string function name and last number is file number index)

root@tqmls1043a-mbls10xxa:~# dmesg | grep MSI
[    2.242754] 003: MSI DEBUG: msir->gic_irq : 51, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[    2.254258] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[    2.261211] 003: MSI DEBUG: msir->gic_irq : 52, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[    2.272709] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[    2.279663] 003: MSI DEBUG: msir->gic_irq : 53, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[    2.291156] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[    2.298102] 003: MSI DEBUG: msir->gic_irq : 54, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[    2.309588] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[    2.316542] 003: MSI DEBUG: probe success ls_scfg_msi_probe 406
[    2.322712] 003: MSI DEBUG: msir->gic_irq : 55, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[    2.334200] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[    2.341147] 003: MSI DEBUG: msir->gic_irq : 56, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[    2.352632] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[    2.359578] 003: MSI DEBUG: msir->gic_irq : 57, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[    2.371063] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[    2.378008] 003: MSI DEBUG: msir->gic_irq : 58, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[    2.389493] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[    2.396445] 003: MSI DEBUG: probe success ls_scfg_msi_probe 406
[    2.402607] 003: MSI DEBUG: msir->gic_irq : 59, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[    2.414095] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[    2.421042] 003: MSI DEBUG: msir->gic_irq : 60, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[    2.432529] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[    2.439475] 003: MSI DEBUG: msir->gic_irq : 61, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[    2.450959] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[    2.457905] 003: MSI DEBUG: msir->gic_irq : 62, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[    2.469391] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[    2.476337] 003: MSI DEBUG: probe success ls_scfg_msi_probe 406

But I cannot find msi interrupts on linux userspace.  /proc/interrupt's output is :

root@tqmls1043a-mbls10xxa:~# cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3
  1:          0          0          0          0     GICv2  25 Level     vgic
  3:      18291      29847      30490      47937     GICv2  30 Level     arch_timer
  4:          0          0          0          0     GICv2  27 Level     kvm guest vtimer
  6:          0          0          0          0     GICv2 138 Level     arm-pmu
  7:          0          0          0          0     GICv2 139 Level     arm-pmu
  8:          0          0          0          0     GICv2 127 Level     arm-pmu
  9:          0          0          0          0     GICv2 129 Level     arm-pmu
 11:          5          0          0          0     GICv2 131 Level     1550000.spi
 12:       2007          0          0          0     GICv2  94 Level     mmc0
 13:          0          0          0          0     GICv2 176 Level     [EDAC] MC err
 15:          0          0          0          0     GICv2  77 Level     bman-err, qman-err, fman-err
 16:          0          0          0          0     GICv2 205 Level     BMan portal 0
 17:          0          0          0          0     GICv2 207 Level     BMan portal 1
 18:          0          0          0          0     GICv2 209 Level     BMan portal 2
 19:          0          0          0          0     GICv2 211 Level     BMan portal 3
 26:          0          0          0          0     GICv2 204 Level     QMan portal 0
 27:          0          0          0          0     GICv2 206 Level     QMan portal 1
 28:          0          0          0          0     GICv2 208 Level     QMan portal 2
 29:          0          0          0          0     GICv2 210 Level     QMan portal 3
 36:          0          0          0          0     GICv2  96 Level     2100000.spi
 37:        224          0          0          0     GICv2  88 Level     2180000.i2c
 38:          0          0          0          0     GICv2  91 Level     21b0000.i2c
 39:       5243          0          0          0     GICv2  86 Level     ttyS1
 45:          0          0          0          0     GICv2 115 Level     2ad0000.wdog
 47:          0          0          0          0     GICv2  92 Level     xhci-hcd:usb1
 48:          7          0          0          0     GICv2  93 Level     dwc3-otg, xhci-hcd:usb5
 49:         52          0          0          0     GICv2  95 Level     xhci-hcd:usb3
 50:          0          0          0          0     GICv2 101 Level     ahci-qoriq[3200000.sata]
 69:          0          0          0          0     GICv2 185 Level     qDMA error
 70:          0          0          0          0     GICv2  71 Level     qDMA queue
 74:          0          0          0          0     GICv2  76 Level     fman, ptp_qoriq
 75:          0          0          0          0     GICv2 163 Edge
 76:          0          0          0          0     GICv2 164 Edge
 77:          0          0          0          0     GICv2 165 Edge
 78:          0          0          0          0     GICv2 174 Edge
IPI0:      8139       9691       8545       5158       Rescheduling interrupts
IPI1:        50        258        198         72       Function call interrupts
IPI2:         0          0          0          0       CPU stop interrupts
IPI3:         0          0          0          0       CPU stop (for crash dump) interrupts
IPI4:         0          0          0          0       Timer broadcast interrupts
IPI5:         0         13        119          0       IRQ work interrupts
IPI6:         0          0          0          0       CPU wake-up interrupts
Err:          0

Also ls of /proc/irq is : 

root@tqmls1043a-mbls10xxa:~# ls /proc/irq/
1   12  15  17  19  26  28  3   37  39  45  48  5   6   7   74  76  78  9
11  13  16  18  2   27  29  36  38  4   47  49  50  69  70  75  77  8   default_smp_affinity

So how can I receive MSI interrupt which sent from fpga and how can I send a MSI interrupt to FPGA?

I cannot anything about MSI interrupt on linux. Frankly I want to bind to msi-interrupt from c application. Also I want to check if MSI interrupt come from fpga with command line (ex. cat /proc/interrupts)

Also if I have to write a driver, how can I do, do you have an example? What kind of driver I have to write?

Tags (3)
0 Kudos
Reply
0 Replies