Hi forum,
There is an LS1043A and a fpga on my setup. I can communicate between ls1043 and fpga over pcie. LS1043A is root complex and fpga is end-point at my setup.
lspci output on my ls1043a :
0000:01:00.0 Memory controller: Xilinx Corporation Device 7011
Subsystem: Xilinx Corporation Device 0007
Flags: bus master, fast devsel, latency 0
Memory at 4040000000 (32-bit, non-prefetchable) [size=64K]
Memory at 4040010000 (32-bit, non-prefetchable) [size=64K]
Capabilities: [40] Power Management version 3
Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [60] Express Endpoint, MSI 00
Capabilities: [100] Device Serial Number 00-00-00-00-00-00-00-00
lspci on my fpga output :
00:00.0 Memory controller: Xilinx Corporation Device 7011
Subsystem: Xilinx Corporation Device 0007
Flags: bus master, fast devsel, latency 0
Memory at <ignored> (32-bit, non-prefetchable)
Memory at <ignored> (32-bit, non-prefetchable)
Capabilities: [40] Power Management version 3
Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [60] Express Endpoint, MSI 00
Capabilities: [100] Device Serial Number 00-00-00-00-00-00-00-00
Capabilities: [128] Vendor Specific Information: ID=0001 Rev=0 Len=038 <?>
I can communicate between ls1043a and fpga bidirectional with devmem on 0x4040000000 address.
LS1043a : $devmem 0x4040000000 32 , output : 0x0000000
LS1043A: $devmem 0x4040000000 32 0xDEAD
LS1043a : $devmem 0x4040000000 32 , output : 0xDEAD
FPGA : $devmem 0x40000000 32 , output : 0xDEAD
I want to send MSI interrupt from ls1043a to fpga and also vice-versa. I know that there is an driver to handle msi interrupt on ls1043 as "irq-ls-scfg-msi.c".
My devicetree related with msi :
msi1: msi-controller1@1571000 {
compatible = "fsl,ls1043a-msi";
reg = <0x0 0x1571000 0x0 0x8>;
msi-controller;
interrupts = <0 116 0x4>;
};
msi2: msi-controller2@1572000 {
compatible = "fsl,ls1043a-msi";
reg = <0x0 0x1572000 0x0 0x8>;
msi-controller;
interrupts = <0 126 0x4>;
};
msi3: msi-controller3@1573000 {
compatible = "fsl,ls1043a-msi";
reg = <0x0 0x1573000 0x0 0x8>;
msi-controller;
interrupts = <0 160 0x4>;
};
And also I added printk to "irq-ls-scfg-msi.c" driver to check if probe and setup_irq methods calling and working. So driver is probed and setup all msi interrupts.
My "irq-ls-scfg-msi.c" driver's debug outputs are on dmesg : (all output's last string function name and last number is file number index)
root@tqmls1043a-mbls10xxa:~# dmesg | grep MSI
[ 2.242754] 003: MSI DEBUG: msir->gic_irq : 51, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[ 2.254258] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[ 2.261211] 003: MSI DEBUG: msir->gic_irq : 52, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[ 2.272709] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[ 2.279663] 003: MSI DEBUG: msir->gic_irq : 53, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[ 2.291156] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[ 2.298102] 003: MSI DEBUG: msir->gic_irq : 54, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[ 2.309588] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[ 2.316542] 003: MSI DEBUG: probe success ls_scfg_msi_probe 406
[ 2.322712] 003: MSI DEBUG: msir->gic_irq : 55, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[ 2.334200] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[ 2.341147] 003: MSI DEBUG: msir->gic_irq : 56, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[ 2.352632] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[ 2.359578] 003: MSI DEBUG: msir->gic_irq : 57, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[ 2.371063] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[ 2.378008] 003: MSI DEBUG: msir->gic_irq : 58, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[ 2.389493] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[ 2.396445] 003: MSI DEBUG: probe success ls_scfg_msi_probe 406
[ 2.402607] 003: MSI DEBUG: msir->gic_irq : 59, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[ 2.414095] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[ 2.421042] 003: MSI DEBUG: msir->gic_irq : 60, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[ 2.432529] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[ 2.439475] 003: MSI DEBUG: msir->gic_irq : 61, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[ 2.450959] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[ 2.457905] 003: MSI DEBUG: msir->gic_irq : 62, msi_data->msir_num = 4, msi_data->irqs_num = 128, ls_scfg_msi_setup_hwirq 271
[ 2.469391] 003: MSI DEBUG: setup irq success ls_scfg_msi_setup_hwirq 286
[ 2.476337] 003: MSI DEBUG: probe success ls_scfg_msi_probe 406
But I cannot find msi interrupts on linux userspace. /proc/interrupt's output is :
root@tqmls1043a-mbls10xxa:~# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
1: 0 0 0 0 GICv2 25 Level vgic
3: 18291 29847 30490 47937 GICv2 30 Level arch_timer
4: 0 0 0 0 GICv2 27 Level kvm guest vtimer
6: 0 0 0 0 GICv2 138 Level arm-pmu
7: 0 0 0 0 GICv2 139 Level arm-pmu
8: 0 0 0 0 GICv2 127 Level arm-pmu
9: 0 0 0 0 GICv2 129 Level arm-pmu
11: 5 0 0 0 GICv2 131 Level 1550000.spi
12: 2007 0 0 0 GICv2 94 Level mmc0
13: 0 0 0 0 GICv2 176 Level [EDAC] MC err
15: 0 0 0 0 GICv2 77 Level bman-err, qman-err, fman-err
16: 0 0 0 0 GICv2 205 Level BMan portal 0
17: 0 0 0 0 GICv2 207 Level BMan portal 1
18: 0 0 0 0 GICv2 209 Level BMan portal 2
19: 0 0 0 0 GICv2 211 Level BMan portal 3
26: 0 0 0 0 GICv2 204 Level QMan portal 0
27: 0 0 0 0 GICv2 206 Level QMan portal 1
28: 0 0 0 0 GICv2 208 Level QMan portal 2
29: 0 0 0 0 GICv2 210 Level QMan portal 3
36: 0 0 0 0 GICv2 96 Level 2100000.spi
37: 224 0 0 0 GICv2 88 Level 2180000.i2c
38: 0 0 0 0 GICv2 91 Level 21b0000.i2c
39: 5243 0 0 0 GICv2 86 Level ttyS1
45: 0 0 0 0 GICv2 115 Level 2ad0000.wdog
47: 0 0 0 0 GICv2 92 Level xhci-hcd:usb1
48: 7 0 0 0 GICv2 93 Level dwc3-otg, xhci-hcd:usb5
49: 52 0 0 0 GICv2 95 Level xhci-hcd:usb3
50: 0 0 0 0 GICv2 101 Level ahci-qoriq[3200000.sata]
69: 0 0 0 0 GICv2 185 Level qDMA error
70: 0 0 0 0 GICv2 71 Level qDMA queue
74: 0 0 0 0 GICv2 76 Level fman, ptp_qoriq
75: 0 0 0 0 GICv2 163 Edge
76: 0 0 0 0 GICv2 164 Edge
77: 0 0 0 0 GICv2 165 Edge
78: 0 0 0 0 GICv2 174 Edge
IPI0: 8139 9691 8545 5158 Rescheduling interrupts
IPI1: 50 258 198 72 Function call interrupts
IPI2: 0 0 0 0 CPU stop interrupts
IPI3: 0 0 0 0 CPU stop (for crash dump) interrupts
IPI4: 0 0 0 0 Timer broadcast interrupts
IPI5: 0 13 119 0 IRQ work interrupts
IPI6: 0 0 0 0 CPU wake-up interrupts
Err: 0
Also ls of /proc/irq is :
root@tqmls1043a-mbls10xxa:~# ls /proc/irq/
1 12 15 17 19 26 28 3 37 39 45 48 5 6 7 74 76 78 9
11 13 16 18 2 27 29 36 38 4 47 49 50 69 70 75 77 8 default_smp_affinity
So how can I receive MSI interrupt which sent from fpga and how can I send a MSI interrupt to FPGA?
I cannot anything about MSI interrupt on linux. Frankly I want to bind to msi-interrupt from c application. Also I want to check if MSI interrupt come from fpga with command line (ex. cat /proc/interrupts)
Also if I have to write a driver, how can I do, do you have an example? What kind of driver I have to write?