Linux PCI Device Driver - Bus v. Kernel IRQ

Question

I am writing a device driver for a PCIe card in Linux. I am trying to use interrupts in my driver.

Reading the "IRQ Line" section of the PCI configuration register (offset 0x3C) reports that the assigned IRQ line for the device is 11. lspci -b -vv also reports that my device's interrupt number is 11.

Heres where it gets weird... cat /sys/bus/pci/devices/<my_device>/irq reports that the interrupt number is 19. lspci -vv also reports that the interrupt number is 19.

Requesting 11 in my driver does not work. If I request 19 in the driver, I catch interrupts just fine.

What gives?

Thanks!!!

Naivete

Answer 1

I believe that it has to do with the difference between "physical" and "virtual" IRQ lines. Because the processor has a limited number of physical IRQ lines it assigns virtual IRQ lines to allow the total number of PCI devices to exceed the number of physical lines.

In this instance, 19 is your virtual IRQ line (as recognized by the processor) while 11 is the physical line (as recognized by the PCI device).

By the way, you should probably really get the IRQ number from the struct pci_dev for that device since they're dynamically generated.

Boarding

Wow, thanks! Any advice on where you gather knowledge like this? LDD3 is outdated or incomplete, at least regarding this... – Naivete

LDD3 is a fantastic place to start, but most of what I picked up has been from googling around and reading kernel source. I had to write a PCIe FPGA driver a while back and that's where I got most of this. That said, wikipedia is a fantastic resource (look up IRQ and APIC for more reading on this topic). – Boarding

@SeanMadden Thanks for highlighting struct pci_dev: But what if there's more than one interrupt? – Semiautomatic

Answer 2

Sean's answer is easy to understand. However here I would try to make it more complete.

CPU's IRQ pin, almost always, isn't connected directly to a peripheral device, but via an programmable interrupt controller(PIC, e.g. Intel 8259A). This helps handling large device fan-out and also heterogeneous interrupt format (pin based v.s. message based as in PCIe).

If you run a recent version of lspci, it would print information like

Interrupt: pin A routed to IRQ 26

Here, pin A as 11 in OP, is the physical pin. This is something saved by the PCI device and used by the hardware to exchange between interrupts controller. From LDP:

The PCI set up code writes the pin number of the interrupt controller into the PCI configuration header for each device. It determines the interrupt pin (or IRQ) number using its knowledge of the PCI interrupt routing topology together with the devices PCI slot number and which PCI interrupt pin that it is using. The interrupt pin that a device uses is fixed and is kept in a field in the PCI configuration header for this device. It writes this information into the interrupt line field that is reserved for this purpose. When the device driver runs, it reads this information and uses it to request control of the interrupt from the Linux kernel.

IRQ 26 as 19 in OP is something that kernel code and CPU deal with. According to Linux Documentation/IRQ.txt:

An IRQ number is a kernel identifier used to talk about a hardware interrupt source. Typically this is an index into the global irq_desc array, but except for what linux/interrupt.h implements the details are architecture specific.

So the PCI first receives interrupts from device, translate interrupt source to a IRQ number and informs the CPU. CPU use IRQ number to look into Interrupt Descriptor Table(IDT) and find the correct software handler.

Ref:

http://www.tldp.org/LDP/tlk/dd/interrupts.html http://www.brokenthorn.com/Resources/OSDevPic.html

Galligaskins