Overview of PCI(e) Subsystem

### Overview In this presentation, Kishon Vijay Abraham provides an overview of the PCI and PCI Express (PCIe) subsystems in Linux, focusing on the root complex (host controller) side. Drawing from experience developing a PCIe root complex driver for Texas Instruments DRA7xx SoCs, Abraham explains how PCIe hardware architecture maps into Linux kernel drivers and device tree configurations, filling a documentation gap that typically favors endpoint device driver development.

### Key Topics - **PCI vs. PCI Express Architecture**: PCIe maintains software compatibility with PCI while shifting from a shared parallel bus to high-speed point-to-point serial links (supporting 2.5 Gbps in Gen 1, 5 Gbps in Gen 2, and 8 Gbps in Gen 3 across 1 to 32 lanes). Multiple devices are connected via root ports, bridges, and switches represented logically as virtual PCI-to-PCI bridges using primary, secondary, and subordinate bus numbers. - **Address Spaces and Address Translation**: PCIe defines four standard address spaces: configuration, I/O, memory, and message space. Configuration space expands from 256 bytes in legacy PCI to 4 KB per function in PCIe, split between standardized headers (Type 0 for endpoints, Type 1 for bridges and root ports) and capability structures. The root complex uses an Address Translation Unit (ATU) to translate CPU addresses in configurable internal address space to PCIe addresses generated via Enhanced Configuration Access Mechanism (ECAM) or mapped to endpoint Base Address Registers (BARs). - **Interrupt Mechanisms**: PCIe replaces dedicated physical IRQ lines with in-band signaling over serial TX/RX pairs. It supports legacy interrupts simulated via eight assert/deassert messages (INTA–INTD) and Message Signaled Interrupts (MSI), where endpoints write message data to a designated address monitored by an MSI controller. - **Linux PCI Subsystem Integration**: Discussion covers how the platform driver in `drivers/pci/host` probes via Device Tree, configures clocks, PHYs, resets, and the ATU (via the `ranges` property), interacts with the ARM PCI BIOS layer (`arch/arm/kernel/bios32.c`), and invokes the core PCI bus enumeration. When a root complex has only a single physical interrupt line to the CPU, it sets up a virtual IRQ domain to demultiplex interrupts for endpoint drivers (e.g., tg3, xhci-pci, ahci).

### Technical Details - **BAR Sizing**: Software determines endpoint memory/I/O requirements by writing all 1s to the BAR, reading back the response, clearing the lower flag bits, inverting the value, and adding 1. - **Device Tree Representation**: Root complexes use standard properties alongside `ranges` (encoding flags, 64-bit PCI destination, CPU source, and size) and `interrupt-map` to map PCI legacy INTx interrupts to hardware or virtual IRQ numbers. - **Diagnostics**: The `lspci` utility (`lspci -v` for verbose output and `lspci -x` for configuration space hex dumps) enables inspection of enumerated buses, bridges, and devices.

### Takeaways Writing a PCIe root complex driver requires handling platform-level hardware initialization (clocks, PHY, ATU) and correctly bridging hardware addresses and interrupts into Linux core frameworks. Proper configuration of virtual IRQ domains and outbound ATU translation windows is essential to isolate interrupts and facilitate transparent memory-mapped access for endpoint drivers.