Modernize BPF for the next 10 years (Alexei Starovoitov)

### Overview At LSF/MM/BPF, Alexei Starovoitov reflects on the 10-year anniversary of BPF (tracing its origins back to 2014 when initial patches and the `bpf()` system call landed) and outlines a roadmap to modernize the subsystem for the next decade. BPF originated primarily as a programmable networking packet processing engine, then expanded rapidly into tracing, observability, and kernel extension. To remain relevant and scalable, Starovoitov argues that BPF must evolve beyond its legacy limitations, deprecate outdated developer practices, and embrace runtime safety, rich algorithms, and architectural flexibility.

### Key Topics - **Evolution of BPF (2014–2024):** Transitions from networking (XDP, Katran, Cilium) to tracing/observability (BCC, bpftrace, BTF, CO-RE, skeletons). Early limitations forced workarounds like forced inlining (`always_inline`) and loop unrolling, which modern features have superseded. - **Interface Modernization:** Replacing rigid, stable UAPI helpers (only ~200 added over 10 years due to review friction and immutability) with dynamic, unstable `kfuncs` that kernel modules can supply. Similarly, transitioning kernel-to-BPF attachment from fixed hooks to flexible struct_ops (used in TCP congestion control, sched_ext, and fuse-BPF). - **Advanced Data Structures & Sched_ext (scx):** Driving complex kernel interactions through kpointers, red-black trees, linked lists, and memory constructors (`bpf_obj_new`), moving toward user/kernel shared structures. - **The BPF Arena & Runtime Safety Shift:** Shifting the verification paradigm from pure static analysis to runtime bounds checking using page tables and 4GB regions, enabling regex engines, memory allocators, and string manipulation. - **Language & Package Ecosystem:** The need for source-based distribution and library sharing (analogous to Rust's model) rather than replicating C/C++ shared library distribution issues. - **Architecture and Instruction Set Architecture (ISA) Enhancements:** Expanding BPF beyond x86_64 biases to better exploit ARM64 and RISC-V registers, supporting more than 5 arguments to kfuncs, jump tables, indirect calls, and divided/dynamic stack allocation.

### Technical Details - **Loops and Iterators:** BPF moved from `pragma unroll` to bounded loops in 2019, the `bpf_loop` helper in 2021, open-coded iterators (`bpf_for`) in 2023, and `may_goto` (conditional break/reschedule points). Notably, a verifier bug in open-coded iterators persisted from 2021 until it was resolved in late 2023. - **Concurrency & Resilient Locks:** Current verification limits programs to holding one spinlock without calling functions. Starovoitov describes ongoing work toward locks resilient to tampering (e.g., preventing kernel deadlocks when a bug overwrites a lock with zeros) and automatic deadlock detection to support multiple or nested locks, showing benchmark results using `lock_torture`. - **Instruction Set Expansion:** Proposes instructions for bit manipulation (popcount, bit rotate, finding first set bit), indirect jumps/gotos (enabling jump tables and writing a BPF interpreter in BPF), and the `no_caller_saved_registers` calling convention to reduce register spill/fill pressure around lightweight helpers like `bpf_rcu_read_lock`. - **Architecture Register Mismatch:** BPF’s fixed register count disadvantages ARM64. Discussed alternatives include virtual registers with in-kernel register allocation, architecture-specific instruction variants, fat binaries compiled for different register sets, or better verifier register spilling directly to spare hardware registers. - **Instruction Limit (1 Million) & Best Practices:** Developers frequently encounter the 1M verified instruction ceiling because of nested inlining and legacy coding habits. The community recommends adopting subprograms/global functions and contributing test cases to the kernel self-tests suite to prevent regressions. - **Kernel Version Decoupling:** Addressing the enterprise burden of long-tail legacy kernels by investigating ways to deliver BPF/verifier upgrades independently of base kernel distribution upgrades without unloading active production programs.

### Takeaways BPF's primary future challenge is shedding 10 years of legacy baggage while modernizing its ISA, verification model, and usability. By transitioning from strict static analysis toward selective runtime safety (Arenas, resilient locking, `may_goto`) and expanding ISA support for modern multi-register architectures, BPF aims to support arbitrary data structures, schedulers, and potentially full kernel compilation targets in the next decade.