BPF Qdisc (Amery Hung)

### Overview Amery Hung presented an update on BPF Qdisc (queueing discipline), a project aimed at allowing users to implement, explore, and innovate packet scheduling policies and algorithms dynamically in the Linux kernel using BPF. Inspired by similar efforts like sched_ext (scx), BPF Qdisc leverages recent BPF enhancements such as relaxed loops, graph collections (BPF lists and red-black trees), and struct_ops to enable programmable scheduling while keeping the developer interface straightforward.

### Key Topics - **Evolution from V7 to V8**: The V7 patch set required allocating local objects and swapping pointers to enqueue SKBs into BPF data structures, incurring memory allocation overhead in the fast path. V8 transitioned to `struct_ops` and direct insertion of `sk_buff` into BPF graph collections, removing user-space API friction and enabling direct BPF link support. - **SKB Lifecycle and Ownership**: Enforcing proper reference semantics for `sk_buff` structures is critical to avoid reference leaks, dangling pointers, and crashes. Discussion focused on moving toward unique ownership rather than standard BPF reference counts (`ref_obj_id`), preventing multiple references to the same packet. - **Verifier and Graph Collection Support**: Extending BPF collections to store kernel objects directly by allowlisting `sk_buff` during BTF parsing, avoiding boot-time overhead, and handling exclusive node semantics in BPF red-black trees and lists. - **Handling Kernel Struct Conflicts (SKB Dev vs. RB Node)**: Resolving union member collisions inside `struct sk_buff`, where `dev` shares memory with the red-black node and gets overwritten when queued. - **Classful Qdiscs and Hierarchy**: Proposals for handling classful Qdiscs, watchdog timers, and class hierarchies, along with audience debate on whether topology creation should be handled in kernel space or configured from user space via Netlink.

### Technical Details - **Direct Graph Insertion**: In V7, inserting an SKB required an extra allocation in the fast path. V8 allows placing `struct sk_buff` directly into BPF graph collections. Benchmark comparisons using an iperf test over a loopback interface demonstrated a 7.6% throughput improvement in V8 over V7. - **Unique Pointer Semantics**: Traditional `KF_ACQUIRE` can be invoked multiple times, yielding duplicate references that break queue operations. The proposed approach uses a struct_ops mechanism to acquire a reference once upon argument access. Reviewers suggested introducing explicit unique pointer ownership in the verifier instead of overloading reference counts. - **Returning Pointers via struct_ops**: Struct_ops did not natively support returning a kernel pointer back to core networking code (e.g., during dequeue). The patch set adjusts verifier exit checks (`check_reference_leak` and `check_return_code`) to permit returning a single acquired reference or `NULL`. - **SKB Dev Restoration Issue**: Inside `struct sk_buff`, `rb_node` overlaps with `dev` in a union. Inserting an SKB into a tree overwrites `dev`, requiring `dev` to be restored before dequeue. Hung noted a temporary helper function (`bpf_qdisc_skb_set_dev`), but reviewers raised safety concerns regarding relying on users to call it. Potential remedies discussed include rearranging fields in `struct sk_buff` or having the BPF verifier automatically rewrite programs to restore `dev`. - **Memory Cleanup and Reset**: Reviewers pointed out that relying on a Qdisc's `reset` callback to flush packets from BPF maps is unsafe if the user neglects to implement it properly. Incorporating kernel-managed map destructors (similar to CPU masks or task structs) was suggested to ensure lingering packets are automatically freed upon destruction.

### Takeaways BPF Qdisc demonstrates significant progress toward flexible, in-kernel programmable packet scheduling with reduced overhead via `struct_ops`. While core data path mechanics and performance show strong improvements, key architectural and safety questions remain before upstream landing. Next steps include refining the verifier's unique ownership model, automating `skb->dev` restoration or layout changes, establishing robust resource destructors, simplifying classful hierarchy creation, and validating performance under real production workloads.