Protected Execution Facility

### Overview This talk introduces the Protected Execution Facility (PEF), an architectural and firmware enhancement developed for IBM POWER and OpenPOWER processors. Designed to support confidential computing in untrusted or multi-tenant cloud environments, PEF enables Secure Virtual Machines (SVMs). The primary objective is to protect the confidentiality and integrity of guest virtual machines while in transit, at rest on disk, and during execution, guarding against threats such as rogue system administrators and compromised hypervisors.

### Key Topics - **Secure Virtual Machines (SVMs):** Isolating virtual machine execution and data from the underlying hypervisor (Linux KVM) and adjacent tenant VMs. - **Ultravisor Architecture:** A new firmware layer operating at a higher privilege mode than the traditional hypervisor, serving as the gatekeeper for secure execution. - **Hardware Privilege Hierarchy:** Introducing a new processor execution mode (Ultravisor mode) alongside supervisor, hypervisor, and problem states. - **Cryptographic Grounding via TPM 2.0:** Leveraging hardware Root of Trust, secure/trusted boot, and TPM 2.0 key storage to anchor VM verification and disk decryption. - **Memory Partitioning and Paravirtualization:** Dividing physical memory into secure and normal memory, requiring targeted paravirtualization changes in Linux KVM and virtio.

### Technical Details - **Processor and Hardware Modifications:** In POWER architecture, privilege was historically controlled by the Hypervisor (HV) and Problem State (PR) bits. PEF introduces an `S` bit in the Machine Status Register (MSR) to indicate secure mode, yielding four distinct execution states. New hardware registers are introduced, including `URMOR` (Ultravisor Real Mode Offset Register) for relocating the ultravisor, `USSR0`/`USSR1`, `USPRG0`/`USPRG1`, and `SMF_CTRL`. A new instruction, `URFID`, enables the ultravisor to enter an SVM. Furthermore, transactional memory is explicitly disabled in SVMs to eliminate side-channel risks. - **Secure Memory Isolation:** Hardware memory controllers enforce partitioning between secure memory and normal memory. Normal virtual machines, peripheral devices (PCI/GPU), and the hypervisor cannot reference secure memory; access is strictly enforced by processor hardware. The kernel's Heterogeneous Memory Management (HMM) subsystem is adapted to help manage secure memory pages. - **Boot Flow and the ESM Blob:** SVM images utilize the `zImage` format and include an Enter Secure Mode (ESM) blob. This blob packages an encrypted symmetric key paired with integrity hashes for the kernel, initramfs, and run tasks, along with DM-crypt passphrases for root filesystem decryption. The symmetric key is encrypted under the public keys of authorized target machines. At boot, the guest VM issues an ESM syscall (`enter_secure_mode`). The ultravisor requests the TPM to decrypt the symmetric key using the machine's non-exportable private key, relocates the VM to secure memory, verifies integrity, and destroys the in-memory symmetric key. - **Hypervisor Interception and I/O:** Interrupts and hypercalls (H-calls) are intercepted by the ultravisor. Hypercall arguments are scrubbed, presenting dummy register states to the hypervisor to prevent state leakage. For I/O and virtio communication, bounce buffering is used, and explicit shared non-protected memory pages must be established between the SVM and hypervisor (with expected payload encryption like TLS).

### Takeaways PEF establishes a minimal Trusted Computing Base (TCB)—consisting of processor hardware, TPM 2.0, Hostboot, and the tens of thousands of lines in the ultravisor, intentionally bypassing Skiboot and the multi-million-line KVM hypervisor. While initial iterations do not support live migration, suspend/resume, memory overcommit, or direct PCI assignment, PEF allows existing guest applications to run without modification while protecting them throughout their operational lifecycle. All associated firmware and software components are being contributed to open source.