Private Model as a Service: Zero-Trust Blueprint for Protecting AI Weights- M. Entenza & J. Freimann

### Overview Presented by Marcos Entenza and Jens Freimann from Red Hat's confidential computing team, this talk introduces an upstream zero-trust architecture for "Private Model as a Service." Due to data gravity and strict regulatory mandates (such as European digital sovereignty rules and DORA for financial services), organizations must run AI workloads locally alongside sensitive data. However, this creates a three-way trust dilemma among the model owner (who must protect intellectual property and model weights), the enterprise data consumer (who cannot expose proprietary data), and the infrastructure provider/cluster administrator (who possesses root access to the host). The speakers demonstrate how confidential computing and confidential containers solve this dilemma without relying on policy promises or contracts alone.

### Key Topics - **Three-Way Distrust**: Mutually adversarial relationships between model owners, enterprise consumers/data owners, and infrastructure operators. - **Separation of Five Personas**: Differentiating roles across the Model Owner, SecOps (Operational Security), Developer, Cluster Admin, and Inference User to ensure developers and root admins do not control or compromise confidential security boundaries. - **Confidential Containers & Orchestration**: Leveraging CNCF Confidential Containers (CoCo), Kata Containers, and Trustee (Attestation & Key Broker Service) within OpenShift/Kubernetes. - **Attestation Architectures**: Exploring the Passport model, cross-domain multi-trust, and DMZ-based attestation relays for network-restricted enterprise environments.

### Technical Details - **Developer vs. SecOps Workflows**: In KServe (via upstream PR 5382 adding a confidential spec), developers only add minimal fields to their inference manifest (`confidential: enabled: true`, a KBS resource ID, and a runtime class name). SecOps configures Trustee, attestation policies, reference values (RVPS), and model key registrations out-of-band. - **Runtime Isolation**: The inference workload (e.g., KServe, Kata agent, vLLM) runs inside a confidential virtual machine (Kata VM) backed by encrypted memory. Host administrators with root privileges are explicitly treated as adversaries and cannot inspect memory. - **Cross-Domain Attestation Models**: - *Passport Model*: When the model owner trusts the customer's attestation service, the Trusted Execution Environment (TEE) attests locally, obtains a signed token ("passport"), and presents it to the model owner's Key Broker Service (KBS). - *Multi-Trust & Relays*: When the model owner distrusts customer attestation, hardware evidence (CPU and GPU composite attestation) must be verified directly by the model owner's Trustee. In network-restricted environments, a relay in the customer's DMZ forwards signed attestation reports to the model owner. The model owner encrypts the model decryption key with the TEE's ephemeral public key and returns it through the relay. The relay cannot decrypt the key because the private key never leaves the enclave. - **Model Decryption Flow**: Storage initializers pull encrypted weights from an OCI registry. If Trustee verifies the hardware measurements, firmware versions, and container image hashes against the model owner's reference policies, the key is released and decrypted inside enclave memory. Otherwise, the key remains locked.

### Takeaways - RBAC and traditional perimeter controls cannot resolve the multi-party trust conflict in on-premise or sovereign enterprise AI; hardware-enforced confidential computing is necessary. - Security must be decoupled from developers and embedded transparently by operational security teams. - While the single-trust passport model works today, cross-domain attestation and multi-trust hardware verification are actively being advanced within the upstream CoCo (Confidential Containers) community.