From the Ground Up: How We Built the Nanos Unikernel

### Overview In this Open Source Summit talk, Will Jhun, a systems programmer at NanoVMs, details the architecture, design philosophy, and operational model of Nanos, a clean-slate unikernel built for running cloud microservices. Unlike general-purpose operating systems designed for multi-user, multi-process environments, unikernels run a single application and address space per virtual machine instance. Jhun explains how Nanos achieves rapid boot times (tens of milliseconds) and a stripped kernel footprint of roughly 750 kilobytes while maintaining binary-level compatibility with unmodified Linux ELF executables and standard libraries.

### Key Topics - **Unikernel Concepts vs. Traditional OSes**: Unikernels reduce system complexity and attack surface by stripping out unneeded facilities such as shells, multi-user access, and process namespaces. While traditional container isolation relies on kernel namespaces, unikernels provide full hardware-virtualized isolation with microsecond-to-millisecond instantiation. - **Binary Compatibility and the Ops Tool**: Rather than taking a library OS approach or requiring custom code compilation, Nanos focuses on running standard Linux ELF binaries. The accompanying orchestration tool, Ops (`ops.city`), automates staging dependencies, generating bootable disk images, and deploying instances locally under QEMU or directly to platforms like AWS EC2, GCP, Azure, and Firecracker. - **Kernel Security Philosophy**: Contrary to unikernels that eliminate ring privilege separation to avoid syscall overhead, Nanos implements standard security boundaries including kernel/user privilege separation, read-only/no-execute page protections, user-space address space layout randomization (ASLR), and an optional "exec protection" feature that makes executable pages immutable. - **Performance Evaluation**: In a demonstration comparing a simple Go web server running on an identical Google Cloud Platform VM type, the Nanos instance achieved approximately 16,000 requests per second compared to 12,500 on Debian—delivering over a 25% performance improvement under Apache Bench testing.

### Technical Details - **Composable Heaps and Number Allocators**: In Nanos, memory allocation is generalized as "heaps" managed through abstract allocate/deallocate interfaces. This extends beyond memory to arbitrary number spaces, such as file descriptors, thread IDs, and interrupt vectors using `id_heap`. Higher-level allocators are composed together: physical heaps, virtual huge page heaps, and page heaps combine into backed heaps, which feed into `mcache` (power-of-two size caches similar to Linux SLUB). Container wrappers also provide functionality such as debugging, poisoning, and spinlock synchronization. - **Asynchronous Operations via Closures and Continuations**: Rather than relying heavily on heavy kernel thread stacks and context switching, Nanos employs C closures—callbacks paired with explicitly bound lexical environments. Closures like `status_handler` and `buffer_handler` serve as continuations for asynchronous I/O. In the logging extent-based file system (`tfs`), complex multi-part parallel read operations are unified using merge completions in roughly 70 lines of code. - **Orthogonal Scheduling**: Kernel scheduling surfaces runnable closures without arguments (thunks) using lock-free queues. The scheduler operates orthogonally to process or thread semantics; it simply dequeues and applies thunks. - **Tuple-Based Configuration and Live Introspection**: At build time, a human-readable manifest is ingested by `makefs` into the file system log. At boot, this populates a hierarchical key-value root tuple representing file system metadata, configuration, and environment variables. This tuple space also supports dynamic methods that generate live metrics on demand (e.g., live network interrupt counters), exposed experimentally via internal HTTP/WebSocket interfaces.

### Takeaways Nanos illustrates that unikernels can maintain Linux binary compatibility without abandoning crucial security mitigations such as privilege separation and memory protection. By avoiding legacy operating system codebases, Nanos employs functional programming patterns in C—specifically composable allocators and closures—yielding a modular, easily inspectable, and high-performance execution environment tailored for cloud workloads.