Skip to content

Bootloops & Coffee: The Daily Grind of an AOSP Developer

Sumit SemwalAmit PundirLinaro Ltd.

Blue TeamThe Linux Foundation26 minOpen on YouTube ↗

Details bootloops & Coffee: The Daily Grind of an AOSP Developer and evaluates key implications for security practitioners.

Summary

## Overview Amit Pund from Linaro presents the challenges and recent developments in maintaining Android Open Source Project (AOSP) on development boards, using the metaphor of "bootloops and coffee" to describe the daily reality of AOSP developers. The talk covers major features landing in AOSP, integration of external projects, and community initiatives emerging in response to recent policy changes by Google.

## Key Topics

### Generic Bootloader Library (GBL) Android suffers from bootloader fragmentation because each vendor implements their own bootloader rather than using a common standard. GBL is an EFI application developed and distributed by Google that provides a standardized, vendor-agnostic bootloader interface. It abstracts underlying platform-specific implementations, reducing the need for vendors to rework bootloaders with every Android release. However, vendors must implement custom EFI protocols to use GBL—it is not a simple drop-in replacement.

### Page Size Agnostic AOSP Builds ASP has been refactored to support both 4K and 16K page sizes without requiring separate builds. 16K page size builds deliver 5–10% performance improvements at a cost of 9% additional memory. Implementation required removing page size assumptions, using 16K-aligned binaries, and employing separate loadable segments. File system configuration varies by platform: F2FS offers best performance when block size matches kernel page size, while ext4 and EROFS support agnostic builds with 4K block sizes.

### System Core Changes and Notable Additions Recent system core updates simplified booting AOSP from external storage (e.g., USB drives) by introducing a kernel command line argument to locate boot devices via partition UUID. Notably, UUID parsing is case-sensitive and requires lowercase formatting. Android also enabled more aggressive asynchronous kernel module probing and introduced ashmem compatibility patches to support legacy applications expecting ashmem IOCTLS on the upstream MEMFD driver.

### Common Development Breakages Amit identifies recurring integration challenges: HID-to-AIDL framework transitions requiring device manifest updates; build configuration tool changes (such as restrictions on external Android.mk files); aggressive movement toward Bazel builds; and GKI-related failures including symbol list modifications, protected kernel module restrictions, and kernel config changes affecting vendor drivers.

### External Projects: Kernel and Graphics Integrating the Linux kernel upstream and Mesa/Freedeno graphics involves ongoing friction: device tree nodes and configuration files lack stability guarantees and are frequently renamed; LSM policies cause difficulties with peripheral device probing; upstream developers may resist Android-specific bug reports without reproducibility on standard distributions; and firmware-related bugs often go unfixed due to vendor disinterest in legacy devices. Mesa/Freedeno integration introduces build breakages from cross-platform changes and incomplete support for certain GPU features.

### Major AOSP Policy Shifts Two significant policy changes affect the community: AOSP main branch entered read-only mode in March 2025, with development moving to the new "Android latest release" branch. Changes submitted to this branch require cherry-picking by Google into internal builds before appearing in quarterly releases—with no notification mechanism for external developers confirming acceptance. Additionally, Google removed Pixel device configurations that had historically served as reference platforms for undocumented AOSP hooks, leaving custom ROM developers without official build targets and forcing reliance on virtual devices like Cuttlefish.

### Community Initiatives In response to these challenges, the Devboards for Android and AOSP devs projects provide umbrella spaces for system developer collaboration. Devboards for Android offers reference boards, pre-built images, and a staging area for pending AOSP patches using AOSP-style code review workflows. These initiatives aim to bridge the gap left by Google's policy changes and foster development of generic AOSP features across multiple platforms and SoCs.

## Takeaways Keeping pace with AOSP requires managing constant updates across the OS core, external projects, and tooling. Community-driven initiatives are essential as official Pixel device support has contracted. Upstream-first development brings long-term benefits but involves friction with kernel maintainers and fragmentation across vendors. The shift to a read-only AOSP main branch and reduced Pixel reference support represent significant changes to the AOSP development ecosystem, making collaborative, community-led projects increasingly vital for device maintainers and custom ROM developers.

Related talks