JDD2019: Java in Container - Best Practicies for containerized Java Apps, Mateusz Dymiński

### Overview In this presentation at JDD 2019, Mateusz Dymiński outlines practical best practices for containerizing Java applications using Docker. Starting from a basic Spring Boot REST service connected to MySQL, Dymiński demonstrates how standard approaches—such as pulling arbitrary Ubuntu-based Java images and packaging monolithic fat JARs—result in severe pitfalls. These issues include ignored termination signals, massive image sizes (approaching 1 GB), slow build pipelines, resource misallocations, and security liabilities. He then details concrete architectural and configuration patterns to optimize container builds, runtime ergonomics, and operational security.

### Key Topics - **Signal Handling & Process Execution**: Ensuring SIGTERM and termination signals properly reach JVM processes for graceful shutdowns. - **Base Image Selection**: Evaluating base Linux distributions (e.g., Ubuntu vs. Alpine) and Java Runtimes (Oracle, OpenJDK, AdoptOpenJDK, Red Hat, Amazon Corretto, Liberica), alongside C library compatibility issues (glibc vs. musl libc). - **Layering & Multistage Builds**: Deconstructing fat JARs into cached layers and decoupling build environments from runtime images. - **JVM Container Ergonomics**: Navigating the evolution of JVM awareness regarding Linux cgroups, CPU quotas, and memory limits across Java versions. - **Container Security & Automation**: Enforcing non-root user execution, read-only filesystems, vulnerability scanning with Clair, and leveraging Google Jib for automated containerization.

### Technical Details - **Signaling and Entrypoint Formatting**: Using string syntax for Docker `ENTRYPOINT` triggers a shell subshell that swallows Unix signals like `SIGTERM`. Containers must use the JSON array syntax (`["java", "-jar", ...]`) or `exec` in shell wrappers to allow the application to receive `SIGTERM` and perform graceful draining before Docker issues `SIGKILL` after the 10-second timeout. - **Image Sizing and Portola**: A default Ubuntu base with Java yielded an 881 MB image. Switching to an Alpine-based OpenJDK 8 image reduced the size to 124 MB. However, Alpine relies on `musl libc` rather than `glibc`. While Java 8 supported musl, Java 9 through 11 suffered compatibility issues, which Project Portola worked to solve (reaching near-completion around Java 13). - **Unpacking Fat JARs & Multistage Builds**: Repacking a 40 MB fat JAR on every code change forces Docker to push the entire layer to registries. By unpacking the JAR into dependencies (`BOOT-INF/lib`) and application code (`BOOT-INF/classes`), unchanged dependencies remain cached, reducing rebuild diffs to kilobytes. Combining this with Docker multistage builds (`FROM ... AS build` and `COPY --from=build`) ensures only source code and POM files are sent to the daemon. - **JVM Ergonomics Across Versions**: Earlier JVMs (prior to Java 8u121/8u191) calculated thread counts, garbage collection, and heap sizes from host VM resources rather than container cgroups, requiring workaround scripts like Fabric8's `run-java.sh`. Starting in Java 8u191 and Java 10/11+, flags like `UseContainerSupport` and percentage-based heap configurations make JVMs natively adapt to container memory and CPU quotas. - **Hardening and Jib**: Dymiński recommends adding non-root users (`USER app`), running containers with `--read-only`, and scanning images against CVE databases using CoreOS Clair and Clair scanner. For developers seeking to bypass manual Dockerfile management, Google Jib (via Maven or Gradle plugins) builds optimized, layered images on top of Google's distroless base automatically.

### Takeaways - Use Java 11 or at least Java 8 update 191+ to ensure native container cgroup awareness. - Always invoke Java processes using the JSON array entrypoint syntax to enable graceful shutdown via `SIGTERM`. - Unpack fat JARs across Docker image layers or use multistage builds to drastically cut build times and registry bandwidth. - Enforce container security best practices: drop root privileges by setting a dedicated application user, use minimal distros, and scan for known CVEs. - Consider tools like Google Jib to automate best practices and GraalVM native images to achieve ultra-fast startup times without a traditional JRE.