GeeCON 2023: Konrad Szałkowski - How to make your electrons do more in Java

### Overview In this GeeCON 2023 presentation, Konrad Szałkowski explores how Java developers can achieve maximum hardware performance by taking advantage of SIMD (Single Instruction, Multiple Data) CPU operations and offloading parallel computing workloads to GPUs and FPGAs. Historically, Java lacked low-level primitives to directly access specialized hardware acceleration, relying primarily on C2 compiler auto-vectorization (superword optimization) or JVM intrinsics. The talk examines modern alternatives that bring direct hardware execution to the JVM: the Java Vector API introduced in Java 16 and the TornadoVM framework developed at the University of Manchester.

### Key Topics - **SIMD Architecture and History**: Overview of vector instructions executing a single operation over multiple data points (e.g., additions, multiplications, vector reductions). Historical context traces hardware vectorization from the Intel Pentium MMX (1997) through SSE, AVX, and ARM Neon/SVE, highlighting use cases across gaming, 3D graphics, physics simulations, image processing, and artificial intelligence. - **Java Vector API**: Introduced in Java 16, the Vector API provides an explicit, high-level programming model that compiles down directly to platform-native SIMD instructions without bytecode execution overhead. - **GPU Programming Challenges in Java**: Hardware architecture differences between CPUs and GPUs, such as massive parallelism across thousands of cores, distinct memory spaces (host RAM vs. GPU VRAM over PCIe), lack of garbage collection on GPUs, and differing mental models for data and thread mapping. - **Historical and Modern JVM GPU Frameworks**: Previous attempts to bridge Java and GPUs (JCuda, JOCL, Oracle's Project Sumatra, Project Rootbeer, and FPGA frameworks like MaxJ), contrasting them with TornadoVM. - **TornadoVM**: An open-source plug-in leveraging GraalVM to compile Java code (particularly static methods annotated for parallel loop execution) to execute seamlessly on SIMD CPUs, GPUs, and FPGAs.

### Technical Details - **Vector API Mechanics**: Uses `VectorSpecies` as an abstraction representing element type and vector bit-width (e.g., a 128-bit species holding four 32-bit integers). Demonstrates loading batches via `VectorSpecies.fromArray()`, stepping with loop offsets based on species length, and manually handling the tail/remaining elements that do not fit into full vector widths. - **Vector API Benchmarks**: The speaker shows naive benchmark comparisons on large arrays: summing 100 million integers completed in 26 ms with scalar Java versus 20 ms with the Vector API; max search dropped from 36 ms to 20 ms; and quadratic equation solving demonstrated substantial speedups. - **TornadoVM Architecture and Ray Tracer Demo**: TornadoVM constructs an execution plan via a `TaskGraph`, managing explicit data transfers to and from device memory and dispatching computational tasks. A live 3D Ray Tracer demo running on a CPU executed at roughly 1.5 FPS, reached 3 FPS using CPU parallel streams (ForkJoinPool), and maintained smooth high-framerate real-time rendering when executed on the GPU. - **Dynamic Reconfiguration**: TornadoVM features dynamic reconfiguration, assessing at runtime whether data transfer latency across the PCIe bus negates GPU execution advantages; if so, it automatically falls back to CPU execution.

### Takeaways While the Vector API gives Java developers explicit control over SIMD processing with measurable latency improvements, it requires manual handling of tail iterations and extra boilerplate. For massively parallel tasks, TornadoVM abstracts driver initialization, memory allocation, and kernel dispatch across heterogeneous hardware (CPUs, GPUs, integrated graphics, and FPGAs) using GraalVM. Developers can leverage these tools on local hardware or cloud GPU instances (e.g., AWS EC2) to accelerate computationally heavy workloads in Java.