Evaluating Autonomous Vehicle Resilience

Overview: Jian Hu and Sachin Gupta from Zoox evaluate the resilience and safety boundaries of Level 4 autonomous vehicle (AV) architectures against faulty and malicious teleoperation inputs. The presentation explores the critical role of remote teleoperation, where human operators provide high-level teleguidance to robotaxis navigating ambiguous, complex real-world driving environments. The speakers introduce a novel, high-fidelity scenario-fuzzing framework designed to stress-test the vehicle's autonomous decision envelope and uncover subtle safety-critical software edge cases.

Key Topics: - Level 4 robotaxi architecture and the teleoperation teleguidance pipeline - Adversarial threat modeling: benign operator mistakes vs. malicious command injection - High-fidelity scenario fuzzing using real-world fleet driving seeds - Discovery and root-cause analysis of prediction and trajectory planning bugs - Retraining autonomous driving decision logic and deterministic safety overrides

Technical Details: In a Level 4 robotaxi system, remote teleoperation does not operate like a remote-control video game with a direct steering wheel or joystick. Instead, when an autonomous vehicle encounters an ambiguous scenario—such as an unmapped construction zone with cones and blocked lanes—it requests assistance, and a remote human operator transmits high-level teleguidance in the form of suggested waypoints and clearance vectors. While onboard AI retains full dynamic control of path execution, human errors, insider threats, or compromised teleoperation communication channels could potentially transmit hazardous or conflicting commands to the vehicle.

To systematically evaluate how vehicles react to flawed or adversarial guidance, the researchers developed an automated scenario-fuzzing framework integrated into a high-fidelity simulation environment. Rather than fuzzing raw code or communication bytes, the fuzzer mutates entire driving scenarios using seeds extracted from millions of miles of real-world fleet driving logs. By applying programmatic offsets to teleoperation parameters (such as lateral and longitudinal shifts, path curvature variations, and command timing), the fuzzer automatically generated over 50,000 scenario variants ("mutants") to identify subtle boundary conditions where autonomous safety checks could fail.

The fuzzer successfully surfaced several critical edge-case failures and simulated collisions. In a parking-lot merge scenario, iterative waypoint mutations found trajectories that bypassed the collision avoidance envelope and brushed parked cars. In an intersection scenario, an unexpected reverse teleguidance command caused the vehicle to halt for oncoming traffic, but a subtle timing bug in the perception-prediction module caused the vehicle to accelerate into the intersection immediately after the first car passed, resulting in a side collision. In a right-turn scenario, the fuzzer injected conflicting emergency stop commands mid-turn, stopping the vehicle abruptly in a blind spot and causing an unavoidable rear-end collision by trailing traffic.

Key Takeaways: Autonomous vehicle architectures must never grant absolute trust to remote teleoperation commands, enforcing deterministic onboard safety envelopes that programmatically reject teleguidance trajectories violating dynamic physical boundaries. High-fidelity scenario fuzzing based on real-world driving logs is essential for uncovering non-obvious perception and prediction timing bugs that escape conventional testing. Automated decision stacks must be continuously retrained to identify conflicting operator signals, execute safe lateral avoidance maneuvers, and safely maintain minimal risk conditions.

(AI summary — Gemini Flash)