Security Observability 101: Thinking Inside the Box!

Jeff Williams opens by arguing that software is hard to secure because it runs as a black box, and that most application security work relies on after-the-fact tools like logs, scans and pen tests that see little of what happens at runtime. He frames instrumentation as a way to watch an application from inside while it runs, pointing to fitness trackers and factory sensors as familiar examples.

The middle of the talk is a live demonstration with JOT, a free and open source Java observability toolkit that attaches as a Java agent and uses YAML rules to place sensors on chosen methods without writing code. He runs a deliberately vulnerable app and captures unprepared SQL statements, logs which encryption algorithms are used across the whole stack, and builds coverage and access control matrices by recording routes, users and role checks as he clicks through pages. He then blocks an expression language injection by prohibiting process execution inside the evaluation scope, and shows that a failing sensor can fail a developer's ordinary test run.

To close, he contrasts this runtime approach with network firewalls and point-in-time scanning, and describes interactive testing and runtime self-protection as commercial versions of the same idea. He also notes that much library code in a typical application never executes, which makes many scanner findings false alarms, and warns that compromised developer machines can inject malicious code into release pipelines. The takeaway is that runtime visibility gives security, development and operations a shared, accurate view of how software behaves.