Skip to content

DEF CON 34 talks are here: 354 talks, summarized and searchable.

Browse DEF CON 34

Things You Wish You Knew About Software Testing

Dan Nagle

AppSecTalks47 minIntermediateOpen on YouTube ↗

Where

Talks
HOPE_16

Duration

47 min

Shows practical unit testing, CI pipelines and surprising edge cases like leap seconds and emoji filenames that break software.

Summary

This talk argues that software testing is widely accepted as important but often done poorly, and it offers concrete techniques for both programmers and non-programmers. The speaker organises the material around what he calls code breakers, which are test inputs likely to expose defects. The first example is a leap second timestamp that network time servers distributed, which caused some systems to reject it. He notes that a time service can smear the extra second over a day instead.

The middle section contrasts a fix-and-hand-off workflow with unit testing, where a bug is captured in a failing test that stays in the repository to catch regressions later. A Python example using pytest shows edge-case inputs such as empty lists, negative numbers, garbage input and None. Other breakers include leap-year logic that fails in 2100, 13-month finance calendars, and emoji filenames that trip up uploads and shell tools. The speaker lists common frameworks for other languages and explains how mocks replace databases and network calls in unit tests, while integration tests use real services.

The closing portion covers automation with CI pipelines in GitLab, Jenkins, GitHub Actions and Azure DevOps, split into stages for testing, building and deploying. Slow scans such as static analysis can run in overnight pipelines. A git pre-commit hook that runs a syntax checker can reject broken commits before they reach the pipeline. The main takeaway is to turn every bug into a test, add edge cases early, and automate the checks.

Related talks