GeeCON Prague 2017: Todd Montgomery - Making It Count: Quality is Not Optional

### Overview In this talk from GeeCON Prague 2017, Todd Montgomery discusses software quality, safety, and accountability, drawing from his early career researching safety-critical NASA systems (including the Space Shuttle and International Space Station). Montgomery argues that quality cannot be treated as an afterthought or patched into a system late in development; rather, safety is an emergent property that depends fundamentally on quality, and quality relies on engineering pride, ownership, and responsibility.

### Key Topics - **Safety as an Emergent Property**: Reflecting on the 1986 Space Shuttle Challenger disaster and subsequent National Research Council report co-authored by Nancy Leveson, Montgomery highlights that safety cannot simply be designed into isolated components. Instead, it must emerge from the system as a whole. NASA established Independent Verification and Validation (IV&V) facilities to scrutinize software and mission risk independently. - **The Fallacy of "Non-Functional" Requirements**: Classifying aspects like performance, robustness, safety, stability, and security as "non-functional" requirements devalues them. Montgomery explains that these are really unspoken or incomplete functional requirements that teams mistakenly ignore until a crisis occurs. - **Reactive Fixes vs. Root Causes**: When quality failures emerge, organizations commonly react by throwing testers at the problem, slapping patches on issues, mandating source code reviews, and holding bug triage meetings. Montgomery terms these reactive steps placebos that fail to solve the underlying root cause. - **Dogma and Lack of Silver Bullets**: Software quality does not stem from choosing Agile over Waterfall, adopting specific frameworks, selecting functional versus object-oriented paradigms, or picking specific programming languages. Quoting Fred Brooks, Montgomery emphasizes that there is no silver bullet, and treating methodologies dogmatically is the antithesis of quality. - **Engineering Accountability and Legal Precedents**: The era of software engineering evading legal accountability through end-user license agreement (EULA) disclaimers is ending. Citing the criminal prosecution and 40-month prison sentence of a Volkswagen engineer in the diesel emissions scandal, Montgomery argues that software engineers will increasingly be held accountable for decisions and ethics, just like traditional electrical or civil engineers.

### Technical Details & NASA Case Study Montgomery shares lessons from evaluating NASA software systems to illustrate practical approaches to quality: - **Assumptions and Edge Cases**: In one case involving a shuttle GPS tracking unit that was flown without prior IV&V, an analyst discovered a critical defect simply by asking the engineer to articulate baseline assumptions. The engineer had hard-coded an assumption that no more than 12 satellites would ever be visible at once. At high altitudes, more satellites become visible, causing memory corruption when writing past array bounds. - **Specifications as Communication**: Exhaustive binders of specifications are far less valuable than the collaborative thinking that produces them. A specification can be an email exchange, structured documentation, or executable specifications such as Test-Driven Development (TDD) or Behavior-Driven Development (BDD) test suites.

### Takeaways - Software quality cannot be outsourced to tools, languages, or processes; it is driven by personal commitment, team pride, accountability, and a sense of ownership. - Early requirements analysis—specifically having domain experts conduct conversational interviews with developers to probe assumptions—yields the highest return on investment. - Developers should strive to develop good engineering "taste" and craftsmanship, ensuring quality throughout the system down to variable naming and maintainability, even in areas end users will never directly see.