AppSecEU 16 - J. Rose, R. Sulatycki - Grow up AppSec-A case study of maturity models and metrics

### Overview At Dun & Bradstreet, an established 175-year-old enterprise, the application security program initially struggled with heavy reliance on outsourced vendors, tool sprawl, non-technical project management, and compliance-driven "security theater." Speakers John and Rohini share their journey of overhauling AppSec into an active, hands-on, internal technical team by transforming culture, upskilling staff, adopting agile delivery workflows, and establishing pragmatic maturity models and business-aligned risk metrics.

### Key Topics - **Internal Upskilling vs. Outsourcing**: Replacing outsourced testing by running intensive internal bootcamps and workshops on tools such as Burp Suite, SQLMap, PadBuster, SoapUI, Postman, and Google dorking to build an authentic internal penetration testing and code review capability. - **Developer Engagement and Security Champions**: Conducting traveling hands-on hacking roadshows based on the OWASP Top 10 for developers, QA engineers, and architects, which fostered collaboration and surfaced local security champions. - **Agile Security Management**: Transitioning security workflows to two-week sprints, daily standups, sprint planning, retrospectives, and Jira tracking, aided by embedded enterprise agile coaches. - **Automation & Vulnerability Correlation**: Deploying custom Python-based automation wrappers around OWASP ZAP for weekly unauthenticated scanning across IP ranges, and utilizing ThreadFix to correlate vulnerabilities across ZAP, Arachni, Burp, Fortify, and third-party audit scans. - **Maturity Models and Goal Setting**: Employing internal scoring across six service dimensions alongside OWASP SAMM (Software Assurance Maturity Model) for roadmap benchmarking, complemented by OKRs (Objectives and Key Results). - **Executive Communication via Business Risk Mapping**: Translating technical vulnerabilities into business impacts using a collaborative card game methodology that maps flaws to technical risks, business risks, and high-level enterprise goals.

### Technical Details To scale coverage across thousands of applications, the team automated ZAP via Python scripts to discover live web apps on company IP ranges and run unauthenticated tests weekly. Vulnerability findings from automated tools (ZAP, Arachni, Fortify, Burp) and external testing were aggregated and de-duplicated in ThreadFix, which integrated into Jira to push remediations directly to development teams. Security health and progress were quantified using core metrics, primarily a **Weighted Risk Trend** (multiplying severity counts by risk values across the application estate) and the **Defect Remediation Window (DRW)**, tracking elapsed time from finding identification to verified closure. For programmatic governance, the team utilized OWASP SAMM to define multi-year maturity targets (such as target levels in training and threat assessment) and instituted quarterly OKRs broken into Jira epics and story points.

### Takeaways - Building an internal technical security team requires investing in people and hands-on tooling rather than relying solely on vendors and compliance checklists. - Defensive security programs benefit from aligning with engineering workflows—using Jira, Slack, sprints, and dedicated backlogs to absorb ad hoc requests. - Translating technical vulnerabilities into direct impacts on core business goals (e.g., system downtime vs. revenue generation) ensures executive support, which can be coupled with top-down accountability reporting and bottom-up developer enablement to accelerate remediation.