Atmosphere 2014: Cheffing the department, one dev at a time

### Overview At Atmosphere 2014, speaker Andrzej discusses the practical realities, human dynamics, and technical hurdles of introducing Chef and Infrastructure as Code (IaC) to a newly formed department at eBay. Distributed across London, Zurich, and Berlin, the department comprised between 50 and 100 primarily Java-oriented developers who needed rapid production and staging environments without relying on slow waterfall processes or corporate platform-as-a-service constraints.

### Key Topics - **Transitioning Developers to Infrastructure**: Encouraging Java developers to manage environments beyond application containers like Tomcat or Node, moving away from unscalable Bash scripts and manual configuration. - **Overcoming Tooling and Mindset Friction**: Transitioning Java developers to writing Chef recipes and DSLs in Ruby, as well as handling copy-pasting unverified cookbooks from public repositories like GitHub. - **Human and Cultural Dynamics**: Managing interpersonal conflict, avoiding "rat race" friction during code reviews, and applying techniques from Linda Rising’s book (including sharing sweets like cupcakes and muffins) to build rapport and psychological safety. - **Infrastructure Patterns and Local Testing**: Utilizing Vagrant for fast local testing and adopting the Phoenix pattern (disposable servers) rather than applying ad-hoc manual hotfixes on running instances.

### Technical Details - **Chef Solo vs. Chef Server**: The team initially adopted a Chef Solo approach with a single shared repository on a single master branch to avoid broken commits impacting everyone's machines simultaneously, later experimenting with Chef Server as maturity grew. - **Dual Execution Model**: Managing Chef’s two-phase compile/materialize and converge/execution model, which surfaced unexpected behaviors and required tools like Chef Zero and local testing. - **Quality and Linting**: Enforcing pair programming, code reviews (initially using Review Kiwi), Foodcritic for linting, and ChefSpec for unit tests, followed by validation inside Vagrant virtual machines running on OS X and Linux. - **Cloud Environment and Scale**: Running virtual machines provisioned on eBay's private internal cloud (typically up to 20 boxes per project), serving test and staging environments alongside automated mobile testing infrastructures (utilizing tools such as Selendroid, Selenium iOS driver, and physical device walls).

### Takeaways - Implementing IaC is predominantly a human communication challenge rather than merely a technical one; building trust and allowing developers to make controlled mistakes is essential. - Simplicity should take precedence over complex architectures: start with localized, low-blast-radius tooling (such as Chef Solo and Vagrant) before introducing central servers or elaborate pipelines. - Disposable infrastructure enforces discipline, ensuring all environment configurations remain strictly versioned in code.