Mind the Gap - A Developer's Roadmap To Building AI Agents

### Overview In this talk, Ivan Pedrazas from Docker shares practical lessons, architectural decisions, and tooling developments from Docker's journey in building AI agents. The session traces the evolution from early efforts like the GenAI stack and Docker's internal agent 'Gordon' to contemporary frameworks such as Agentic Compose, Docker Agent Builder, and Model Context Protocol (MCP) integrations. Pedrazas highlights how building agents differs fundamentally from standard software development due to non-deterministic behavior, context constraints, and the shifting dynamics of tool execution and local security.

### Key Topics - **Evolution of AI Tooling at Docker**: Tracing Docker's path from analyzing poorly maintained Dockerfiles on GitHub to announcing the GenAI stack, developing the Gordon agent, launching Docker Model Runner, and supporting multi-framework setups via Agentic Compose. - **Agents vs. Workflows**: Distinguishing between deterministic workflows (or rules engines) and autonomous agents powered by LLMs that make independent runtime decisions. - **Model Context Protocol (MCP) and Tool Budgeting**: How MCP connects LLMs to external tools and services, the role of an MCP Gateway/registry, and the criticality of managing 'tool budgets' to prevent exhausting context window limits. - **Testing and Error Handling in Agentic Systems**: Rethinking unit and integration testing when interacting with non-deterministic agents, including how tools handle initialization failures and errors. - **Agent Security and Local Secrets Management**: Why standard local development security assumptions fall short when agents run third-party MCP servers loaded with sensitive API keys, and Docker's approach to process-isolated secret injection.

### Technical Details - **Agentic Compose and Agent Frameworks**: Docker introduced Agentic Compose to orchestrate agents alongside models, tools, secrets, and volumes in Compose files. It provides reference architectures (such as the Sock Shop demo) across frameworks including CrewAI, Agno, and A2A. Tooling has progressed toward Docker Agent Builder and running agent CLI tasks via 'cagent' and 'dagent'. - **Non-Deterministic Error Recovery in MCP**: Traditional APIs fail early on missing configurations. Pedrazas demonstrates that MCP servers should instead connect and expose tools even with invalid configurations. This allows the LLM to inspect errors and proactively invoke self-healing or setup tools (e.g., executing a Docker command to spin up and initialize a local Couchbase database on behalf of the user). - **MCP Gateway**: Acts as an API gateway/ingress between the agent and various MCP servers, providing a centralized boundary to apply policies, manage interceptors, and ensure agents call tools securely. - **Local Secret Isolation**: While developers historically tolerated plaintext secrets on local workstations, running multiple MCP servers with third-party tokens (e.g., Notion, Google) demands stronger boundaries. Docker introduced container labels and local secret definitions (e.g., via `local mcp secrets`) where secrets are injected as files. Only the specific container process can read the secret file; inspection via `docker debug`, `docker exec`, or lateral access from another container results in 'permission denied'.

### Takeaways - **Adaptability Over Sunk Costs**: Rapid iteration in AI requires developers to be willing to drop tools and frameworks they spent months building if better paradigms emerge. - **Design for Agent Consumption**: Build MCP tools recognizing that the caller is an autonomous, non-deterministic agent rather than a human or rigid API client; error messages should empower the agent to self-correct. - **Security Cannot Be Deferred**: The introduction of autonomous agents with access to broad tooling necessitates robust gateway interception and strictly bounded secrets management even within local developer environments.