Docling: Get Your Documents Ready for Gen AI

### Overview In this presentation, speakers Michaela and Caesar introduce Docling, an open-source library and SDK maintained by a 15-person team in Zurich, Switzerland, under the Linux Foundation for AI & Data. Docling focuses on converting and parsing complex documents—such as PDFs, Word files, Excel sheets, and CSVs—into a unified, structured representation known as the `dockling document`. The project addresses common failure modes in document ingestion, including broken multi-column layouts and hidden prompt injection instructions, ensuring documents are accurately prepared for generative AI and LLM workflows.

### Key Topics - **Document Ingestion Challenges & Bad Parsing**: Naive PDF extraction often blends adjacent columns into a single line of text. The speakers highlighted a 1959 scientific paper where separate columns containing the words "vegetative" and "electron microscopy" were merged by parsers into the nonexistent phrase "vegetative electron microscopy," which subsequently poisoned LLM training sets. Furthermore, security and trust issues arise from hidden text, such as white-on-white text containing prompt injections aimed at manipulating automated peer review systems. - **Unified Architecture & Pipelines**: Docling uses customizable processing pipelines depending on document type: - *Basic Pipeline*: Operates without AI to programmatically parse structured files like Word, PowerPoint, and Excel. - *Standard PDF Pipeline*: Employs PDF readers combined with layout analysis, reading-order reconstruction, and table structure parsing. - *Enrichment Pipeline*: Integrates lightweight classification models (such as an 8 MB model for distinguishing charts, logos, and signatures) and vision-language models (VLMs) for generating contextual picture descriptions. - *Small Docling (VLM Pipeline)*: A custom 256-million-parameter vision-language model developed by the team. Small Docling performs layout segmentation, bounding box detection, table reconstruction, and image description in a single shot, and is lightweight enough to run directly on mobile phones. - **Agentic AI & MCP Integration**: Aligning with the growth of autonomous agents, the team introduced Dockling MCP (Model Context Protocol), enabling AI agents to read, summarize, manipulate, and generate Docling documents. In live demonstrations using LM Studio as an MCP host, Docling parsed remote URLs into markdown, extracted specific page thumbnails, and created structured synthetic documents on the fly. - **Ecosystem & Deployment**: Beyond Python SDK and CLI access, Docling natively integrates with Langflow for no-code/low-code pipelines, LlamaIndex, Pydantic AI, and Llama Stack. It features `dockling-serve` (a REST API wrapper) and an OpenShift operator developed in collaboration with Red Hat.

### Technical Details - **Chunking by Structure**: Unlike basic text splitters that cut text strictly by character length, Docling provides specialized chunkers that respect document hierarchy (headers, table cells, lists, and sublists). - **Doc Tags & Standardization**: Docling uses "doc tags" tokens to capture bounding boxes, reading order, and table headers. The team aims to establish this layout representation as an open industry standard for LLM ingestion. - **OCR & Layering**: For handwriting and annotations, the standard PDF pipeline provides modular plug-ins allowing users to swap OCR backends, while Small Docling provides native handwriting recognition out of the box. Additionally, auxiliary content such as slide presenter notes can be routed to separate "content layers" so they do not contaminate the primary export.

### Takeaways Docling provides an open-source, robust document parsing layer essential for high-fidelity generative AI and RAG architectures. By accurately capturing structural context, tables, and multimodal visual elements, it mitigates training-data corruption and prompt-injection risks while offering native integrations for agentic workflows.