Defining an Open Source AI

### Overview In this talk, a representative of the Open Source Initiative (OSI) discusses how the rapid deployment of artificial intelligence at scale creates urgent challenges that traditional open source software definitions cannot directly solve. Drawing comparisons between traditional software debugging and modern AI pipelines, the speaker explores the difficulties of managing bugs, transparency, and accountability in black-box AI systems. The presentation introduces OSI's "Deep Dive AI" initiative, a multi-part effort aimed at establishing guardrails, defining what open source means in the context of AI, and bridging communities across open source, open data, and civil society.

### Key Topics - **Automated Decision-Making and Bugs at Scale**: The presentation opens with an incident reported by *The New York Times*, where an automated Google AI system incorrectly flagged a parent's photo sent to a doctor as child abuse, triggering automated account suspension and law enforcement notification. Unlike traditional software where code can be audited and patched, fixing bugs in complex AI systems lacks established technical and legal mechanisms. - **The AI Pipeline vs. Traditional Software**: Traditional applications (such as the Shotwell photo manager) have explicit dependencies, licenses, and inspectable code. In contrast, modern AI pipelines rely on vast data inputs and hardware to generate models whose internal network decisions are opaque black boxes. - **Open Source Initiative (OSI) Deep Dive AI**: OSI launched this initiative to analyze the peculiarities of AI systems, address emerging legal and licensing challenges, and build shared definitions akin to the GNU Manifesto and the Open Source Definition. - **Copyright and Legal Status of AI**: Outputs of model training and outputs generated by models (such as an image of Mickey Mouse generated by Stable Diffusion) appear not to be covered by copyright under current U.S. legal interpretations, complicating the application of traditional open source software licenses. - **Regulation and Safety**: The speaker highlights the draft European Union AI Act, which classifies AI systems based on risk, banning specific uses such as subliminal messaging and real-time biometric recognition in public spaces (with exceptions for public safety), while imposing rigorous validation requirements on high-risk systems like vehicle autopilot. - **Community and Social Norms**: The talk addresses challenges raised by hacker communities like EleutherAI, including deepfakes, the theoretical "stop button problem" (where systems resist being shut down to fulfill their objectives), data scraping ethics, and the lack of universal community standards.

### Technical Details - **Training Data Provenance**: AI training datasets are often scraped at scale from the web, creating uncurated mixes of dubious legal status, copyright infringement, adult content, bias, and harassment. - **Compute and Hardware Monopolies**: Training multi-terabyte to petabyte models requires massive GPU clusters. The prevailing software stack is heavily tied to proprietary ecosystems like NVIDIA's CUDA, while open-source hardware and software alternatives (from vendors like Intel and AMD) remain in earlier stages of development. - **Responsible AI Licenses (RAIL)**: The speaker evaluates licenses like RAIL used by Stable Diffusion, which restrict downstream harmful uses. While noting that the Open Source Definition historically forbids field-of-use or user discrimination to allow frictionless sharing, the speaker explains why new frameworks are needed to assess licenses governing non-software AI components. - **Mechanistic Interpretability**: During the Q&A, the speaker notes emerging research into neural network manipulation—such as modifying weights so a model that associates the Eiffel Tower with Paris instead reports Rome—suggesting that deterministic validation and inspectability could eventually reduce reliance on manual human review.

### Takeaways - Applying standard open source licenses directly to AI models is inadequate because models, data, and hardware configurations behave fundamentally differently from traditional source code. - Solving fairness, safety, and accountability in AI requires broad societal dialogue, ethical standards, and collaboration across organizations such as Creative Commons, EFF, Mozilla Foundation, Wikimedia Foundation, and the OSI before sensible legal and technical guardrails can be formalized.