LF AI Mini-Summit

### Overview Presented at the Open Source Summit Europe 2020, the LF AI Mini-Summit provided an overarching update on the Linux Foundation AI (LF AI) Foundation and featured technical progress reports across several key open-source artificial intelligence, machine learning, and data initiatives. A major milestone highlighted was the merger between LF AI and ODPi to form the unified LF AI & Data Foundation, aligning open-source AI, ML, and deep learning with data platforms and governance to drive ecosystem interoperability, vendor-neutral hosting, and shared open standards.

### Key Topics - **Foundation Growth and ODPi Merger:** Announcement of the consolidation of LF AI and ODPi under LF AI & Data Foundation, supported by a unified Technical Advisory Council (TAC), governance boards, and specialized committees (ML Workflow and Interop, Trusted AI, and BI & AI). - **TAC Operations and Incubation:** The TAC meets bi-weekly to vote on project proposals, incubation, and graduation criteria (requiring multi-vendor governance and asset transfer). Examples include Horovod's graduation and Feast's incubation acceptance. - **Trusted and Responsible AI:** Initiatives centered on infusing trust and transparency via the Trusted AI Committee, covering ethics principles, use cases, and three donated IBM open-source toolkits. - **Data Discovery and Metadata Management:** Updates on the Amundsen project for indexing, search, lineage, and data quality across modern corporate data stacks. - **Model Standardization and Runtime:** Updates on the ONNX format, ONNX Runtime optimizations across diverse hardware devices, and transformer operationalization. - **Large-Scale Machine Learning Frameworks:** Development updates for Angel, focusing on high-dimensional parameter servers, graph learning, and privacy-preserving federated learning. - **Data Lineage and Observability:** The Marquez project roadmap and the launch of the OpenLineage standard to unify lineage collection across engines and platforms.

### Technical Details - **Trusted AI Toolkits:** IBM contributed three incubation toolkits: AI Fairness 360 (AIF360), which provides over 70 metrics and 10 algorithms across Python and R to detect and mitigate bias in datasets and models; AI Explainability 360 (AIX360), delivering local post-hoc, global post-hoc, and directly interpretable models using contrastive techniques and feature importance; and the Adversarial Robustness Toolkit (ART), designed to evaluate and secure ML models against evasion, poisoning, extraction, and inference threats. - **Amundsen:** Serves as a data discovery platform indexing over 150k tables at Lyft. Key integrations include Apache Atlas, Apache Ranger (for role-based security access), Jira (for data quality issue tracking), and Redash/Tableau dashboards. Q4 roadmap priorities focused on table lineage UX/RFCs and programmatic metadata push/pull APIs, followed by ML feature discovery in Q1 2021. - **ONNX and ONNX Runtime:** ONNX provides a standardized specification supporting deep neural networks and traditional ML models. ONNX Runtime delivers cross-platform acceleration (Linux, Windows, macOS, Android, iOS) supporting backends from NVIDIA, Intel, and AMD. Azure Cognitive Services observed a 10x reduction in model productization time. The runtime also supports inference and training acceleration for large transformer models such as BERT and GPT-2. - **Angel Framework:** Version 3.1 features auto model sharding across multiple nodes in the parameter server, executor auto data parallelism, customizable PS functions, and hash/range partitioners. It includes traditional, embedding, and deep graph learning algorithms, alongside an upcoming federated learning module utilizing encrypted parameter exchange for joint training without raw data sharing (yielding up to a 15% AUC improvement in financial anti-fraud trials). - **Marquez and OpenLineage:** Marquez addresses data availability, freshness, and quality by tracking job-level metadata, schema versions, SQL parsing (Postgres dialect derivatives), BigQuery job details, and Spark execution plans via Java agents. To avoid duplicated integrations across tools like Marquez, Amundsen, and DataHub, the OpenLineage initiative establishes an open standard API for data lineage collection inspired by OpenTelemetry.

### Takeaways The LF AI & Data Foundation acts as a vendor-neutral home offering legal, marketing, certification, security scanning, and technical governance support to help projects transition from single-vendor origins to sustainable multi-vendor ecosystems. Community participation across working groups, mailing lists, and open repositories remains the primary engine advancing standards in machine learning interoperability, trusted AI, metadata discovery, and data lineage.