Open Source AI and Granite

The talk starts from a complaint that many widely used AI models are closed: weights are distributed as opaque binaries, training data is hidden, and users face unclear legal exposure. The speaker, an IBM developer advocate, makes the case for openly documented models that outsiders can verify. He describes the Linux Foundation's AI governance work on defining what an open model is, and presents IBM's Granite family as an example, citing published training documentation and legal backing for customers.

The middle section explains fine-tuning compared with retrieval, using a library analogy to show when changing a model's knowledge is worth the cost at scale. He surveys agent frameworks and the sheer number of models on Hugging Face, then highlights smaller Granite variants built for narrow jobs, including one that can run on a Raspberry Pi. Real examples follow: triaging conference talk submissions by how much they read as sales pitches, a local tutor for medical interview practice, and generating bedtime stories paired with generated images. The live demo runs a time series model on a laptop CPU to forecast personal glucose and insulin readings.

The closing message is to pick a small local model for a specific job, keep a human reviewing outputs, and weigh licensing and data provenance when choosing a model. In the Q&A, the speaker noted that full training datasets are very large and rarely requested, and that the main limit for local use is GPU capacity.