Quickstart To Building Your Own Private AI Chat

Sam Panicker opens by framing AI as something even experts do not fully understand, then argues for running models privately rather than relying only on public chat services. He describes the risks he sees with public models, including how easily adversarial prompting can push them off course, and explains why he runs models on his own hardware for work such as assessment notes and reporting.

The middle of the session is a live demo of free, open-source tooling. He walks through LM Studio and Ollama for downloading and serving open models from a model hub, then shows Open WebUI running in Docker as a chat front end. He connects the setup to an Obsidian notes vault so answers draw on his own documents, explains how text and images are split into chunks and turned into vector embeddings for retrieval, and touches on prompt patterns and using a model to generate API code.

He closes with a discussion of control: how a private model can be tuned without the guardrails a public service must enforce, and why that tradeoff matters. The takeaway is that local retrieval-augmented setups are now accessible to newcomers, but model behavior is still hard to predict, so decisions about access and guardrails deserve deliberate thought. The session ends with an open invitation for audience questions.