Local Language Models in OT - Basics and Considerations

The speaker, who spent years as a gas turbine controls engineer before moving into OT security, argues that most AI discussion centers on frontier models that plants often cannot use because of regulation, data sensitivity, or isolated networks. Local models, including quantized versions that run on a laptop or desktop, are presented as a way to get value from OT data while keeping it on site. Benefits include privacy and governance, predictable costs, low latency on a local network, and consistent behavior that does not shift when a vendor updates a model.

Use cases covered include drafting policy and compliance mappings, building knowledge bases from old equipment notes, helping sort alarm floods after a process trip, suggesting setpoints for changing conditions, and speeding up slow controls tuning. On hardware, the talk compares consumer NVIDIA cards with limited video memory, unified-memory systems from NVIDIA and Apple, and renting compute to test before buying. Open-weight model families such as Gemma, Qwen, and Mistral are mentioned, with tooling built around llama.cpp-based runtimes, GGUF and MLX formats, and fine-tuning workflows.

The lessons learned focus on practice. Early setups should stay connected to the internet so dependencies can be downloaded, with air-gapping added later. Users should plan the end goal first rather than follow a fixed flowchart, check every generated command before running it, and compare several models because none suits every task. A demo that analyzed decades-old gas turbine data showed the work dropping from about three hours to under twenty minutes.