Listen in the Podli app 🎧
Follow your favourite podcasts, listen offline and in the car with CarPlay and Android Auto, and always pick up where you left off. Free to try.
Episode Summary
Dave spent a weekend running AI models entirely on a 24GB MacBook Air with no fan, and used them to build a zooming, panning video from a script for $0. Six months ago the same job meant signing up for a stack of paid APIs. Dave and Dan walk through the local stack, where it falls short of frontier models, and the use cases where local inference already makes sense.
▶ Subscribe to The Velocity Lab on YouTube: youtube.com/@app-vitals
Key Topics
- Building a full video from a script on a MacBook Air, with every model running locally and no internet connection
- The free local stack: OpenCode in the terminal, Ollama serving the model, open weights downloaded from Hugging Face
- Memory is the real limit: Ollama used 22 of the laptop's 24GB, and linking multiple Macs over Thunderbolt pools memory for larger models
- The next step up: a roughly $5K Mac Studio with 128GB of RAM, which Dave estimates can run a model in Sonnet's range
- A privacy pattern from Dan's neighbor: route mail through a local model first to strip sensitive data like Social Security numbers, then hand the heavy lifting to Claude Code
- Why Apple shipping a small on-device model with the OS could make sense, and why App Vitals is making Shipwright's core agent model-agnostic
Notable Quotes
- "Turns out I can build an awesome video off of a script locally for $0."
- "But for five grand, you can run a model that's free, that's local, that's equal to Sonnet."
- "Six months from now, when I retry this effort, I'm gonna be even more blown away."
About The Velocity Lab
Dave O'Dell and Dan McAulay work inside engineering organizations every day helping them ship faster with AI. No hype, no BS — just what's working in the field.
Subscribe: RSS