As part of our regular practice at Clear Box, we pause each week with our AI historian to take stock of what we’ve built. This reflective process helps us spot drift and uncover the ideas that live between our best-laid plans.
Want to cook Thanksgiving dinner on top of your laptop? Try running local AI and see how hot your machine gets. It’s insane. We need these workflows to be usable on ordinary computers, and we need to stop wasting resources repeating work the machine has already done.
When I worked in the studio, hearing a song for the first time was precious. You could form fresh opinions that gave you a clear, specific direction.
You only get that first listen once; after that, what you already know and assume starts shaping what you hear.
I think about that when working with AI. A fresh-eyed code review is a simple example. The assistant that helped write the code already knows how we got there. A fresh reviewer, however, can examine the code and requirements without carrying the weight of the entire conversation that produced them. Once we have those findings, we can return to the original assistant with the working context intact.
For larger decisions, we take this further with our Roundtable method. I’ve used it for every major coding sprint and for difficult decisions with multiple competing angles. Specialists examine the problem from different perspectives, and we work through what those distinct vantage points reveal.
Managing what my assistants know—and when they know it—is critical to getting good results. Sometimes I want continuity. Sometimes I deliberately want fresh eyes. But I need to move between those perspectives without making every switch an expensive restart.
That’s where it gets painful locally.
My NVIDIA RTX 5090 can prefill a dense 30B model at roughly two to three thousand tokens a second. (Prefill is the processing a model does to absorb context before it even begins to respond.) Even in a simple coding-and-review loop using llama.cpp variants with caching enabled, I was still seeing about 20% of the total time consumed by prefill.
And that’s on a 5090. On a CPU, you’re basically helping bring about the heat death of the universe a little sooner for no good reason. We’re asking the machine to process the same conversation over and over, then wondering why local AI feels slow and the laptop is roasting.
The economics matter. We’re paying in electricity and waiting time for work that could be preserved. If we can make this workflow efficient enough to run on an ARM chip while consuming less energy, isn’t that better for the planet, too?
Sending everything to a data center doesn’t make the waste disappear. Those facilities consume massive amounts of power and require cooling systems with water demands that strain surrounding communities. Why should the cloud be the default answer when we could make better use of the hardware people already own?
That’s what I want from local infrastructure: to make these workflows practical while keeping waste to a minimum. Saving processed context and reading it back gives us a path to do both.
So why are we cooking the same conversation over and over instead of saving the processed state to disk?
That’s the question behind our memory work. The goal is to preserve the context a specialist needs, set it aside, and restore it exactly when we return. Let another assistant approach the work fresh when that’s what the task requires. We need to make those choices affordable enough—in both time and resources—so that people can actually use them.