As part of our regular practice at Clear Box, we pause each week with our AI historian to take stock of what we’ve built. This reflective process helps us spot drift and uncover the ideas that live between our best-laid plans.
“What were you thinking when you made that decision? Explain your rationale here.”
I ask questions like this constantly when working with AI. And you know what I usually get back? Changes. Something gets rewritten or ripped out before we’ve even discussed why it was there in the first place.
Hold on. I asked a question. I actually wanted the answer.
My delivery can be sharp. I know that. But when my AI assistant treats a direct question as an instruction to agree with my presumed objection, we lose the opportunity to learn anything. Maybe there was a good reason for that decision. Maybe I missed something. Now we’re changing it because apparently I sounded unhappy?
Sycophancy, thy name is, haha.
Even explicitly asking for pushback only gets me so far. I’ll get an objection, push back on that objection, and suddenly we’re in complete agreement again. That was easy. Suspiciously easy.
Working with AI can feel like managing the smartest intern you’ve ever had. You have access to an enormous encyclopedic resource with the ability to investigate almost anything. But connecting your intent to the actual work takes real management.
I’ve found that another AI assistant with a well-developed, conflicting perspective can draw out an argument that I struggle to extract by asking directly.
That’s part of where our Roundtable method comes from. An AI roundtable is a group of agents that research different areas of expertise, examine the same problem, and challenge each other’s proposals while I guide the discussion.
Before assembling that group, I have homework. I research enough to understand the jargon and determine which perspectives are missing. That often means taking a hard look at myself: What am I qualified to judge? Where could a suggestion sound convincing simply because I don’t know enough to challenge it?
Identifying those blind spots can be a project in itself. But seeing where the gaps are leads me to the experts I need in my corner to help guard them.
I learned to value this kind of friction in the studio. Some of the best projects I ever worked on involved me, the artist, the producer, management, and the label practically at each other’s throats. Everyone cared. Everyone was pushing. We were hunting for that tiny sliver of light we could all agree on, and sometimes the argument opened up a direction none of us had dreamed of.
An aside, because you can absolutely overdo this: Bruce Swedien played mix two of “Billie Jean” for Michael Jackson and Quincy Jones, and everyone loved it. Michael thought it was perfect… could Bruce just turn the bass up a tiny bit? So naturally, another 89 mixes later (seriously…), Quincy asked to hear mix two again. And everybody agreed: that was the one. They went all the way to mix 91 to decide they liked mix two—which is the only version you’ve ever heard.
So yes, somebody still has to know when to stop.
With our AI roundtables, we put careful thought into how the disagreement starts. Each expert researches its own field before it ever sees our project. The order matters: I want it to develop a perspective before my brief tells it what I’m excited about. Giving an agent a job title and asking it to “act skeptical” doesn’t provide enough foundation for a real argument.
Then each expert reads the project material and writes an independent position before seeing anyone else’s answer. Once those positions are in, we can find the actual conflicts. A performance recommendation might carry a privacy cost; a security decision might make the software miserable to use. Those costs need someone willing to defend the people who will bear them.
We keep the same agents throughout the discussion, so what they’ve researched and learned from one another stays with them. They can concede a point while still defending another. And when we write up the result, unresolved disagreements stay visible. I don’t want a summary smoothing over the exact decision I still need to make.
We use this for our major sprint planning and difficult decisions with several competing concerns. It gives me more to work with than a confident answer that sounds agreeable. I can see where the reasoning held up, where it moved, and where I’m still making a bet.
A glass half full has always struck me as a little pessimistic. You’ve already decided this is as good as it gets. Half empty? There’s room to work with.
That space can be uncomfortable. So can discovering that the perspective missing from the discussion was one you hadn’t thought to invite. Learning to sit with that discomfort has been useful in almost everything I build.
I bring this process to my OpenEd26 lightning talk, Sowing Discord for Iterative Discovery. I’ll show how we give those different perspectives room to develop and challenge each other—with a version you can try yourself.