Working with Process Revision Control

 I took time to play with a new Dolt enabled app example called Quorum last night. Quorum sets 13 LLM agents with different defined personas loose on a users question. The agents come up with solutions to the question and then discuss their individual solutions with each other to arrive at a consensus. There's much more detail in this blog post that accompanies the app.

Quorum is cool. It is not, however, what I wanted to talk aobut here. Instead, I'm going to focus on the blog post for the app. In short, I'm very excited to see ideas that I've used to manage verification processes for years get codified into tools for LLM agents.

Here's one of the important parts

"I can shut down the app, lose the server, or disappear entirely — and the deliberation history remains, publicly accessible and cryptographically verified."

Imagine what an engineer can do to work back through their debug hypothesis tree with that sort of infrastructure!

As the article's author points out, 

"If you ran a Quorum session today and came back in six months, the full reasoning chain for every agent — every score, every decision, every iteration — is still there, publicly accessible and independently verifiable. A regulator can audit it. A journalist can cite it. A policymaker can defend a recommendation with a complete evidence trail."

In other words, as engineers, we can audit our decision making process to learn how to make better decisions next time. This is a huge part of working effectively in an agile team. Essentially, tools like Dolt enable agentic teams to operate in a fashion similar to one the one quoted  below by Steve Yegge, 

"But my SageOx friends Ajit and Ryan actually want the entire work stream to be public, because it’s incredibly valuable for forensics: figuring out exactly how and why a teammate, human or agent, got to a particular spot"

Another useful way I've found to think about all of this is in terms of grad student lab notebooks, 

"it gives them a framework they can’t escape [...] and lets the structure do the work" 

Used properly, grad student lab books—at least in the field of Physics—are repositories of everything that happened in the lab. A grad student should document what they're going to do, then do the thing, then document the results.

Many grad students don't see the value in this until their professor turns up, wants to know what happened and why. The lucky students who just followed the process and kept a detailed lab book—read Dolt database—can simply move back in time with their lab notebook to demonstrate every step. The students who will be ok figure out how to do it for their own efficiency of workflow and to hedge against the next time the professor visits. The unlucky students never pull off any of this and take years longer to complete their PhDs with a pristinely clean notebook in hand. 

Now, think of yourself as an engineering manager or as a graduate physics professor. With agents, you've been gifted a team of effective engineers/grad students. You may as well give yourself an incredibly effective strucutre for managing and leveraging their efforts. Especially since this structure is already built into a growing number of Dolt enabled apps like Quorum and Gas Town.

To me, being able to track work processes has always been the key advantage of working with an issue tracking database integrated with a revision control system. It makes maintaining a system of historic steps that we can build from as we move ahead on a project easy: 

"With Dolt, the reasoning history is the proof. Every commit, every branch, every decision is dated and signed, and pushed to a public remote that no one, not even the person who built the system, can quietly rewrite."