Skip to main content

Lab-Book 2026-07-19 How to Get Your Thinking Time back from LLMs

 I've seen a few posts in the last week that talk about engineer's loosing their time to think because they're too busy watching what their LLM agents are doing.

Anthropic's models, even under GasTown, have decide to spontaneously fork and find other ways to farm work out to subagents, occasionally causing agent storms on my project in the last week, so I can sympathize a bit more now that I did at the start of the week.

There are a few solutions I'd like to propose for this issue. Here goes.

How to Get Your Thinking Time Back

  • First, if you're agents are doing marginally what you want, this is pretty simple. Stand up, back away from your desk, and leave. If it makes you feel better, setup a notification so the agent messages you when it's done. Perhaps, something like this


  • If you don't trust your agents for the moment, still don't watch them. Come up with metrics to know if they worked or not and control your agents or modify their activities AUTOMATICALLY. Once again, walk away. For example, my next task will be to either have the GasTown mayor pole my agents to make sure they're not creating two many subagents. Since GasTown is actually constructed for exactly this kind of thing, I can ask the mayor to control the agents by nudging, or in extreme cases nuking them.
  • You don't want to have the mayor doing this forever, so the next step is to document what went wrong. Yes, in an issue database.
  • Set up an eval space where you can try different prompts or control structures until the bad behavoir stops. Good experimental method says to change one thing at a time, measure resultts, and iterate. Agents are chatty though, so you'll want to use an isolation strategy like this one. 
  • Once you've got a fix, document it and revision control it. Do not complain. You can have the LLM do most of this for you.
  • Setup a timed analysis run to repeat the experiment with your  fix. Run it perhaps once a day to start, and then maybe thin out to once a week or month when the test case keeps passing. The point here is that models create somewhat random rsults. When models are changed they can definitely relapse to old behaviors. You need a way to automatically detect that that has happened and deal with it. Thing of the reruns (erm regressions) like a retry exponential fallback timing loop.
And that's that! Ahem... Yeah, it's some work. It is, however, work that I'll be doing with you this week, so I'll keep you posted on my findings and progress as I go.



Labels:

Comments

Post a Comment

Please leave your comments on this topic:

Popular posts from this blog

Cool Math Tricks: Deriving the Divergence, (Del or Nabla) into New (Cylindrical) Coordinate Systems

Now available as a Kindle ebook for 99 cents ! Get a spiffy ebook, and fund more physics The following is a pretty lengthy procedure, but converting the divergence, (nabla, del) operator between coordinate systems comes up pretty often. While there are tables for converting between common coordinate systems , there seem to be fewer explanations of the procedure for deriving the conversion, so here goes! What do we actually want? To convert the Cartesian nabla to the nabla for another coordinate system, say… cylindrical coordinates. What we’ll need: 1. The Cartesian Nabla: 2. A set of equations relating the Cartesian coordinates to cylindrical coordinates: 3. A set of equations relating the Cartesian basis vectors to the basis vectors of the new coordinate system: How to do it: Use the chain rule for differentiation to convert the derivatives with respect to the Cartesian variables to derivatives with respect to the cylindrical variables. The chain ...
10 comments
Read more

The Alcubierre Warp Drive Tophat Function and Open Science with Sage

I transferred yesterday's Mathematica file with the Alcubierre warp drive[2] line element and space curvature calculations to the  +Sage Mathematical Software System  today, (the files been  added to the public repository [3]).  If you haven't used Sage before, it's a Python based software package that's similar in functionality to Mathematica.  Oh, and it' free.  I also worked a little more on understanding the theory, but frankly, I made far more progress with the software than the theory.  What follows will be a little more of the Alcubierre theory, plus, a cool Sage interactive demo of one of the Alcubierre functions[1], as well as a bit about my first experience with using Sage. Theory The theory is fun, but it's moving slowly.  Here's the chalk board from this morning's discussion Alcubierre setup the derivation using something called the 3+1 formalism which means we consider space to be flat, (in this case), slices that are labelled ...
2 comments
Read more

The Valentine's Day Magnetic Monopole

There's an assymetry to the form of the two Maxwell's equations shown in picture 1.  While the divergence of the electric field is proportional to the electric charge density at a given point, the divergence of the magnetic field is equal to zero.  This is typically explained in the following way.  While we know that electrons, the fundamental electric charge carriers exist, evidence seems to indicate that magnetic monopoles, the particles that would carry magnetic 'charge', either don't exist, or, the energies required to create them are so high that they are exceedingly rare.  That doesn't stop us from looking for them though! Keeping with the theme of Fairbank[1] and his academic progeny over the semester break, today's post is about the discovery of a magnetic monopole candidate event by one of the Fairbank's graduate students, Blas Cabrera[2].  Cabrera was utilizing a loop type of magnetic monopole detector.  Its operation is in...
1 comment
Read more