Copasetic Flow

 Antrhopic's Haiku model can routinely pass —about 300 times so far--the extra class US ham exam based solely on what's contained in its model. (The LLM was not permitted, programatically I might add, to access the interent and therefore the question pool .) I'm studying this to reinforce my grad school business statistics class for use in writing agentic evals. Yesterday, I reasoned that the order the questions were asked might influence the results since the single LLM—essentially a single chat session—might get discouraged if it missed the first question and then project onto a 'poor test taker' part of its model space. I found out I was half right. The question order does seem to matter, but instead of influencing the entire exam, it influenced the results on the first question asked. I ran three 90 exam test fllets, one each with a first question that Haiku always gave the correct answer, the wrong answer, or a mixed result (sometimes correct, sometimes wrong...

 I've seen a few posts in the last week that talk about engineer's loosing their time to think because they're too busy watching what their LLM agents are doing. Anthropic's models, even under GasTown, have decide to spontaneously fork and find other ways to farm work out to subagents, occasionally causing agent storms on my project in the last week, so I can sympathize a bit more now that I did at the start of the week. There are a few solutions I'd like to propose for this issue. Here goes. How to Get Your Thinking Time Back First, if you're agents are doing marginally what you want, this is pretty simple. Stand up, back away from your desk, and leave . If it makes you feel better, setup a notification so the agent messages you when it's done. Perhaps, something like this If you don't trust your agents for the moment, still don't watch them. Come up with metrics to know if they worked or not and control your agents or modify their activities AUTOM...

 With all the work that's been put into making agents "correct" by construction, I gotta say, sometimes I need an LLM agent to take a chance at just being wrong. I'm working on a book project called The Gladych Files . While the book is narrative nonfiction about the history of general relativity research, it explores the liminal space inhabited by very rich fringe scientist speculators of the 1950s who funded mainstream general relativity advances, (more or less on accident.) In those spaces, you'll find Tesla, the architect of the FBI building, Timothy Leary's LSD explorations and many,  many other things, institutions, and people.  I've accumulated hundreds of pages of historical documents from various archives, and I'm using orchestrated agentic AI, (in the form of Gastown), to review those documents. So far, the analysis has gone well, but last week I saw something that made me look up. I'd accidentally input the same archive page twice, so i...

 I deployed the LLM Psy-ops detection app earlier today! For those of you just hopping onboard, the WhyFiles ran an episode highlighting a simple, logical scoring method publicized by NCI for determining if a piece media or new article was emotionally manipulative, (think propaganda), or not.  I was looking for a good app to practice deployment, guardrails, and evals, and this one suggested by a @somethingLethal on reddit seemed promising in all those regards. If you'd like to try it, you can find the app at  https://projecttoucans.com/gladych_files_psy_ops  .  LLMs, Simple Math, and Pricing The Psy-op scoring instrument requires that the model sum the scores for the twenty categories. gpt-4o-mini did not sum any of the scores correctly. It got close, but that was about it. I experimented with the python code interpreter to cure the simple math issue. The code interpreter seemed reasonable at first. I mean, three cents per compute minute , not bad right? Ins...