Write-ups from running these agents in production — what worked, what failed, with the numbers from the logs. When a run teaches something worth writing down, it lands here.
Not a ranking — a method. Fix the eval first, walk the catalog from cheap to expensive, stop at the first pass. Why model choice is per-agent, where the money in an agentic loop actually goes, and when moving up a tier is worth it. (For our measured numbers, see the benchmark post.)
Three ways an agent remembers — word-overlap ranking, embedding similarity, and mem0-style fact extraction — compared on the failure each one actually fixes, with the degrade paths stated out loud.
Your agent is only as good as the thing that scores it. The three eval shapes that exist here — deterministic checks, a goal judge, a rubric judge — compared honestly, and why the eval is the axis every other axis is read against.
An agent is not one thing. It is a goal plus a set of choices — memory, planning, retrieval, state, guardrails, model, tools, frameworks, delivery, evaluation — each one swappable, each one changing what the agent does. A plain-language tour of the parts.
We scored 14 models two ways: as isolated tool calls, everyone lands 0.82–0.95; as real agent loops, completion fans out from 0.06 to 0.77. A leaderboard can't see that gap. The finding, the full table, and what a (small) production corpus adds.
We watched 55 campgrounds every 15 minutes for four weeks. 783 sites came back, and half of them were gone in 45 minutes. A quarter never survived a single check.