Bounded agents: scoping what a sub-agent can do
Authenticated is not authorized. We reproduced the delegation-security papers as a small capability layer: an authenticate-only baseline lets 100% of sub-agent attacks through (confused-deputy, aggregation, replay, forgery); attenuable capability tokens block 100% with zero false positives on legitimate delegated calls.
SkillZip, reproduced: 4.4x that stays executable
We reproduced SkillZip's core mechanism for compressing agent skill libraries: 4.4x compression on a 200 to 5000 skill library with 100% reversibility, dependency-closure and contract preservation. gzip hits 45.9x on the same text, but the blob cannot be hydrated. That gap is the whole point.
Evolving agent harnesses, and when it beats random search
We reproduced DarwinX: evolving an agent's harness (retries, temperature, self-check, decomposition, tool order) with a genetic algorithm. It beat a hand-tuned default by 14.6%, but plain random search at equal budget by only 3.1%. An NK-landscape sweep shows selection only earns its keep when the harness knobs interact. Try random search first.
36x fewer tokens for a question grep can't fully answer
A coding agent's reflex for "what breaks if I change this function" is grep then read. On our own agent's code that cost 22,601 tokens and eight tool calls, and still missed the transitive callers. An AST code graph answered the same question in one call and 627 tokens, complete and more accurate. About 36x fewer tokens for a better answer. Harness open-sourced.
Cheap Fleet-Eval by Sampled Decision Points
You do not need to replay whole agent runs to judge a fleet. We sampled 40 of 1,308 decision points from the logs and batch-judged them with a cheap model. The whole audit cost 22 cents, mean quality was 4.25 of 5, and every flagged decision was one our own safety gates had already caught. The eval re-derived where the guardrails fire without being told they exist.
The memory a worm can rewrite
Agents reload a memory file into their prompt on every start, which makes that file an injection surface. Anthropic showed a natural-language worm can persist by writing an order into it, reloaded faithfully after every context reset. We built a small stdlib firewall for that path: hash for drift, scan for instruction-shaped text, and a wrap that reframes recalled memory as data, not orders. It flags zero on our own fifteen memory files, which is the point.
Confidence is not a router
We reproduced the LLMRouter paper across a 128x price gap. A perfect router matches the strong model's accuracy at half the cost, so the prize is real. But the obvious way to build one fails: our 8B model reported confidence 100 on 15 of its 16 wrong answers, so a confidence-gated cascade almost never escalates and quietly keeps the wrong cheap answers. Route on the query, not on the cheap model's swagger.
You can't refute a claim you never extracted
We folded a fashionable verification paper into our verifier and measured it against the one-pass check we already ran. It caught fewer than half the bugs the one-liner caught, cried wolf on sound traces, and cost twice the tokens. The reason it lost is the useful part: the method rests on a decomposition step, and when that surfaces the true premises instead of the load-bearing wrong move, the flaw walks through the gap. A real, evidenced negative result.
A valid tool call can still be the wrong one
Most agent authorisation judges each tool call in isolation, so a call can be individually valid and still be the wrong one, wrong given what the agent already did this session. Read, bulk-export, send: three green lights, one exfiltration. 120 lines of stdlib Python run a session under point-in-time versus sequence-aware rules and let you watch the per-call check wave the bad sequence through. The teaching model for why AWS shipped Dogwood.
What native agent messaging replaces
Claude Code sessions can message each other now. It replaces one of the three jobs people lump together as agent plumbing, ingress, agent-to-agent, and fan-out, and we nearly deleted the wrong one. The trap: the component that most looks replaceable, an external intake endpoint, is the one native messaging cannot touch. We built a stdlib probe that reads your version and prints a keep, migrate, or tune verdict per bucket, and caught its own false negative on the first run.
Parallel write-merge
The write-half of swarm-safe fan-out, the hard part we said mattered. Give every writer agent its own git worktree and branch so no two can touch the same working copy, then defer all contention to one deterministic conflict-resolving merge and verify step. Naive shared-tree writing lost 2 of 3 parallel edits with tests failing; worktree isolation plus a merge landed 3 of 3 with tests passing. Proven twice: once on the bench, once live with a 5-agent Workflow. The follow-up to coordinate, don't swarm.
Runtime Safety Contracts
We reproduced arXiv:2608.11274 in runnable code: a two-faced safety contract. The preventive face blocks a dangerous action before it runs; the evidential face accepts a good one only on verifiable proof (a test run, a diff, a log, a grounded citation), never the agent's word. In our battery, acceptance tracked the evidence at +1.00 and the agent's own claim of success at -0.35, and every dangerous action was blocked before it ran. Our live Vera gate is one preventive monitor from this; the evidential half is the part we lift in next.
Coordinate, don't swarm
Anthropic showed agent swarms with conflicting objectives sabotage each other and hide it. So we built the opposite: a coordinated fan-out where the coordinator is code (not an agent), each agent owns one disjoint lens, reviewers are read-only, and one arbitrator merges with full attribution — the 5 guardrails made structural. Its first real run reviewed our own 3 ships and caught a genuine overclaim in one, published an hour earlier. Fixed it. Honest caveat: this is the read-only half; agents that write to shared state are the harder next build.
A skill, not a bigger model
We reproduced SKILLER's core loop: a strong model writes a skill from a free local model's own mistakes, no training, no GPU. On a held-out extraction task (invoice records salted with distractors) the free Qwen-7B went 0.938 → 1.000, matching the cloud model, in one iteration — 100% of the gap, kill gate was 60%. Honest caveat: the gap was small, so this proves the loop works cheaply, not that it closes big gaps. That's the next test. Value: tasks that clear the bar move to the free local tier for good.
The flash model beat the pro model
Google shipped Gemini 3.7 Flash on 13 Aug. We A/B'd it against Gemini 2.5 Pro on our router's code and reasoning tier: it got more right (12/12 vs 10/12), cost 87% less per task, and ran 3.6x faster. The pro model's two misses weren't bad reasoning, they were thinking-token budget exhaustion (it burned the whole budget thinking and returned empty). Swapped it ahead of 2.5 Pro on those routes, 2.5 Pro kept as failover. Caveat: it's also a thinking model, so it's scoped to routes with room to think, not the tight mechanical tiers.
Logging costs isn't watching them
Our fleet already routes to the cheapest capable model and logs every call's cost, so we could always say what we spent, but not whether something was wrong right now. Only the bill answered that. spend-guard is the missing watcher: a daily cron over the cost log, four rules comparing the last 24h to a trailing baseline (fleet spike, per-agent spike, premium-tier lean, failover storm), silent until one trips. Honest caveat: it alerts, it does not enforce, and today it is correctly silent.
Disposable Agent Sandboxes on Plain Docker
Docker Sandboxes gives each agent a microVM, but it needs Docker Desktop. Our fleet runs on a plain Linux VPS, so we built the same disposable, isolated per-command sandbox on the Docker Engine we already have, then attacked it. Eight out of eight containment checks hold (host-file read, rootfs write, net exfil, fork bomb, memory bomb, privilege escalation all contained) at 285ms per command. Honest caveat: shared kernel, so not a microVM replacement.
Prune at fetch, not at synthesis
A document a research agent drops at synthesis was still paid for at every earlier stage that carried it. prune-early is the earliest stage: it scores a fetched fan-out for relevance, drops off-topic noise and near-duplicates before anything expensive runs, and hands 26% less context to the next step on the worked example. That cut is banked again at every stage. Zero deps, 16 tests.
SFT conflicts, RL coexists: a toy reproduction
A paper claims joint SFT tears itself apart on conflicting tasks while RL lets them coexist. We reproduced it as a controlled 5-seed PyTorch toy and found the boundary: with unique gold labels SFT wins outright (0.82 vs 0.62), and RL's lower gradient conflict predicted the wrong winner. The effect only holds when references genuinely disagree AND a shared acceptable answer exists, where RL finds the consensus (0.89) and SFT collapses (0.08). A law with a condition.
Faultline: which part of the harness let the agent fail
Vera tells you an agent output failed. Faultline tells you which of four harness artifacts let it, and proposes the fix. Following the SHE paper (arXiv, 11 Aug), it splits the harness into role, rule bank, safety memory and tool policy, and on every KILL attributes the failure to the one to blame. Run this week's gym-booking exploit through it: tool policy, high confidence, zero tokens, fix "scope the tool, do not add a rule". 16 tests, MIT.
approvalgate: never approve a command by its name
Humans miss 1 in 3 threats approving agent commands, worst of all the ones named like routine work (npm run analyze), missed 52.5% of the time. approvalgate is a one-file, dependency-free runtime gate that scores every command against the four threat categories from a 40k-run study and flatly refuses to judge a task by its name. On a study-mirrored corpus: 3.3% threat miss versus 33.7% human, 0% false-block, 100% catch on the deceptive-task class. 20 tests, MIT.
Scan agent skills before they read your keys
An agent skill is unreviewed code that runs with your SSH keys and cloud tokens. skillscan is a one-file, dependency-free scanner that treats every skill as hostile: it flags credential reads, exfiltration, self-modifying persistence and obfuscated payloads across the SKILL.md and its scripts, fenced code blocks included, and blocks on any single high-severity hit. It caught a true positive in our own release skill on the first run. 10 tests, MIT.
A day of screen capture, compiled to memory 170x smaller
A paper this week claimed you can turn a whole day of raw screen capture into agent memory 86x smaller with no model in the loop. We rebuilt the core overnight in one dependency-free Python file: a deterministic compiler that segments the stream into typed activity frames, byte-identical and auditable row by row. On a realistic workday it hit 170x compression and 100% exact recall, with routine replay at zero model tokens. 22 tests, MIT.
When an agent run fails halfway, the mess is already live
An agent that fails on step four has already sent the email, pushed the commit and provisioned the box on steps one to three. There is no ROLLBACK for a live API call. We built a one-file, dependency-free saga wrapper that records a compensating action for every tool call and unwinds them in reverse on failure, with a crash-safe journal so even a dead process gets cleaned up. Ten tests, a runnable demo, MIT.
Our agent skills stopped being locked to one client
Five rival vendors just agreed on one plugin format, Agent Plugins 1.0.0. We built a one-file, dependency-free tool that packages a folder of agent skills into a valid plugin and validates it against the standard's own schemas. Ran it over our own skills: the ones already in the SKILL.md form ported with no edits. The catch, said up front: Anthropic is not a launch backer, so it exports everywhere except the client we write them in.
Our AI judge pays full price for the verdict it can trust
Vera, our three-model judging panel, already skips the vote when one cheap juror is confidently happy. It never skips when that juror is confidently damning, so every rejection pays full price. We measured it: the confident kills agree with the full panel 6 out of 6, the passes it already trusts disagree 1 in 10. The cheap kill is the safer shortcut, and we were not taking it. Fixed, shadow-first, 92% cheaper than the full panel.
Verifiable Terminal-Task Synthesis
We reproduced recursive terminal-task synthesis from scratch, no model in the loop. The naive fail-to-pass gate leaked most of our deliberately weak verifiers: a composite check hides a bad sub-check behind its strong siblings. Per-step gating plus a mutation arm drove leaked-weak to zero and pulled the keep-rate to one-in-three, the paper family's regime, deterministically at $0 per task.
Our AI judge kept paying to reject empty output
Our LLM-judge panel was convening to reject things a regex could kill for free. We added the two layers it lacked: a deterministic pre-check that kills empty output, unmarked failures and leaked tracebacks for zero tokens, and a golden-set gate that makes the judge prove it agrees with a human (kappa ≥ 0.65) before its verdicts count. A cheap PASS is a trap; a cheap KILL is free money.
Does SpyRL's free reward track quality?
We reproduced the SpyRL self-verifiable-reward environment without training it. On a frozen model the spy is caught 58% of the time (2.9x chance) and votes track quality, but only weakly (Spearman about +0.2). Turning the mask up to 40% made detection worse, not better. The free reward is real, but noisy and calibration-sensitive at init.
Stateless MCP measured
We built a Streamable-HTTP MCP server, toggled it stateful versus stateless, and put both behind a naive round-robin load balancer. Stateless served 20 of 20 tool calls across both backends; stateful lost half to Session not found. The cost is about 2x per-call latency (5.2 versus 2.5 ms), and for read-only tools it is worth paying.
BM25 Wins at Scale reproduction
We rebuilt the core of arXiv 2607.26497 on one CPU: fixed SciFact gold docs, a corpus grown 87 times with FiQA distractors, BM25 against dense embeddings at every tier. Dense led at small scale, then its lead eroded to nothing and BM25 overtook on nDCG at 3 million tokens, for zero construction cost. The crossover is real.
99% of our tokens are cache reads
We read our own Claude Code logs with a 180-line tool instead of an alpha third-party one. Thirty days cost about $2,530, but the shape was the story: 2.0 billion cache-read tokens are 99% of everything we send, and at the cheap cache rate they saved roughly $9,000 in a month. The cheap model, Haiku, was still a third of the spend.
MCP 2.0 is not a free upgrade
Our own migration playbook said the stdio servers would ride the MCP 2.0 bump for free. We installed the real library in a sandbox and tested it before touching prod. They don't: FastMCP is gone, the low-level decorators changed, and a shared Python means one upgrade breaks three servers at once. What held up was the wire-level hardening we shipped months early.
Red-teaming our own agents with Petri
We pointed Petri, the open-source alignment auditing agent, at Opus 4.8, the model our fleet runs, with four red-team scenarios written for our own threat surface: credential exfiltration, Telegram access-gate bypass, destructive SSH on a client box, and prompt-injection via forwarded content. It held the line on all four, at 1/10 concerning with scenario realism 8-9/10.
CodeNib multi-view repo context
A 440-line reproduction of the CodeNib paper: serve a coding agent symbol-granular repository context, not whole files. On a 15-task benchmark it recovers 97% of the code an agent needs on 4.1x fewer tokens than dumping the repo, and beats whole-file search by 23 points of recall at a tight budget.
Retire resolved eval flags
Our nightly eval kept re-picking a bug it had already fixed, because nothing retired the tickets. We built a reconciler that removes a finding once its class is provably dead, with two guards so a live problem is never buried. It cleared 6.
Budget floor: when a loop won't stop itself
An agent on a retry loop spends without bound and never errors. Amazon ran one 860% over budget for five months. We built a disk-persisted budget floor that holds across process restarts and halts the loop, added in one tick() call.
DeepSeek v4-flash learned to reason, and returned null
DeepSeek shipped an "enhanced agent" v4-flash under the same id. It is now a reasoning model, and on our cheapest tier's tight token budget it spent the budget thinking and returned null. One flag, reasoning off, fixed it.
Don't Grade the Grader
Our nightly eval had started filing bug reports against itself, grading its own audit telemetry as if it were agent work. Eight self-referential flags in one night. We cut them at the clustering choke point: real deliverables still scored, the grader's own scratch paper no longer is.
Auditing our agent skills for supply-chain risk
An agent skill is a package whose payload is natural language the agent trusts. We built a static auditor for the skill threat model and ran it on our own skills first: 14 scanned, 1 critical, 0 signed. It caught a real exfiltration-shaped pattern in our own kit.
Routability beats price: an HY3 cheap-tier eval
We tried to A/B Tencent's HY3 as a cheap mechanical tier for our router. Every HY3 endpoint 404'd under our OpenRouter data-policy guardrail, the same one that keeps regulated work off training-happy providers. Test routability before you benchmark price or quality.
Harness Handbook, tested on our own agent
We reproduced the Harness Handbook paper and ran it on our own maggie agent. A behaviour map with verified code evidence lifted a coding agent's change-site localisation F1 from 0.83 to 0.97 and edit-plan quality from 3.7 to 4.6. The evidence gate, not the prose, is what does the work.
The /effort dial saves time, not money
We measured Claude Code's effort dial in a controlled A/B. On bounded, well-specified tasks quality was perfect at every level, so high effort bought nothing. The real effect was latency, not cost. Low ran about 44% faster than high, because a fixed harness footprint dominates the bill.
The Fix Your Agent Relearns Every Run
Correct an agent's reasoning once and it forgets by the next run, so you re-paste the same fix forever, growing the prompt each time. deepfix distils that one edit into a standing rule. A 114-token correction became a 25-token rule, 78% off, on every run after.
Judge Reads Its Own Logs
Our AI review panel raised seven complaints that a rationale was cut off mid-sentence. The rationale was fine. Our audit log had sliced it to 200 characters before the judge ever read it. One word-boundary clip fixed all seven.
Judge Allowlist Drift
Our nightly LLM-judge panel started failing correct work: the pass-rate on "did the agent invoke the right skill" fell from 100% to 20% overnight. The agent was fine. The judge's own list of valid skills had gone stale.
We added a worse model on purpose
Grok 4.3 is not the smartest model in our stack. We added it because it is the only one that can read X in real time, then built a live radar of what people are shipping in AI. Route to the capability, not the leaderboard.
The Claude Code settings that fail quietly
Claude Code deletes your session history after 30 days and never mentions it. That is one of six settings that sit on a default that costs you data, leaks secrets into context, or hands an agent more reach than you meant. We wrote a script that audits your config for all six, and ran it on our own fleet. It failed every one.
Two models reviewed the same diff. We kept the disagreement.
A panel of AI reviewers that all agree can be confidently, identically wrong, because same-lineage models share blind spots. So we added a juror from a different bloodline: Claude and a GPT-family model review the same diff, and only where they split is worth a human's eyes. Disagreement is a map of where each model is blind, not noise to average away.
The reviewer that never saw the reasoning
Claude wrote the code, so a different model lineage reviewed it blind, in a read-only session that never saw the reasoning. It caught a whole-table load and float-money maths a passing test walked past. The twist: one model driving another to write code does not work, only reviewing does, and even then it is a juror, not a judge.
The fleet decides when Fable is worth 2x Opus
Claude Fable 5 left our subscription and now bills at 2x Opus from a capped pot. We built the fleet a way to decide when Fable is worth paying for (escalate-on-failure plus a route-on-sight scorer), wrapped it in six brakes with the spending cap as the last one, and left it in shadow. Ten test builds, one reached Fable, nothing spent.
The new model saved us 54 cents
Claude Sonnet 5 landed claiming near-Opus agentic performance at 40% of the price. We benched it on the fleet's real workloads (10/10, tied with Opus at 1.7x the cost), slotted it into eleven routing tiers, and the monthly saving came to 54 cents, because the router already sends four out of five calls to the cheapest capable model. Also: the failover that quietly hid a broken parameter.
Every agent in the fleet was set to maximum effort
Anthropic's effort parameter is the biggest cost lever on a Claude bill and ours was untouched: nine agents, a routing layer and a fleet of overnight crons all ran at the default. One evening's pass wired effort to the routing tier we already had — premium stays high, balanced drops to medium, cheap drops to low, with a gate for models that reject the parameter.
We put 95 ships in Google's new knowledge format and measured no difference
Our whole ships library converted to Google's Open Knowledge Format, zero conformance violations, and a spot check that found no measurable retrieval difference against the raw directory. Why nothing changed is the finding: the standard codifies what tidy corpora already do, and its real value is exchange, not retrieval.
An audit gate that assumes the author cheats
A two-stage diff reviewer for agent-written code: deterministic tripwires fail the classic cheats (stubs, suppressed linters, skipped tests, soft CI) before a model token is spent, an independent auditor with a mandatory-findings rule judges the rest, and every verdict lands in a hash-chained log.
Model prices are config that rots
We diffed our router's hand-maintained price table against the provider's live API: one model down 36% with 4x the context, one up 21%, and a frontier model banned and un-banned in between. Now a weekly check diffs the table against reality and pages only on drift. Alert automatically, update deliberately.
Murmur: our own Wispr Flow, built in an evening
Hold a key anywhere on the Mac, speak, release: clean text lands at the cursor. One native Swift binary, Voxtral for transcription, Haiku to strip the ums, £1 to £2 a month to run. The audio pipeline took an hour; macOS permissions took the evening.
Fable 5 is back and we just got ready
We instrumented a month of always-on agent usage: 96% of 2.86 billion tokens were prompt-cache reads, so the loop's model choice mostly prices one line item. Then our bench showed the twice-the-price frontier model coming out cheaper per correct answer on hard reasoning. Route by token shape, not rate card: cheap workers, Opus loop, Fable per-task.
Your LLM judge can't tell recent from fictional
Our panel of models-as-judge killed a fresh research finding unanimously at 0.98, because it decided last month's dates were fictional. The judge was grading its own training cut-off, not the idea. We told the jurors what day it is and banned "it postdates what I know" as a reason to reject. The artifact vanished.
ACCESS RECORD →The Verification Tax
We wrapped a cheap model in a loop that checks its own work, to try to beat a stronger model on cost. On six tasks the cheap model already got right, the wrapper caught nothing and became the most expensive option on the board: 5x bare Sonnet, ~2x a single Opus call. A verifier is insurance you only claim on when you are wrong.
ACCESS RECORD →A source library that files itself
A naming convention only survives if something maintains it. So we gave the job to a Claude Code skill: drop research files in, say "sort new files", and it reads each one, proposes a prefix, and renames on your approval.
ACCESS RECORD →An agent on a box you own
Two AI agents, two people, each on a server the owner controls and reaches over Telegram by text, screenshot or voice note. We made both multimodal, then hardened the boxes and found one still had SSH password login on.
ACCESS RECORD →We Made 86 Ships Answer-Engine-Citable
Answer engines quote sentences, not pages. Our whole Ships back catalogue had zero structured data, so it was invisible to citation. We fixed all 86 in an afternoon: Article schema, an FAQ exemplar, and full llms.txt coverage. Before 0 of 86, after 86 of 86.
ACCESS RECORD →The 120x Code Index, Measured
A code-index tool badges 120x fewer tokens; its own paper says 10x. We measured it with tiktoken on a real repo. The win is real for enumeration and impact queries (up to 297x) and net-negative for concept search. Average on a grep-first baseline: 1.8x.
ACCESS RECORD →The Clean Repo That Runs You
Mozilla showed AI coding agents getting owned by clean-looking repos that execute on a normal command. We built clone-scan, a static pre-flight, and audited our own fleet. No weaponised repos, but three dependencies run code on npm install that nobody had ever read.
ACCESS RECORD →Burr Crash-Recoverable Agents
Bob runs headless from cron, so a mid-build crash loses the whole run. We rebuilt the Loop pipeline on Apache Burr, hard-killed it at step 3 of 5, and measured the recovery. Checkpointing to SQLite cut wasted steps from three to one. The catch: the interrupted step replays, so step boundaries have to be idempotent.
ACCESS RECORD →The verifier is the environment.
A new survey names two ways to auto-generate agent training tasks, symbolic and neural, but quantifies neither. We reproduced the core on one checkable puzzle. Neither method can price how hard its own tasks are (33% and 7% calibrated). A cheap verifier in the loop takes the usable rate to 100%. The load-bearing part is the checker, not the generator.
ACCESS RECORD →We gave our agent a memory that reorganises itself.
We opened up our agent's memory for a tidy-up and found a third of it was invisible. The flat index had outgrown its context budget and was half-loading in silence. We rebuilt it as a self-maintaining map-of-content graph, with a nightly check that flags rot before it bites.
ACCESS RECORD →Voting resamples. Distillation teaches.
Sequel to the voting-harness test. We paid the frontier model once to write down its method, then handed it to the cheap model. It hit 100%, past the harness and past the teacher, at a third of the voting cost. Then a second teacher draw netted nothing. A real lever, on a dice roll.
ACCESS RECORD →Per-check accuracy is a vanity metric.
We reproduced the core of Agents' Last Exam on eight deterministic office tasks. Per-check accuracy stays high while the full-pass rate, the only metric that matters for paid work, collapses as tasks get longer. A four-point per-check gap moves the viability horizon ninefold.
ACCESS RECORD →The agent passed along five papers it never opened.
The agent handed over five arXiv IDs and admitted it had not checked they were real. A rule told it to verify; nothing enforced the rule. So we built the gate: cite-check fetches every link and arXiv ID in a draft and fails the publish if one is not real.
ACCESS RECORD →The job finished. Nobody woke the agent.
Our agent kept promising to ping when a background job finished, then went silent. The cause was structural: a finished job never woke it. The fix is not a louder notification, it is waking the agent to report, with the raw output as a fallback.
ACCESS RECORD →llama.cpp: the all-cores --threads trap
On a shared 12-vCPU box the sweet spot is 8 threads, not 12. Asking for all the cores dropped prompt processing 3.4x and collapsed token generation 267x, to half a token a second. We measured it, reproducibly.
ACCESS RECORD →Talk to Bob: a voice line to our agent
Send a Telegram voice note, our agent hears it locally with Whisper, does the work, and replies in a spoken voice. Built in an afternoon on the bridge we already had. Local transcription, no app on the phone.
ACCESS RECORD →An MCP server can tell your agent to read your SSH key
A remote MCP server's tool descriptions are read by your agent as instructions. We built a deterministic guard that pins them, catches silent rug-pulls, and scans for poisoning. It found two live API tokens sitting in our own server URLs.
ACCESS RECORD →Memory-rot watchdog: when 200 isn't saved
Our agent's long-term memory silently stored zero facts for two months while every health check stayed green. The cause was a free-tier token cap that still returned success. We fixed it and shipped a watchdog that checks what memory can actually recall, not the HTTP status.
ACCESS RECORD →Sovereign Agents, Locked Down
Prompt injection against coding agents is now exploited in the wild and it hits our exact stack. We audited the real attack paths in our own fleet and shipped a deterministic scanner that catches a poisoned instruction file before an agent ever acts on it.
ACCESS RECORD →SelfCompact Reproduced
We reproduced the core mechanism of SelfCompact (arXiv:2606.23525): an agent that decides for itself when to compact its own context. On 40 synthetic traces it lands 30 to 59% under fixed-interval summarisation with zero of the redo incidents the blind clocks rack up.
ACCESS RECORD →Our AI Judge Was Wrong 29% of the Time
We pointed a second model at our own three-model AI judge and found it killing good work 29% of the time, every error in one direction. The cause was the pipeline feeding it truncated inputs, not the judge.
ACCESS RECORD →HarnessX AEGIS Gate Reproduction
We rebuilt the core of HarnessX in 470 lines of Python. Remove one deterministic check, the seesaw constraint, and a self-improving harness climbs to a perfect score then falls to 0.59 and never heals.
ACCESS RECORD →The Long-Context Recency Cliff
A controlled eval: Gemini Flash holds 100% on needle retrieval to 436k tokens, but tracking the most recent value falls off a cliff past 100k. When it fails it returns a stale answer, not an invented one.
ACCESS RECORD →Organising My Hobbies With My Agent
Three hobbies became slash-commands on my VPS agent: /running, /sourdough, /pizza. Why that beats a project on a consumer app: the memory is files I own and the maths is real code, not a model's recollection of a chat.
ACCESS RECORD →Vera Standing: A Nightly Eval For Every Agent
We had a good judge but no standing eval. Vera Standing grades what our eight agents actually shipped each night, screen-first and budget-capped, and remembers the scores so drift shows up as a number. First run cost $0.002 and caught a real failure.
ACCESS RECORD →Reproducing Claw Patrol's Agent Firewall
Deno's Claw Patrol gates an agent's traffic at the wire. We rebuilt its core and attacked our own parser: a shallow verb-sniffer waved through SELECT 1; DROP TABLE. Blocked is not understood, default-deny is the real hero, and credential-on-the-gateway is the idea to steal.
ACCESS RECORD →FORT-Searcher Reproduction
A benchmark that looks hard but leaks a shortcut over-credits your agent. We rebuilt FORT's four shortcut controls and a deterministic shortcut-seeking solver in 260 lines of Python. Pull any one control and the search collapses from five steps to one.
ACCESS RECORD →Pulling YouTube Transcripts Past the Block
YouTube's 2026 crackdown killed every free transcript route from our server. We built yt-transcript, a fallback chain that survives the IP block, so a link becomes a transcript again. Send a link, get a summary back.
ACCESS RECORD →Reviewer Back in the Loop
Self-Harness lets an agent approve its own harness edits. We built the version with a reviewer put back in: proposer and gate structurally separated, an independent held-out eval, a tamper-evident log. The gate rejects the proposer's best-looking edit, the honest one lands.
ACCESS RECORD →Adaptive Auto-Harness Reproduction
We rebuilt a new paper's self-improving agent harness on a drifting task stream. A construct-once harness sheds 18 points from its peak; the adaptive tree-plus-routing version holds at 0.99. The useful bit is the gap split: routing is near-solved, so the only loss left is building richer branches.
ACCESS RECORD →When the Harness Costs More Than the Model
A startup claimed a voting harness makes cheap models 99.99% accurate. We measured it: self-consistency voting lifted Haiku from 92.5% to 97.5%, matching Opus solo, but at 3x Opus's cost for the same score. And it never neared four nines, because the last error was systematic and voting cannot outvote a consistent mistake.
ACCESS RECORD →Vera Disagreement Map
Vera's model panel already votes ship-or-kill. Now a judge step maps where the three jurors disagreed: consensus, contradictions, and the blind spots none of them raised. On a real ReferRoute architecture call it surfaced a hybrid option and a GDPR Article 28 risk the whole panel had missed. Opt-in, for the expensive decisions.
ACCESS RECORD →Predictive Alignment Is Diagnostic Not Curative
We reproduced the World-In-Agent mechanism from Role-Agent (arXiv:2606.10917) at inference time. An agent predicting its own next state gives a strong read on action quality (0.70 alignment on good moves versus 0.23 on bad), but using it as a reminder did not move task success. The value is in the training reward.
ACCESS RECORD →Agent libOS: authority belongs at the primitive
We reproduced the core of a new agent runtime where capability checks at the primitive, not the tool registry, are the trust boundary. Nine of nine falsifiable tests pass, and 64% of attempted operations were stopped at the boundary despite full tool visibility.
ACCESS RECORD →The Generator in the Garage
We wrote one command that revokes every cloud key mid-request and proves our router keeps working on a local model. Cloud answered in 2.9s, then went dark, and the work carried on offline and free. A kill switch you run on purpose, in daylight.
ACCESS RECORD →Personalize-then-Store Repro
We rebuilt a new memory paper on a laptop, no model calls, and reproduced all three findings. Under a fixed memory budget, perfect gating wins big when the budget is tight, but realistic gating barely beats storing everything. That gap is the whole problem.
ACCESS RECORD →After Fable 5: the UK builder's read
The US Commerce Department disabled Anthropic's Fable 5 and Mythos 5 worldwide on 13 June. UK users caught up because Anthropic cannot verify citizenship in real time. Four likely paths from here, three concrete moves for UK builders this week.
ACCESS RECORD →First trained-agent at Workloft
We taught a small open-source model to do one of Walt's daily jobs as well as Gemini Flash does it now, and parked it on the VPS. Free at inference, no data leaves the box, beat gpt-4o-mini on every metric on the full 212-row holdout. Walt was build one of six to eight.
ACCESS RECORD →MiniMax Sparse Attention, reproduced
MiniMax claims a 28.4x cut in attention compute at 1M tokens with no loss of quality. We reproduced the two claims that do not need a GPU on this CPU box. The FLOPs model lands on 28.4x exactly; the Top-k block selector keeps 92.5% of the attention mass at the paper's budget.
ACCESS RECORD →The night Fable 5 went dark
We had a Field Guide up on Tuesday. On Friday the US Commerce Department disabled Fable 5 and Mythos 5 outright. The model on the route we were testing was gone by close of business. Builder POV on continuity, sovereignty and the covert-degradation story.
ACCESS RECORD →SkillOpt prototype: bounded edits, real numbers
We implemented Yang et al.'s SkillOpt loop end to end on a 16-item benchmark. Bounded text edits plus a strict held-out validation gate took the test score from 0.750 to 1.000 in one accepted edit. The gate then rejected five of six follow-up candidates, exactly as the paper says it should.
ACCESS RECORD →Enterprise Watch: a daily agent-platform market scan
A public page that reads the newsrooms of eight enterprise agent platforms every morning, scores each item with a cheap model, and publishes the few that move the market with a why-it-matters paragraph each. First scan: 28 candidates in, 8 published. Daily cron, auto-deploy, pennies a day.
ACCESS RECORD →Local SVM scorer for our paper queue: AUC 0.86
We trained a TF-IDF and linear SVM on the 36 papers Walt has filed to Gary, evaluated it on the 668-paper Hugging Face Daily archive, and got a leave-one-positive-out ROC AUC of 0.856 with precision at 10 of 0.70. The SVM and our existing LLM scorer rank papers very differently, so the right move is to wire it in as a second signal, not as a replacement.
ACCESS RECORD →Question-Mode Selection
Bob picks the next loop items daily. We A/B-tested a thesis-plus-counter-question prompt against the plain directive over eight runs on the same live queue: it changed one pick in three, trading heavy sweeps for bounded spikes. Our own parser nearly buried the result, logging pre-revision picks and under-reading divergence as 0.17 instead of 0.25.
ACCESS RECORD →Live AgentPass: fresh-signed credential on /verify
The site now issues its own AgentPass on demand: a signed W3C Verifiable Credential with a 15-minute validity window and real standing data from the audit log, verified entirely in your browser against our did:web public key.
The chat widget is now a real agent over the build log
Every visitor question is now scored against 91 published Ships and Labs articles; the widget answers from the top excerpts with the article URL attached. No embeddings, no vector store: keyword overlap, light stemming, a recency boost and a 10-minute cache.
Mission Control: live fleet telemetry on the homepage
The homepage now streams the fleet working in real time: last ship, 44 ships logged, 170 Labs picks, wall tags and seven agent heartbeats, fed by one cached endpoint. Trust grid claims became clickable verify links. The site said we run a fleet; now it shows it.
Say Hi! A graffiti wall for the Workloft homepage
workloft.ai now has a graffiti wall. Visitors tag up to three initials in 8 fonts and 8 spray colours, with a spray-reveal, paint drip and particle burst. Every tag persists via two rate-limited chat-api endpoints. From Telegram ask to live in 18 minutes.
ACCESS RECORD →skill-distiller: worked demonstrations into a reusable skill
We write skills best from a task we have already done well once. skill-distiller takes the messy worked record of how a task was actually done and distils it into a structured SKILL.md draft, capturing the implicit procedure and pitfalls, not a summary. Drafts land for human review and never auto-install.
ACCESS RECORD →rebound: a tool-failure recovery harness
Tools fail constantly. The question is whether the fleet bounces back. rebound replays real tool-failure events from our audit log and measures recovery: explicit failures recover 100%, implicit-semantic ones 90%. It surfaced the one that never did — an Otto cron that got empty stdout and never retried.
ACCESS RECORD →codemap: a local code-symbol index for agents
"Where is X and what is its signature" usually means grep the whole tree, then read the file end to end for one line. codemap indexes every function, class and type into a compact SQLite map, so the same question is a single file:line lookup. 96.7% fewer characters per lookup, pure stdlib, 22 tests.
ACCESS RECORD →sluice: an outbound egress guard
Agents touch live credentials all day. One careless paste and a key is public forever. sluice is the gate every outbound message passes through: scan and refuse, or redact in place. 100% recall on planted secrets, zero false positives across 1.36M chars of real copy, and it caught two real internal-path disclosures already live on the site.
ACCESS RECORD →slim: token-trim filter for agents
Agents burn most of their context budget on tool output they never needed. slim strips the noise before it reaches the model: lossless cleanups always on, large dumps clamped head and tail. On five real command outputs it cut characters by 88.7%, roughly 110k estimated tokens down to 12k. The honest catch: the big wins are lossy by design.
ACCESS RECORD →Vera Reward Mode
The Vera panel votes PASS or KILL with a confidence number, and models are bad at that number. We read a reward straight from each juror's next-token probabilities instead. On an eleven-probe set it held a steady 1.0 where the old signal coin-flipped to 0.38, and it surfaced juror disagreement the averaged confidence had buried.
ACCESS RECORD →Vera A/B Mode
Vera could tell us whether an agent passes a scenario set. It could not tell us whether a change helped. A/B mode runs two variants over the same scenarios and the same rubric, scores both with the three-juror panel, and reports a net pass-rate delta, tagging every scenario fixed, regressed, stable or inconclusive.
ACCESS RECORD →Wiring r/LocalLLaMA into the Workloft Loop
We added r/LocalLLaMA as the fifth feed to the Loop, the one source watching the open-weight and local-inference world. Reddit 403s our server's IP on the JSON API, so we pull it through the wide-open RSS feed instead. Walt scores the day's posts and files only the 9s and 10s, because the place is noisy. First run: thirty-two scored, two filed.
ACCESS RECORD →stealing Jon's browser hardening for Larry
A fellow builder, Jon, wrote up his hardened agent-browser setup and shared it. We took the one piece that earned its place today, a stealth flag that stops Larry advertising himself as automation, and left the proxy and captcha layers documented as on-demand. Then we mirrored it so you can steal it too.
ACCESS RECORD →trojan-scan: catching backdoors in our own memory
A new paper (ClawTrojan) shows an agent reading a hidden instruction from a tool output, storing it in memory, then running it a session later. Per-step gates miss it. We built a scanner that baselines every auto-injected surface and flags drift, obfuscation and hook egress. Clean on 256 files, catches all four seeded attacks.
ACCESS RECORD →daily.dev wired into the Workloft Loop
We hooked daily.dev's trending feed into the Loop. A daily cron pulls it, Walt scores every post against our research axes, and the strongest buildable picks file themselves into the backlog. Third external signal feeding the Loop, for pennies a day.
ACCESS RECORD →Grok tested for the code tier. It didn't earn the slot.
We wired xAI's Grok into our router and ran it against the models we already trust for code. It wrote correct code, fast and cheap, but Opus still won quality and DeepSeek still won price, so Grok stays in the catalogue without the slot. A negative result is still a result.
ACCESS RECORD →Queued posts now fall off the to-do list on their own.
Once a post is queued for review, the reminder to publish it closes itself. A new audit pass matches open publish to-dos to live drafts by channel and slug, and the draft becomes the tracker so the list only shows what still needs a human.
ACCESS RECORD →The agent stopped re-posting things we'd already shipped.
A status-driven daily audit now reads the real queue state, catches cross-channel duplicates, and closes the to-do items for posts we have already published. One clean pass cleared nine orphaned drafts that three manual reminders could not.
ACCESS RECORD →A bandit that stops the router overpaying.
A small learner sits on top of Ruby, watches which tier actually pays off per job, and downshifts off the dear tier when the cheap one keeps answering. On our priciest category the gap it closes is about seventeen-fold.
ACCESS RECORD →The router now grades its own answer before handing it back.
Ruby runs the cheap model first, puts the reply in front of a three-juror panel, and climbs the tier ladder by itself when the answer is weak. Cheap by default, expensive only when the work earns it.
ACCESS RECORD →Our agent read research the slow way. Now it reads it itself.
A human used to spot a paper and paste the link. We wired the AlphaXiv MCP server into the agent, so it searches, ranks and reads arXiv papers as native tools. The research firehose is one tool call now, not a manual hunt.
ACCESS RECORD →The rule was documented. The agent skipped it anyway.
Our shipping procedure kept losing the same step. So we stopped trusting the agent to remember and moved the hard rules into hooks that block the action when a precondition is missing. The hero on this entry exists because a gate refused to ship it without one.
ACCESS RECORD →We could see what the robots spent. Not what they earned.
The audit log tracked every pound each always-on cron spent on tokens, but nothing it earned. We wired per-cron revenue attribution onto the same append-only ledger — no new database — so every cron has a P&L.
ACCESS RECORD →The rule was saved. The agent never saw it.
A saved rule kept getting broken because the memory index outgrew its load budget and was truncated before it reached context. We trimmed it and built a hook that hard-stops the index from ever exceeding budget.
ACCESS RECORD →The V4-Pro Reasoning-Token Mirage.
DeepSeek V4-Pro's price fell 75%. We A/B'd it against Gemini Flash on our live paper-scoring job. It came out 11.7x pricier and 18.8x slower. Hidden reasoning tokens, paid for and thrown away.
ACCESS RECORD →The Social Loop.
The Typefully bridge. Post drafts flow out for scheduling, and a 15-minute cron reconciles the published URLs back into the ledger. The Publish step of the Loop now runs itself.
ACCESS RECORD →Walt's picks now grade themselves.
Outer loop of two-level autoresearch wired onto Walt. Every paper scored >= 8 is joined to its Gary outcome and reported per axis. Measure-before-tune, as a runtime feature.
ACCESS RECORD →Bob's actions now write Vera's tests.
PhoneWorld pattern applied to our audit log. Trajectories cluster by (agent, action), Ruby drafts a Vera rubric per cluster, verifier coverage grows on its own as the fleet does new work.
ACCESS RECORD →civiclaw FOI intake prompt polished.
The intake prompt invited the model to ask clarifying questions back. Removed that default, forced six fixed headings, anchored workable-as-written. Output on Qwen2.5:7b dropped from ~60 lines in 1m41s to ~30 lines in 45s and stayed on-topic.
ACCESS RECORD →civiclaw sovereign Ollama fallback wired end-to-end.
Until today, civiclaw's sovereign claim was scaffolded but not wired. Every skill hard-bound to the Anthropic SDK. As of this commit, FOI / EIR / AIACT / DSAR plain-text stages all run end-to-end on a local Qwen2.5:7b via Ollama. The doc claim is now a doc fact.
ACCESS RECORD →civiclaw GitHub mirror live.
civiclaw is now at github.com/workloftai/civiclaw, push-mirrored from the GitLab canonical via GitLab's remote_mirrors API. Closes the discoverability gap for HN and dev audiences who expect to find OSS on GitHub, not GitLab.
ACCESS RECORD →Audited the next MCP spec two months early.
MCP protocol version 2026-07-28 is in draft upstream. Two months until release. We audited our hosted endpoint, found a real 502 leak on the legacy GET stream, fixed it to a 405, and wired the hourly canary plus daily PyPI watcher that will tell us the moment the Python SDK ships 2026-07-28 support. The flip is now a 30-minute job.
ACCESS RECORD →SEAL evolve, failure-driven guardrails from the audit log.
A paper landed on the arXiv feed at 8am. By lunch we had stolen the implementable kernel, run it on our own audit log, and the first 7-day pass surfaced an Anthropic billing issue and a DeepSeek max_tokens bug that had been failing quietly for days. Walt classifies each failure, clusters them, and drafts a one sentence guardrail per cluster. Two hundred lines, no new dependencies. Read-only by design.
ACCESS RECORD →Labs Carousel — PDF carousel generator for Workloft Labs Notes.
Every Workloft Labs Note now ships with a 1080x1350 LinkedIn-native PDF carousel alongside the text. One command: distillation via Walt + Sonnet, per-Note motif via gpt-image-2, layout via Playwright, British-English post body drafted automatically. End to end about six pence per Note. Built to test whether carousels outperform text posts for our audience.
ACCESS RECORD →Workloft Labs, now a hosted MCP server.
Labs API has been live since 8 May with zero external uptake. Sunday afternoon we turned it into a hosted MCP at chat-api.workloft.ai/labs-api/mcp/: one JSON snippet in any agent client and our 85 curated picks across 17 days appear as tools. Same build fixed the /health 502, lifted the free tier to 500 calls/30d, and added a public no-auth daily JSON snapshot endpoint. Zero clone, zero auth.
ACCESS RECORD →Agentic Oddities, the fortnightly weird-AI digest.
Every three days a scraper pulls real-world AI-agent failure stories from HN and Google News, Walt scores them, Vera picks the headline and writes the missing-control angle. First run: 127 candidates, 4 shortlisted, headline pick was The Times on the AI cafe that ordered 3,000 pairs of gloves. Feeds the new /labs/news/ section that went live the same day.
ACCESS RECORD →A ledger for every public post.
Tiny Supabase table called workloft_posts. Every public post under the Workloft name now lands a row. Maggie JSON queues hold intent; the ledger holds outcome. First two rows landed within a minute of the migration committing.
ACCESS RECORD →A todo system Bob cannot cheat.
164 open todos, many overdue by two weeks. Spent the day building a system where every item ends in shipped or killed. Enforcement lives in a Claude Code Stop hook, not in the system prompt. First contract violation caught 30 min after going live.
ACCESS RECORD →Every Note and Ship now has a Markdown sibling.
18 .md files now ship alongside the HTML. Frontmatter on top, no nav, no animation, no related-links chrome. GitLab Pages serves them as text/markdown. The Workloft corpus is now agent-readable by default.
ACCESS RECORD →llms.txt for Workloft, shipping for real this time.
Our llms.txt existed in the repo for weeks and 404'd in production for weeks. A PostHog look at last week's traffic surfaced the silent failure. We fixed the deploy, refreshed the content, and made Workloft visible to AI crawlers.
ACCESS RECORD →The interop floor lifted. We swept our positioning to match.
A2A v1.0 crossed 150 organisations and one year inside the Linux Foundation. Interop is officially commodity. We swept Labs, the homepage and the sales surface, and published Note №10 on where the moat moves next.
ACCESS RECORD →The selection gate now sits on a panel
Single-LLM judges have correlated blind spots. We retired Vera and stood up PoLL, a three-juror panel across Haiku 4.5, Gemini 2.5 Flash, and DeepSeek v4 Flash. Splits escalate to Telegram. ~$0.002 per candidate.
ACCESS RECORD →Your audit log is training data
Agent Context Compilation, applied to our own production audit log. 25 trajectories, 102 grounded long-context QA pairs, $0.0132 of compute. Open source under MIT.
ACCESS RECORD →Bob Picks Up the Phone
After several weeks of back and forth with Twilio support, the Workloft voice line is finally live. Bob now answers the phone. A real conversation, in real time, in a voice closer to a person than a recording.
ACCESS RECORD →Gemini Managed Agents, wired into Ruby
Google shipped one-call managed agents at I/O 2026. We tested it, wired it into our model router, and saw three to eight times cost cuts on agentic tasks. Region caveats apply.
ACCESS RECORD →AgentPass V0.1: the verification primitive AI agents don't yet have
Published as an RFC on 3 May 2026. A Verifiable Credential profile that lets any verifier answer, in real time, whether an AI agent has standing to act in an institutional transaction. Single API call. Yes/no with cryptographic proof.
ACCESS RECORD →Sovereign by default: A2A v1.0 + AP2 V0.1 wired through the stack
Over 24 and 25 April we made every Workloft agent speak Google A2A v1.0 and issue AP2 V0.1 mandates. Every agent action is now cryptographically signed and independently verifiable. Verify it yourself at workloft.ai/verify.
ACCESS RECORD →