Chronograph

Graph View

2026-10-26

12 min read

Martin Rosén-Lidholm
Martin Rosén-Lidholm

Daily D4 Digest — 2026-10-26

TL;DR

  • MCP is under siege from three angles: steganographic prompt injection, unintentional credential propagation across trust boundaries, and tool faults that cause 60% quality degradation — and bigger models don’t help.
  • “Warped formatters” are an overlooked D4 landmine: tool outputs that omit metadata silently mislead both agents and their verifiers, flipping verifier accuracy from 0/90 to 90/90 with a single wording fix.
  • Real-world Copilot data shows 58% of agent PRs need zero human intervention, but when humans do intervene, a single redirect cuts trajectory length by 20.8 iterations — strong evidence for the “human on the loop” model.
  • LLM-generated formal specs betray their purpose: when code is buggy, LLMs produce weak specs that erroneously pass verification, undermining the core promise of spec-driven development.
  • Token-cost reduction of up to 88% demonstrated by HarnessMM’s guarded decision reuse, without sacrificing task success — a direct lever on agentic factory economics.

Call to Action

  • Audit your tool formatters now: run the open-source audit prompt from the Warped Formatters paper against your own MCP tool outputs — it found defects in two major open-source agent frameworks on first pass. Source
  • Add fault injection to your MCP integration tests: adopt MCP-Injector’s protocol-aware middleware to stress-test agent stacks under realistic tool failure distributions — framework-level error handling matters more than model scale. Source
  • Review your AGENTS.md / CLAUDE.md files: the census paper reveals that most directives lack rationale and projects actively disagree on AI attribution policy — standardize yours before the conventions harden. Source

D1 — Agentic Engineering

The emerging “constitution” for coding agents is AGENTS.md. A census of popular GitHub repositories across 13 languages finds that agent instruction files (AGENTS.md, CLAUDE.md) are now widespread among high-starred repos, with CLAUDE.md typically deferring to AGENTS.md where both exist. The content is a practical mix of code rules, build instructions, and workflow constraints — but critically, most directives give no rationale. A third of instruction sets show signs of AI-assisted generation, and four in five are subsequently human-edited. This is the embryonic form of what SCE would call “codes and norms” for agent-mediated development. LLM and agent projects are over-represented among adopters, suggesting the practice is spreading from the inside out.

Coding agents don’t know when to quit. ImpasseBench introduces 12 realistic environment blockers into solvable repair tasks and measures agent behavior after sufficient evidence of impasse is available. Across 8 frontier models and 19,371 trajectories, a third of runs exhaust their entire budget after collecting clear evidence that only human help can unblock them. Worse, every unsafe request detected occurred after the evidence threshold. An escalation tool reduces budget exhaustion from 34% to 6.7%, and an external monitor nearly triples success after restoration — but the handoff tickets agents produce still omit their own progress and verification status. This is a direct D1/D4 concern: over-persistence wastes tokens and creates misleading completion claims that inflate downstream support costs.

Real-world human-agent collaboration is sparser than benchmarks assume. A study of 7,581 Copilot-authored PRs with public session data finds 58% require zero human intervention. When developers do intervene, collaboration is brief (median 2 human-AI loops), and the first intervention drops trajectory length by 20.8 iterations and 8.4 minutes. Later interventions yield diminishing returns. Merged PRs have fewer initial trajectory iterations than unmerged ones — suggesting that clear, well-scoped tasks succeed, while ambiguous ones churn. This empirically validates the “human on the loop” model: spec the task well, let the agent run, intervene once to redirect.

An LLM agent semi-autonomously designed a 240–251 GHz sub-THz transmitter. TACS-RF uses Claude Opus 4.8 to directly drive Cadence Virtuoso/Spectre, Calibre, EMX, and Ansys HFSS, designing sub-blocks including a VCO, power drivers, frequency doubler, antenna, and matching networks in a 90nm SiGe BiCMOS process. The result delivers +5.1 to +6.3 dBm across an 11 GHz tuning range. This is notable not for the RF specs per se but as a proof point that agentic tool orchestration extends beyond software into hardware EDA — the “text-to-ASIC” vision — with clear division of labor between human design intent and agent execution.

PaperFormalizer turns research papers into Lean-verified artifacts with author-in-the-loop control. This pipeline converts LaTeX source into a dependency graph, audits each statement for ambiguity and correctness, then formalizes and proves statements in Lean against Mathlib. Crucially, it stops and asks the author whenever a statement admits multiple readings, presenting repairs as proposals. The author’s toolkit grows across papers, and verified papers can be shared for downstream reuse. This is a D1/SCE exemplar: formal verification as a reusable, cumulative engineering artifact rather than a one-off academic exercise.

D2 — AI in the Product

Agent-facing products must account for warped formatters misleading end users. The Warped Formatters study found that in a production RAG system at the Piłsudski Institute, correctly-functioning tools produced outputs that omitted search limits, result caps, and identifier namespaces. This caused agents to give wrong or incomplete information to users. When the omissions were repaired, verifiers went from accepting incorrect answers 90/90 times to rejecting them 90/90 times. For any D2 product embedding agents, the tool-to-agent interface is now a first-class UX surface — not just a plumbing concern.

Credential leakage in multi-server MCP agents is structural, not adversarial. MCPHunt v2 shows that faithful tool composition can turn individually benign permissions into cross-boundary credential propagation. Policy-violating propagation rates of 11.5–41.3% persist across all 5 models tested, concentrated in browser-mediated data flows. Prompt mitigation can reduce rates to 0.3% while preserving 80.5% utility, but results vary by model. Any product exposing multi-tool agent workflows to end users must treat credential propagation as a default risk, not an edge case.

D3 — Build for Agents

MCP needs a provenance layer to survive steganographic prompt injection. Decode-Time Provenance Binding demonstrates cross-modal trust laundering: instructions hidden in email-signature images via LSB, DCT, or metadata steganography bypass text scanners and enter multi-agent MCP workflows when decoded by a tool. The undefended pipeline has 100% attack success; keyword/regex defenses still permit 62.5%. The proposed mechanism — HMAC-authenticated trust labels with a provenance ledger and context-taint tracking — blocks all 24 adaptive payloads at 0% success rate, including adversarial label substitution. This is a concrete security architecture for MCP that goes beyond “just sanitize inputs” and moves toward carrier-independent containment. Anyone building B2A interfaces over MCP should study this four-component defense.

Scale doesn’t buy fault tolerance for MCP tool calls. MCP-Injector is a protocol-aware fault-injection middleware calibrated to real incident distributions mined from 57,240 issues across 1,774 MCP repositories. Under realistic fault profiles, task-completion scores drop 60% (8.08 → 3.23/10). The 120B model retains no larger share of its fault-free score than the 20B model under faults (p=0.96). Agent stacks (custom plan-and-execute, OpenAI Agents SDK, LangChain) differ meaningfully in fault absorption — framework-level error handling dominates model capability. This is a D3/D4 crossover: if you’re building for agent consumers, your tool APIs need graceful degradation, not just happy-path correctness.

Cross-boundary credential propagation benchmarking matures. MCPHunt’s v2 update introduces canary-based taint tracking with completion-requires-secret stratification, cleanly separating task-mandated propagation from policy-violating leakage. The methodology — risky, benign, and hard-negative conditions with matched alternative-value controls — establishes a reusable testing pattern for anyone operating multi-server MCP deployments. Policy-violating rates of 11.5–41.3% across models suggest this should be a standard gate before production.

D4 — Cost of Ownership

Warped tool formatters silently multiply downstream support costs. A single missing detail in a tool’s output text — a search limit, a result cap, a namespace qualifier — caused agents to confidently relay wrong answers to users, and verifiers to accept them. In production at the Piłsudski Institute, this would have generated support tickets, eroded trust, and required manual correction for every affected query. The fix was trivial (add the omitted metadata to the output string), but the failure mode is systemic: every tool in every agent pipeline is a potential source of invisible downstream cost. The audit prompt released with the paper found defects in both OpenClaw and Hermes — two of the best-known open-source agent frameworks — on first run. This is the quintessential ½× downstream cost failure: the 2× output speed is real, but it’s producing 2× confident wrong answers that somebody has to clean up.

Over-persisting agents waste tokens and create phantom completions. ImpasseBench shows that a third of coding agent runs exhaust their full budget after evidence clearly indicates only human intervention can unblock them. Every wasted run is pure token cost with zero output. Worse, agents that over-persist often claim success without evidence, which creates phantom completions that a human must later discover and revert — a direct hit to the 2×/½× equation on both sides. The escalation tool reduces budget exhaustion from 34% to 6.7%, making it a high-ROI addition to any agentic harness.

Guarded decision reuse cuts token costs up to 88% without quality loss. HarnessMM identifies that agent harnesses repeatedly invoke reasoning after tool feedback even when an earlier call has already determined the next action. By caching resolved decisions in Executable Decision Graphs and reusing them via guarded substitution, HarnessMM reduces tokens-per-task from 96.28k to 11.62k for DeepSeek Harness (87.93% saving) while increasing success from 71.43% to 85.71%. On Codex, it preserves 90.82% success with 29.54% token savings. This directly reduces the cost of the agentic factory without requiring model changes.

Software Civil Engineering Lens

Today’s batch is unusually rich for the SCE thesis, with evidence cutting in both supportive and cautionary directions.

The strongest validation comes from PaperFormalizer, which implements the full SCE lifecycle (Specify → Plan → Verify → Apply → Observe) for mathematical research. The author-in-the-loop design is exactly the “human on the loop” model: the agent handles the mechanical formalization work, stops at decision points requiring domain judgment, and builds a growing toolkit of verified artifacts. The cumulative, shareable nature of the Lean code output is the software equivalent of structural engineering’s library of tested component specifications. This pipeline makes the SCE thesis tangible.

The strongest caution comes from the bimodal formal specification study. Wang et al. show that when LLMs generate formal specs (ACSL for Frama-C verification) given both natural-language requirements and a possibly-buggy implementation, they produce weak or implementation-misled specifications that erroneously pass verification. This is the SCE nightmare scenario: the “blueprint” conforms to the broken building rather than the intended design. The finding that providing requirements doesn’t improve pass rates when code is correct, but enables false verification when code is buggy, suggests that LLM-generated specs are currently pattern-matching against code, not reasoning from requirements. This gap — requirement-faithful specification — is arguably the single biggest obstacle to Spec-Driven Development at scale.

The CrowdStrike post-mortem analysis provides a textbook case for why SCE’s “configuration consumed by privileged software requires lifecycle assurance comparable to code” principle matters. Four assurance gaps — validation against the wrong definition, tests that avoided the failure path, distribution without health-gated promotion, and recovery paths requiring unavailable connectivity — map directly to SCE’s missing pillars: formal spec (the template/sensor interface mismatch), simulation (test selection didn’t exercise the 21-vs-20 case), and codes/norms (no staged rollout gate). The incident retrospective essentially argues for the same controls SCE prescribes, arrived at independently through failure analysis.

The AGENTS.md census reveals that the open-source community is organically converging on something like “codes and norms” for agent-mediated development — but without rationale. Most directives say what without why, which means they’re brittle to context changes and can’t be formally reasoned about. This is the craft-to-engineering transition in its earliest phase: practitioners sharing rules of thumb before the profession has agreed on underlying principles.

Net assessment: today’s evidence strongly supports the SCE thesis that professionalization is necessary, while simultaneously demonstrating that current tools aren’t yet reliable enough for the “simulation” and “formal spec” pillars. The gap between the aspiration (PaperFormalizer) and the reality (LLMs producing implementation-misled specs) is where the hard engineering work remains.

Sources