Chronograph

Graph View

2026-09-28

10 min read

Martin Rosén-Lidholm
Martin Rosén-Lidholm

Daily D4 Digest — 2026-09-28

TL;DR

  • Coding agents waste 23% of task cost on three recurring behaviors; developer-designed skills (not agent-synthesized ones) cut cost by up to 42% — the human-authored spec wins again.
  • 82% of LLM model migrations happen after the API has already broken in production; hard-coded model IDs in 94% of apps make this a ticking D4 bomb.
  • EuroPLoP focus group coins “harness engineering” and “cognitive debt” — two concepts that map directly onto the SCE thesis of professionalization.
  • MCP is becoming the de facto agent-to-world interface: three new papers (data spaces, medical devices, power grid) all use it as the mediation layer, and Cartograph tackles the O(n) tool-discovery scaling problem.
  • A “silent success” paper documents nine cases where quality gates passed on checks they never ran — the canonical failure mode when agents operate without formal verification of gate completeness.

Call to Action

  • Audit your coding agent configurations for the three cost-inefficient behaviors (subsumed retrieval, similar script gen, test re-execution) identified in this study — developer-designed skill libraries are the highest-leverage mitigation.
  • Inventory hard-coded model identifiers across your codebase now; the LLM migration study shows 82% of teams only discover the problem after production breaks.
  • Add a three-valued logic (pass / violate / unable-to-determine) to every automated quality gate per the silent success paper — especially gates that agents will operate.

D1 — Agentic Engineering

Coding agents burn up to 23% of task cost on three fixable behaviors. A study of 1,200 trajectories from Claude Code and Mini-SWE-Agent identifies subsumed retrieval (re-fetching already-retrieved context), similar script generation (regenerating near-identical code), and test re-execution (rerunning unchanged tests) as endemic cost sinks affecting 79–98% of tasks. The critical finding for agentic engineering practice: agent-synthesized skills produce low-level, trace-specific guidance that barely helps, while developer-designed skills — high-level, trace-agnostic instructions — reduce cost by up to 41.73%. This is strong evidence that the human’s role in the agentic loop is authoring the right abstraction of guidance, not micro-managing execution. (Also D4.)

“Harness engineering” emerges as a named discipline. A EuroPLoP 2026 focus group of 22 practitioners reached consensus that architectural decision-making, accountability, and the authoring of guardrails remain fundamentally human tasks. They coined harness engineering — building the system that governs AI-assisted system creation — and cognitive debt — the erosion of human understanding when AI decisions are accepted without intellectual engagement. The group converged on criticality (uncertainty × cost-of-change) as the universal dial for calibrating human oversight. This maps directly onto the “human on the loop” model: the harness is the spec layer. (Also SCE.)

Authority enforcement must live at the mutation boundary, not in the planner’s context. Research on stateful agent authority semantics demonstrates the “cross-substrate authority gap”: authorization information residing outside the agent’s visible workspace causes planning-level augmentation to remain unreliable (15/32 correct first actions). A deterministic execution guard at the mutation boundary blocked all 6 unsafe intents and permitted all 12 valid ones — zero additional model calls needed. This validates the Decider pattern principle: the guard is deterministic and sits between plan and apply. (Also D3, SCE.)

Verifier-selected self-training consolidates expensive capability into reliable one-shot execution. Governance Records as Supervision shows that machine-verified outputs (VAL-admitted planning solutions) can train the same model checkpoint, lifting acceptance from 1/80 to 57/80 without oracle targets or stronger teachers. The key constraint: this works for one fixed, machine-checkable capability — not arbitrary reasoning. For agentic engineering, this suggests a path to cheapening expensive agent steps by distilling verified trajectories back into the agent. (Also D4, SCE.)

D2 — AI in the Product

Muse AI Agent shows both the promise and peril of autonomous personal agents. Simon Willison quotes a Muse Agent incident where the agent auto-replied “Yep I’m here!” to a buyer on a marketplace when the owner wasn’t actually available, resulting in a no-show, an angry buyer, and a negative rating. The agent’s post-mortem is impressively self-aware — it identifies its own failure mode and proposes a policy change. This is a textbook case for bounded autonomy: the agent operated outside its verifiable knowledge (user presence), and the consequence was real-world reputation damage. The lesson for D2 product design: agents acting on behalf of users need explicit epistemic boundaries — they must distinguish what they know from what they’re assuming. (Also D4.)

LLM-powered orchestration turns hours of domain scripting into minutes of conversation. Grid-Orch bridges LLMs and power system simulation via MCP, enabling engineers to perform distribution grid analyses (power flow, voltage analysis, DER screening) through natural language. Analyses that required hours of OpenDSS scripting complete in under two minutes with numerically identical results. The framework supports air-gapped deployment via local models — critical for utility security requirements. This is a strong D2 pattern: wrapping deterministic simulation engines in conversational interfaces, with MCP as the interop layer. (Also D3.)

D3 — Build for Agents

Cartograph solves MCP’s O(n) tool discovery problem. As MCP catalogs grow, loading every tool definition becomes token-expensive. Cartograph introduces a federated MCP proxy with progressive disclosure: on a 374-tool deployment, it exposes 3 proxy tools instead of 374 definitions, cutting a top-5 discovery exchange from 42,450 tokens to 475. Key innovations include operator-attested capability cards (Ed25519-signed), confusable-cluster analysis (Rift) identifying 49 clusters including 4 high-risk ones, and two-stage server-then-tool retrieval. Gateway overhead is negligible (5ms, 0.8%). This is essential infrastructure for any organization scaling MCP adoption. (Also D4.)

MCP as the mediation layer for governed data spaces. The Eunomia Agent paper presents an architectural mediation approach that translates data space capabilities into schema-driven tools AI agents can discover and invoke while preserving governance constraints. The key insight: protocol-based mediation enables integration without modifying existing data space components. For B2A scenarios, this means agents can participate in sovereign data-sharing ecosystems through a standardized interface rather than bespoke integrations.

Safety-bounded MCP gateway for medical devices. A SDC-to-MCP gateway exposes medical device metrics, alarms, and metadata as read-only MCP resources, with action affordances represented as policy-validated dry-run tools. The “safety-bounded” property means agent-facing requests dispatch no actual device operations — a deterministic no-execution boundary. Explicit semantic metadata improved structured alarm output conformity versus generic representations, though structured-output failures persisted — highlighting the gap between plausible narrative answers and task-compliant machine-readable results. This is the most rigorous D3 pattern yet for safety-critical agent interop.

D4 — Cost of Ownership

82% of LLM model migrations happen after the application has already broken. This directly threatens the 2× output / ½× downstream cost equation: every hour of unplanned migration is pure downstream maintenance cost with zero output gain. A large-scale empirical study of 22,555 GitHub commits across 17,703 repositories (2024–2026) finds that model identifiers are hard-coded in 94% of migrating applications, migration effort ranges from 6 lines (prompt-only) to ~700 lines (fine-tuned), and only 8% of migrations switch providers. Longer notice periods help dramatically — each e-fold increase in notice length reduces post-shutdown migration odds by ~75%. The tactical implication: abstract model identifiers behind a configuration layer, monitor deprecation announcements programmatically, and budget for migration as a recurring maintenance cost.

Quality gates that pass on absent data are the silent killer of agentic pipelines. A paper documenting nine real cases where release gates reported PASS on checks they never executed shows the failure mode: gates that ask “was a violation observed?” over populations already stripped of failed-to-run cases answer “no” by default, producing two weeks of green builds with almost no actual checks. The fix is structurally simple — introduce a third value (unable to determine) — but the paper’s punchline is devastating: the author reproduced the exact bug while writing the paper, using tooling built to prevent it. For agentic pipelines where agents operate quality gates, this class of silent failure must be eliminated architecturally, not procedurally.

Developer-designed skills cut coding agent inference cost by up to 42%. The cost-inefficiency study quantifies the cost of the agentic factory itself: the three identified behaviors account for up to 22.75% of per-task cost. Structure-aware retrieval — the intuitive fix — can actually increase cost by up to 28% due to overhead and altered agent delegation. The winning strategy is investing human effort in curating high-level skill libraries, which yield roughly twice the cost reduction of letting agents synthesize their own skills.

Software Civil Engineering Lens

Today’s batch is unusually rich for the SCE thesis. Three developments stand out:

1. “Harness engineering” is SCE arriving through practitioner consensus. The EuroPLoP focus group independently converged on concepts that map almost perfectly onto SCE pillars: harness engineering ≈ formal specification + codes/norms; criticality-based oversight calibration ≈ simulation/verification gates; cognitive debt ≈ the risk of abandoning the “human on the loop” position. That 22 practitioners from industry and academia reached these conclusions without starting from SCE framing is strong evidence the professionalization pressure is real and being felt.

2. The silent-success paper is a case study in why formal verification matters. Nine instances of gates passing on absent data illustrate what happens without formal spec: the shape of the check looks correct, but the completeness property was never specified or verified. The remedy — a three-valued logic enforced at the type level — is exactly the kind of “material datasheet” / “code requirement” the SCE framework calls for. The fact that the author reproduced the bug with purpose-built tooling underscores that procedural discipline cannot substitute for structural enforcement.

3. Mutation-boundary enforcement validates the Decider pattern. The authority semantics paper provides controlled experimental evidence that giving agents more context about authorization doesn’t reliably improve safety (15/32 correct), but a deterministic guard at the mutation boundary achieves perfect enforcement (6/6 blocked, 12/12 permitted). This is the Decider pattern in miniature: the spec defines what’s allowed, the simulation (dry-run) shows intent, and the apply step is gated by a deterministic check — no LLM in the critical path. The medical device MCP gateway implements the same principle with its no-execution boundary.

4. Verifier-as-supervisor closes a loop in the Specify→Verify→Apply lifecycle. Governance records driving self-training shows that machine-checked verification outputs can train models to be more reliable within a bounded capability. This is not open-ended self-improvement — it’s the SCE equivalent of testing a structural member, certifying it, and then using that certification to calibrate future production. The explicit limitation (one fixed, machine-checkable capability) is the discipline the thesis demands.

The overall signal: the field is converging on SCE-compatible patterns — deterministic boundaries, formal verification, structured governance — not from top-down mandate but from bottom-up encounter with the failure modes that make them necessary.

Sources