Chronograph

Graph View

2026-11-03

11 min read

Martin Rosén-Lidholm
Martin Rosén-Lidholm

Daily D4 Digest — 2026-11-03

TL;DR

  • Formal verification is converging with agentic engineering at speed: three independent papers use Lean 4 to prove GPU kernel correctness, enforce runtime guardrails on tool-using agents, and repair concurrent proofs — all with LLM agents driving the proof search.
  • The SIMPLE law quantifies agentic inference cost at 13× non-agentic energy per request, with KV-cache growth as the irreducible driver even after prefix caching.
  • CARE demonstrates a production-grade recursive self-improvement loop for customer-service agents that replaces frontier models with self-distilled mid-sized models, directly attacking support-tier cost.
  • Memorix cuts agent memory assembly cost by up to 100× through precomputed, versioned context views — a systems-level intervention for the agentic factory’s token budget.
  • The “Internet of Agentics” paper maps the full MCP/A2A/AGENTS.md protocol stack into a three-layer maturity model from fragmented → interoperable → governable.

Call to Action

  • Instrument your agentic pipelines with the SIMPLE energy model to forecast per-request cost as reasoning budgets and context lengths grow — the 13× agentic multiplier means your inference bill is likely nonlinear in ways you haven’t modeled. SIMPLE Law paper
  • Evaluate LeanGuard-style runtime guardrails for any tool-calling agent in production — the 57.9% → 5.9% prompt-injection reduction with zero benign task degradation is a concrete D4 win. LeanGuard paper
  • Prototype precomputed agent memory views (à la Memorix) for your coding agents — 100× amortized cost reduction on context assembly is low-hanging fruit for teams running multi-session agentic workflows. Memorix paper

D1 — Agentic Engineering

Strategist automates the meta-problem of inference strategy design. Instead of hand-coding chain-of-thought, tree-of-thought, or other reasoning scaffolds per task, Strategist introduces a meta-agent that evolves inference strategies as reusable “blueprints” — executable code that composes into a growing library. On 30 benchmarks it matches the best hand-tuned baselines at 39% lower inference cost, and strategies transfer across models without further search. This is significant for D1 because it moves orchestration design from artisan craft to an automated, accumulating practice. The blueprint library is effectively institutional memory for how to reason — a capability that compounds. (Cross-cutting: D4 — the 39% inference cost reduction directly addresses factory cost.)

LARK formally verifies LLM-generated GPU kernels against PyTorch references. LARK uses Lean 4 to prove functional and memory equivalence of generated CUDA/Triton kernels, decomposing whole-program proofs into agent-tractable segments. On the KernelCraft benchmark, it fully verified 10/12 Level-1 and 5/5 Level-2 kernels, and on 10 production kernels from open-source LLM serving frameworks, it verified 9 and discovered a real data-race bug in the tenth. The key D1 insight: the trust boundary shifts from “review every line” to “verify operator-level Lean semantics once” — a scalable model for agentic code generation where output velocity exceeds human review capacity.

Parametric CAD Bench v3 benchmarks agentic engineering in physical-world design. This benchmark evaluates 13 model-agent systems on 100 parametric CAD tasks — text-to-CAD creation, editing, and reconstruction from engineering drawings — using FreeCAD with a programmatic verifier that checks geometry, dimensions, and modeling history. Claude Code with Opus 5.5 leads at 61% mean reward. The benchmark measures complete systems, not isolated model capabilities, which is exactly the right unit of analysis for D1: the orchestration, tool integration, and verify-then-submit loop matter as much as the model.

TLAPS-Bench brings agentic proof construction to TLA+ system specifications. TLAPS-Bench provides 265 proof targets from 95 real-world TLA+ specs (cache coherence, Byzantine consensus, write-ahead logs) and an execution harness for AI agents to construct machine-checked proofs. GPT-6.1 Sol proved 44.3% of targets. This directly serves D1 teams that use TLA+ for distributed system design — agents can now attempt to prove properties that were previously left unproven due to human effort constraints. The benchmark includes safeguards against reward hacking, addressing a known failure mode of agentic proof search.

VerusWeave benchmarks agentic proof repair for concurrent Rust programs. VerusWeave targets the hardest verification niche — concurrency — with 75 focused tasks and 17 full-module variants derived from real Verus projects. Current coding agents achieve 75–85% on focused tasks but only 29–47% on full-module repairs, quantifying the gap between local and holistic reasoning in concurrent code. For D1 teams writing verified concurrent infrastructure, this benchmark provides a concrete yardstick for when agents become trustworthy enough for the verify step of the Specify→Plan→Verify→Apply→Observe lifecycle.

Bounded canvas for AI agents formalizes drift detection and decision records. Nimisha Asthagiri’s presentation at InfoQ introduces a “canvas” model that gives agents bounded autonomy through continuous drift detection and decision records. The architecture applies proven product engineering principles — constraints as enablers — to create a resilient scaling model for agentic development. This maps directly to the D1 concern of keeping agents productive within guardrails rather than letting them wander. (Cross-cutting: SCE — the “bounded canvas” is an explicit implementation of the bounded autonomy concept.)

D2 — AI in the Product

CARE replaces frontier models with self-improving in-house agents for customer service. CARE presents a complete production framework: cold-start on governed traces → serve alongside incumbent → 24/7 failure inspection with evidence-blinded review → attribution (model vs. tool vs. harness) → on-policy self-distillation from truncated failure trajectories. The goal is explicit: replace a frontier proprietary model with a mid-sized in-house model that continues learning. This is a mature D2 pattern — the AI is the product surface for customer service — and the recursive self-improvement loop means the product gets better from its own production traffic without manual fine-tuning cycles. (Cross-cutting: D4 — directly attacks support-tier cost and model inference cost simultaneously.)

GURU demonstrates neuro-symbolic agents for mission-critical engineering operations. GURU enforces a strict partition: learned models propose and rank, a symbolic rules engine checks against hard limits, and validated physics decides. In CFD, it raised converged-simulation rates from <1% to 74% on unseen geometries. In space domain awareness, a frozen self-supervised model under validated propagation beat fully fine-tuned LLMs at conjunction risk assessment with lower variance. For D2, GURU shows how to embed AI agents in products where failure has physical consequences — nuclear, aviation, space — by making the safety basis explicit and non-learnable.

D3 — Build for Agents

The “Internet of Agentics” paper provides the most complete architectural map of the agent interoperability stack to date. This perspective paper organizes the emerging protocol landscape — MCP for agent-tool interaction, A2A for agent-agent coordination, AGENTS.md and Skills manifests for capability declaration, AGNTCY and SLIM for discovery and trust — into a three-layer maturity model: fragmented → interoperable → trustworthy/governable. The key argument is that agentic systems reinterpret classical SOA/microservices concerns (discovery, composition, governance, pricing, trust) at the level of cognitive capabilities rather than deterministic functions. For a CTO building B2A surfaces, this is the roadmap: your API isn’t consumed by developers writing code, it’s consumed by agents needing capability manifests, trust signals, and runtime governance hooks.

Memorix redefines how agents consume project context. While primarily a D1/D4 story, Memorix has D3 implications: its versioned, immutable Store and reproducible View architecture means that agents consuming your project’s documentation or API surface get the right version every time. If you’re building for agent consumers (D3), providing Memorix-compatible versioned context sources — rather than flat documentation — could become a differentiator.

D4 — Cost of Ownership

CARE’s recursive self-improvement loop directly attacks the 2× output / ½× downstream cost equation by systematically reducing support escalations. CARE doesn’t just deploy a customer-service agent — it builds a closed loop where production failures are attributed (model error vs. tool defect vs. harness bug), model errors are repaired through on-policy self-distillation, and component failures are routed to the appropriate repair channel. The 24/7 inspection process with evidence-blinded review creates an observability layer that turns every incident into training signal. This means the system’s error rate decreases over time without human retraining effort, directly reducing the rate at which conversations escalate from automated → L1 → L2 support. The simulation-and-inspection gate before promotion prevents regressions, so you get compounding improvement without compounding risk.

The SIMPLE law quantifies agentic inference as a 13× energy multiplier, with KV-cache growth as the structural cost driver. This paper derives closed-form energy cost expressions for LLM inference and finds that agentic serving — characterized by longer inputs and multi-turn context accumulation — costs 13.0× more energy per request than non-agentic serving. Prefix caching eliminates 93% of prefill cost, but the KV-cache term is unaffected, leaving cached agentic requests still at 4.8× non-agentic. Additionally, increasing reasoning token budgets drives super-linear energy growth: “Intelligence per Joule” peaks at an intermediate budget and falls by up to 3.14× across a 4× reasoning-budget range. For any team scaling agentic pipelines, this is the cost curve you’re riding — and it argues strongly for Strategist-style inference optimization (39% cost reduction) and Memorix-style context compression (100× assembly cost reduction) as first-order D4 interventions.

Memorix cuts the agentic factory’s context assembly cost by up to 100× through precomputed, versioned memory views. Memorix addresses a surprisingly expensive operational cost: assembling the right context for each agent task. Current systems splice files into prompts ad hoc, which is both token-expensive and error-prone (agents reading docs for the wrong library version). By precomputing context selections per project and reusing them across tasks, Memorix raises task resolution rates by up to 1.5× while cutting amortized cost by 100× and build time by 38×. The versioning guarantee also eliminates a class of subtle failures — agent actions based on stale documentation — that would otherwise surface as hard-to-diagnose incidents downstream.

Software Civil Engineering Lens

Today’s batch is the strongest single-day signal for the SCE thesis I’ve seen. The convergence is striking: five independent research groups are building formal verification tooling for agentic systems, each addressing a different pillar of professionalization.

Formal specification is being automated. TLAPS-Bench and VerusWeave don’t just benchmark formal proofs — they demonstrate that agents can construct and repair machine-checked proofs for real distributed systems (cache coherence, Byzantine consensus) and concurrent programs. This directly addresses the SCE “formal spec” and “simulation” pillars. The 44.3% success rate on TLA+ proofs is low enough to keep humans essential but high enough to make previously unproven properties provable within practical time budgets. We’re at the inflection point where proof becomes cheaper than not proving.

Codes and norms are becoming executable. LeanGuard writes domain policies in Lean 4 and checks every tool call at runtime — fail-closed, with proved equivalence to formal LTL semantics. This is literally a building code for agents: a machine-verified set of rules that constrain what an agent may do, with proof-guided repair when violations occur. The 57.9% → 5.9% prompt-injection reduction shows the code works. GURU’s symbolic rules engine serves the same function for mission-critical domains, encoding hard limits that learned models cannot override.

Material datasheets are emerging. The SIMPLE law is essentially a datasheet for LLM inference energy: given architectural hyperparameters, you can predict cost without per-model calibration. The Parametric CAD Bench’s continuous task reward metric — combining geometry, dimensions, and modeling history — provides a “material strength test” for agentic CAD systems. Both enable engineering decisions based on known properties rather than empirical hope.

The bounded autonomy thesis is confirmed. The Asthagiri presentation explicitly implements bounded autonomy with drift detection and decision records. GURU’s partition (“AI proposes, symbolic engine checks, physics decides”) is bounded autonomy for safety-critical systems. LeanGuard’s fail-closed guardrails are bounded autonomy for tool-calling agents. Three different architectures, three different domains, same principle: constrain the agent formally, and you can trust it more while reviewing it less.

The missing pillars remain licensure and education — no one is proposing certification for agentic engineering practitioners yet — but the tooling for formal spec, material properties, codes/norms, and simulation is arriving faster than the profession is organizing to adopt it. The gap is no longer “we lack the tools” but “we lack the institutional structures to mandate their use.”

Sources