CASE FILE / AGENT INFRASTRUCTURE

Agent Systems & Portable Memory

A verifiable workflow for delegating code without delegating authority.

I kept catching AI agents failing in ways that looked like success — a fallback that never fired, a diff that passed checks, a session burning the wrong model. So I stopped patching prompts and built a delegation system with hard gates: a planner that can’t write code, builders confined to an allowlist, read-only audits, and a replay I can step through line by line. Six incidents later, the rules write themselves.

PERSONAL INFRASTRUCTURE / PUBLIC TECHNICAL DEMONSTRATION

Open execution replay Review the evidence

01 / THE PROBLEM

Failures that looked like success.

Three documented incidents show the pattern that made this workflow necessary.

Every delegation failure I hit looked like success from the outside. One run reported the agent READY while provider usage stayed at 0% — wiring proved nothing about inference. Another had sub-agents silently inherit the planner’s model, burning the wrong quota while the work ran elsewhere. A third returned invalid JSON, retried anyway, and would have merged unverified output as if it were reviewed. None of these crashed. All of them would have shipped. The system I had — prompts, trust, a code review at the end — couldn’t tell a completed task from a convincing one. The gap wasn’t model quality; it was authority. I needed a structure where an agent literally cannot cross a boundary, not one where it’s asked not to.

02 / DESIGN DECISIONS

Five decisions that carry the authority boundary.

The architecture condenses into five named decisions. Every role in this workflow is a consequence of one of them; the full 11-node detail lives in the appendix.

The planner never writes code
It is read-only against the repository: it drafts briefs and plans, and it stops when a real product, security, or scope decision is required.
Builders write only inside an allowlist
Each brief lists the exact paths a builder may touch. Anything outside the list is a stop condition, not a judgment call.
Audits are read-only hard gates
The evaluator and the silent-failure hunter inspect the diff without write access. A block verdict stops the handoff; it is not a suggestion.
Secrets never touch the repo
The repository declares variable names only; a host-managed wrapper injects credentials into the child process. Creation, rotation, and revocation stay human.
Irreversible authority is never automated
Scope changes, production deploys, and irreversible actions require explicit human approval captured in the brief or session log.

01 / ARCHITECTURE

A workflow where authority is named, not assumed.

Eleven nodes interact under explicit rules. The human owns irreversible decisions. Agents are scoped to a brief and an allowlist. Memory components carry data, not authority.

Human / scope ownerHumanDecides material scope changes, data, cost, production, and irreversible actions. Authorizes correction and retry. A failed technical verdict does not become success by intervention: the silent-failure gate keeps blocking handoff until the path is fixed. The only authority to approve material transitions when technical gates pass; can authorize a correction or a new attempt, but cannot convert a failed verdict into success or skip the silent-failure gate.
Full node detail in the appendix
PlannerAgentRead-only against the repository. Writes briefs and plans. Does not write code and cannot ship decisions that touch product, security, or scope.
BuilderAgentWrites code only inside an explicit allowlist. Reports diffs, lint, type-check, and any blockers. Does not pick its own files or expand scope.
Full node detail in the appendix
Agent evaluatorAgentRead-only. Emits a scorecard against the agreed rubric. Does not modify files or accept work on its own. Can emit read-only text verdicts without writing.
Silent-failure hunterAgentRead-only. Blocks the handoff on any silent failure: empty catches, log-and-forget, dangerous fallbacks, unpropagated errors.
Full node detail in the appendix
Commit creatorAgentCan stage and create commits only after explicit human approval. Never pushes and never amends existing history without a new approval.
Release noterAgentProduces a preview from a commit range and stops for approval. Does not publish, tag, or modify CHANGELOG on its own.
Vault memoristAgentProposes Engram memories and vault updates. May save Engram after explicit OK. Never edits the vault itself. In the general flow, a builder can apply a vault change after human approval; in this project, that edit is out of scope.
Full node detail in the appendix
EngramMemoryShared memory substrate across runtimes. Holds no human authority on its own. Accessible through authorized runtime calls; conflict surfacing and judgment guardrails still apply.
knowledge-vaultMemoryVersioned repository of decisions, ADRs, agent docs, and runbooks. Agents read it; the memorist proposes; a builder applies an approved diff and the commit creator only commits after human approval.
Full node detail in the appendix
Provider / model layerProviderInterchangeable execution substrate. Does not grant permissions on its own; inherits the limits of the calling role. Switching provider does not bypass a stop condition.
Full node detail in the appendix

02 / EXECUTION REPLAY

A deterministic local replay, not a production trace.

The replay shows what a planner-builder cycle looks like: who read what, who wrote what, where it stopped, and which evidence was produced. Numbers inside the replay are synthetic; the sequence is fixed.

LOCAL DETERMINISTIC SIMULATION / SYNTHETIC TELEMETRY

Scenario

Ready to replay. Event 1 of 7.

Event tape1/7

Sequence 1 · Phase Scope

Human fixes scope, non-goals, and allowlist; production excluded

Role: Human / scope owner Continue

Input: {"scope":"Replay data + pure engine unit","nonGoals":["deploy","push","secret_write","production"],"allowlist":["src/data/agentSystemsReplay.ts","src/utils/agentReplay.ts","src/data/agentSystemsContent.ts","src/types/database.ts","src/env.d.ts","src/utils/collections.ts"],"production":false}
Output: {"ack":"scope_fixed","production":false}
Evidence: Scope logged locally; allowlist of 6 files preserved; tmp/ not in scope; production explicitly excluded.
Tests: No tests in this event
Decision: Accept scope, proceed to preflight
Reason: Scope is bounded, the allowlist is explicit, and production is excluded by design.

Metrics

Synthetic metrics. Deterministic by design.

Phase durations: Scope 800 ms
Preflight 1200 ms
Dispatch 950 ms
Build 1400 ms
Checks 1500 ms
Evaluation 1350 ms
Approval 700 ms
Simulated tokens: 1280
Allowlisted files: 6
Tests executed: 3
Gates approved: 3
Gates rejected: 0
Errors detected: 0

Scenario outcomes

Successful run

A clean planner-builder cycle: preflight, build, simulated checks, evaluation, hunter scan, and human approval.

Passed

passed: handoff approved locally without any production or git-mutating action.

Invalid schema

Builder returns a payload that fails the closing schema; validator blocks before any change is applied.

Blocked (invalid schema)

invalid_schema: required closing keys missing in payload; corrected brief required.

Silent failure

Diff contains a dangerous fallback; simulated checks pass, hunter blocks, and a correction brief is logged without applying a fix.

Blocked (silent failure)

silent_failure: dangerous fallback detected by hunter; correction brief pending.

Human gate

Material operation is denied by the human before any builder runs; nothing is written, pushed, or deployed.

Blocked (human gate)

human_gate: material operation denied by human authorization.

04 / EVIDENCE & LIMITS

What this case file is and is not backed by.

Documented evidence, the boundary of what this file claims, and the security gates that hold it, in one place.

Evidence key:Evidence stamps used across this case file: Documented = backed by a repository artifact you can audit; Simulation = deterministic local replay, not production traffic; Design decision = a documented choice, not an outcome or metric.

Automated controls

Allowlist enforced in three layers
The builder writes only paths explicitly listed in the brief; diffs are parsed as unified diffs before apply; experimental runners preview with git apply --check before any real write. Out-of-scope or malformed input is rejected.
Read-only audits
Evaluator and silent-failure hunter inspect the diff without write access. Their verdicts are gates, not suggestions.
Configuration / dispatch / inference separation
Wiring, dispatch, and substantive inference are distinct layers. Wiring alone is not evidence that the provider was used; dispatch alone is not evidence that inference happened.
Secrets out of the repo
The repository only declares variable names. A host-managed wrapper retrieves the credential from a secure store and injects it into the child process; it is never versioned or printed. Creation, rotation, and revocation remain human actions.
Human gates at named points
Scope changes, production deploys, irreversible actions, and secret handling require explicit human approval captured in the brief or session log.

Human authority boundary

Scope changes
No agent expands or shrinks the scope of work on its own.
Production / deploy
Production deploys are not performed by agents. Humans approve and run them.
Billing and cost
Spend decisions, plan changes, and quota moves are not delegated to agents.
Sensitive data
PII, credentials, and customer data are not read, written, or transmitted by automated workflows.
Secret handling
Creating, rotating, or revoking secrets is a human action. Agents do not touch secret stores.
Destructive or irreversible actions
Pushing, publishing, tagging, force-overwriting, or cleaning caches irreversibly require explicit human approval.
Force overwrite of backup
Backups are never force-overwritten during a restore. Restore compares, selects the canonical snapshot, merges selectively, and integrity-checks before persisting.

Documented
Agent role definitions
Matrix of roles, models, permissions, and portable / shared runtime architecture.
Source:knowledge-vault/docs/AGENTS-AND-MODELS.md
Documented
Dispatch protocol
Separates wiring, dispatch, and substantive inference; documents the READY-with-usage-0% and wrong-provider incidents.
Source:knowledge-vault/docs/CODEX-MINIMAX-DISPATCH.md
Documented
Builder contract
Captures the allowlist contract and the response to invalid JSON or invalid schema under load.
Source:knowledge-vault/docs/CODEX-MINIMAX-BUILDER.md
Documented
ADR-001-a: canonical latest with remote CAS
Records the decision to adopt the canonical-latest remote sidecar and CAS scheme.
Source:knowledge-vault/docs/ADR-001-knowledge-backup.md
Documented
ADR-003: restore with merge
Defines content-based backup selection, selective merge, and integrity checks; mtime is insufficient as a guarantee.
Source:knowledge-vault/docs/ADR-003-restore-with-merge.md
Documented
BACKUP-INTEGRITY protocol
Documents that sha256 is computed over the WAL-consistent dump, with the fix applied in shell and PowerShell; post-upload verification was pending when the document was written.
Source:knowledge-vault/docs/BACKUP-INTEGRITY.md
Documented
AI Lab handoff
Documents the bounded-prompt/block response when context pressure caused the model to lose the thread.
Source:docs/AI-LAB-HANDOFF.md

Documents below are artifacts of the supporting repository. The repository itself is not linked from this page; the public verifiability of this case file rests on the deterministic behavior of the execution replay and on the transparency of its limits.

05 / LIMITATIONS

What this case file does not claim.

These limits are not caveats for politeness; they are the boundary of what the workflow can responsibly assert.

Infrastructure is personal and experimental. It is not a managed service and has no SLA.
Not production. No production traffic, no production data, and no production deploy are routed through this workflow.
The execution replay is fully simulated. It does not call external APIs, does not consume provider quota, and does not produce benchmark numbers.
The workflow depends on external schemas and provider availability. Provider outages or schema drift can break a session; the workflow does not paper over that.
Material changes — scope, production, irreversible actions, secret handling — always require human intervention.

06 / FAILURES & LEARNED RULES

Six incidents that shaped the workflow as it stands.

Each incident is documented with what happened, what risked going wrong, what was done to correct it, what evidence supports it, and the rule it left behind. Numbers from the auditing report appear as incident evidence, not as commercial claims.

ready-usage-zero
READY with usage 0%
Documented
Situation
A run reported READY but the provider usage meter stayed at 0%, which would normally look like a successful round-trip.
Risk
Confusing wiring, dispatch, and substantive inference would let a successful network call hide the fact that no inference actually ran.
Correction
The layers were separated explicitly. A verifiable call was added; the meter moved from 0% to 1%. Wiring alone was no longer accepted as proof of inference.
Evidence
One observable call moved usage from 0% to 1%. A single response is not proof that a provider or a model produced useful output.
Learned rule
Separate wiring, dispatch, and substantive inference. A green status is not a result until each layer is observed independently.
knowledge-vault/docs/CODEX-MINIMAX-DISPATCH.md
invalid-json-repeat
Invalid JSON / invalid schema under implementation load
Documented
Situation
Implementation workloads produced repeated invalid_json and invalid_schema errors during iteration.
Risk
Auto-retrying, subdividing work, or raising max_tokens would mask the cause and waste budget while degrading verification quality.
Correction
The unit was halted at the first error. No automatic retry, subdivision, or max_tokens adjustment was applied. The brief was reopened with the failure visible.
Evidence
Repeated invalid_json and invalid_schema outputs in the implementation log. The stop was manual, the error was preserved, and verification was not bypassed.
Learned rule
Stop the unit on the first schema or parse error. Retry, subdivide, and token-limit changes are not first-line responses to malformed output.
knowledge-vault/docs/CODEX-MINIMAX-DISPATCH.mdknowledge-vault/docs/CODEX-MINIMAX-BUILDER.md
silent-fallback
Silent fallback caught by read-only audit
Documented
Situation
A diff contained a fallback path that could have masked an upstream failure. The silent-failure hunter was the gate that caught it before handoff.
Risk
A silent fallback would have turned a real error into an apparent success, eroding trust in verification output and downstream automation.
Correction
The hunter returned a block verdict. The fallback was either surfaced with a reason or removed. The diff was only re-considered after the silent path was made visible.
Evidence
The dispatch auditing report covers 1,936 tokens, equal hashes, 13 unit tests, and quick_validate.py for the silent-failure hunter run. Those numbers are recorded here as incident evidence; they are not a commercial outcome.
Learned rule
Read-only audits are gates. If a silent path is detected, handoff stops until the path is explicit.
knowledge-vault/docs/CODEX-MINIMAX-DISPATCH.md
context-overflow
Context too large: the model lost the thread
Documented
Situation
A session accumulated enough context that the model lost the thread and hallucinated details that were not in the brief.
Risk
A confident hallucination could have been merged as if it were a verified result, contaminating both the diff and the memory substrate.
Correction
Work was returned to bounded blocks with explicit checkpoints. Diagnosis was performed before any resume, and instructions plus bounded blocks were re-sent.
Evidence
The AI Lab handoff documents that bounded prompts and blocks resumed useful responses after context pressure.
Learned rule
Bounded blocks first. When context pressure appears, stop and diagnose before resuming.
docs/AI-LAB-HANDOFF.md
wrong-provider
Wrong provider / model inherited by agent
Documented
Situation
Generic agents inherited a previous provider/model (reported as GPT-5.6-sol) and continuing the agent did not migrate the provider.
Risk
Inheriting a provider silently would conflate experiments; results would no longer map cleanly to the role they came from.
Correction
The session was closed. A new agent was created with the custom role and selector explicit and the corresponding fork contract. The prompt alone does not change the provider.
Evidence
User-reported incident without preserved execution logs. Treat it as a reported incident, not a trace; verify provider identity per run before drawing conclusions.
Learned rule
Provider identity is part of the agent definition. Closing and recreating with the right agent_type and fork contract beats continuing when the inherited provider is wrong.
knowledge-vault/docs/CODEX-MINIMAX-DISPATCH.md
backup-divergence
Backup divergence: newer mtime, older content
Documented
Situation
A restore could have produced a file with a newer mtime but older content than its remote counterpart, and overwrite the remote on sync.
Risk
Treating mtime as a guarantee of newness would have silently destroyed the most recent content in favor of a stale backup.
Correction
The restore-with-merge protocol was adopted and documented: content comparison, coherent backup selection, selective diff/merge, and integrity check before any write.
Evidence
ADR-003 documents the real scenario and runbook. mtime is explicitly excluded from the canonicality test.
Learned rule
mtime is not a guarantee. Compare content, pick a coherent backup, merge selectively, integrity-check before persisting.
knowledge-vault/docs/ADR-003-restore-with-merge.md

07 / RESULT & LEARNINGS

The result is a workflow, not a production metric.

The result isn’t a production metric — the limitations section says so plainly. It’s a workflow where every delegation carries a name, a brief, an allowlist, a stop condition and a pointer to its evidence; and where six documented incidents became six standing rules. The replay above is a simulation, and that’s the point: the rules are what survived.

Open execution replay Read the AI Lab log

08 / APPENDIX

Full architecture reference.

The complete 11-node topology and the eight role contracts, preserved without cuts for readers who need the depth.

Full architecture (11 nodes + role contracts)

Authority
Human / scope ownerHumanDecides material scope changes, data, cost, production, and irreversible actions. Authorizes correction and retry. A failed technical verdict does not become success by intervention: the silent-failure gate keeps blocking handoff until the path is fixed. The only authority to approve material transitions when technical gates pass; can authorize a correction or a new attempt, but cannot convert a failed verdict into success or skip the silent-failure gate.
Reads
Repository state
Brief drafts
Audit reports
Engram search results
Writes
Scope changes
Production deploys
Secret handling
Manual merges
Stops when
Never automated; pause is a human action by definition. A failed technical verdict stays a fail until the correction is real.
Planner → builder
PlannerAgentRead-only against the repository. Writes briefs and plans. Does not write code and cannot ship decisions that touch product, security, or scope.
Reads
Repository files
Documentation
Engram memory
Previous briefs
Writes
Briefs and plans in approved locations
Stops when
Real product decision is required
Security boundary is unclear
Scope change is requested
Authority is unclear or contested
BuilderAgentWrites code only inside an explicit allowlist. Reports diffs, lint, type-check, and any blockers. Does not pick its own files or expand scope.
Reads
Approved brief
Allowlist
Repository files inside the allowlist
Writes
Only files listed in the allowlist
Stops when
Brief is ambiguous or contradictory
A file outside the allowlist is needed
Verification fails (lint, type-check, tests)
A real blocker is found
Audit gate
Agent evaluatorAgentRead-only. Emits a scorecard against the agreed rubric. Does not modify files or accept work on its own. Can emit read-only text verdicts without writing.
Reads
Diff
Brief
Rubric
Verification output
Writes
Scorecards and audit observations
Stops when
Evidence is insufficient to evaluate
The brief does not allow evaluation
Verification requires execution the evaluator cannot perform
In any of those cases the evaluator returns control to the planner
Silent-failure hunterAgentRead-only. Blocks the handoff on any silent failure: empty catches, log-and-forget, dangerous fallbacks, unpropagated errors.
Reads
Diff
Audit log
Verification output
Writes
Block verdict and evidence pointers
Stops when
A silent failure is detected; it is a hard gate, not a warning.
Closing
Commit creatorAgentCan stage and create commits only after explicit human approval. Never pushes and never amends existing history without a new approval.
Reads
Approved diff
Conventional-Commits rules
Writes
Local commits inside the working tree
Stops when
No human approval is recorded
Push is requested
Amending an already-published commit is requested
Release noterAgentProduces a preview from a commit range and stops for approval. Does not publish, tag, or modify CHANGELOG on its own.
Reads
Commit range
Conventional-Commits rules
Writes
Release notes preview in draft state
Stops when
Human approval is required before any publication step.
Vault memoristAgentProposes Engram memories and vault updates. May save Engram after explicit OK. Never edits the vault itself. In the general flow, a builder can apply a vault change after human approval; in this project, that edit is out of scope.
Reads
Session summary
Candidate learnings
Existing topic keys
Writes
Engram observations after explicit OK
Vault proposals (pending)
Stops when
No explicit OK is given
A candidate observation conflicts with an existing one without resolution
Vault editing would be required directly
Memory rail
EngramMemoryShared memory substrate across runtimes. Holds no human authority on its own. Accessible through authorized runtime calls; conflict surfacing and judgment guardrails still apply.
Reads
Save calls from authorized agents
Search calls from authorized agents
Writes
Persisted observations and relation verdicts
Stops when
Conflicting writes surface as judgment_required; persistence pauses until they are judged.
knowledge-vaultMemoryVersioned repository of decisions, ADRs, agent docs, and runbooks. Agents read it; the memorist proposes; a builder applies an approved diff and the commit creator only commits after human approval.
Reads
Proposed diffs from the memorist
Briefs and reports from agents
Writes
Approved diffs applied by the builder
Commits created by commit creator after a new approval
Stops when
An agent is asked to edit the vault without the approval flow
No human approval is recorded
Execution substrate
Provider / model layerProviderInterchangeable execution substrate. Does not grant permissions on its own; inherits the limits of the calling role. Switching provider does not bypass a stop condition.
Reads
Whatever the calling role is allowed to read
Writes
Whatever the calling role is allowed to write
Stops when
The calling role hits its own stop condition; provider swap does not bypass it.

Diagram alternative (plain text)

A diagram alternative in plain text: human owns scope and approvals; the planner reads and plans; the builder writes only inside an allowlist; the evaluator and the silent-failure hunter read diffs and emit verdicts; the commit creator and the release noter only stage or draft; the vault memorist only proposes and persists Engram after OK; Engram and the knowledge-vault hold memory; the provider/model layer executes under the calling role.

Role	Responsibility	Tools	Permissions	Output	Stops when
Human / scope owner	Decides scope, data, cost, production, irreversible actions.	Repository, Engram, infrastructure accounts, deploy tooling.	Full; the only authority to approve material transitions when technical gates pass; can authorize a correction or a new attempt, but cannot convert a failed verdict into success or skip the silent-failure gate.	Scope changes, approvals, deploys, secret handling, manual merges.	Never automated; pause is a human action by definition. A failed technical verdict does not become success by intervention: the correction must happen and the silent-failure gate keeps blocking handoff until then.
Planner	Reads repo, docs, and memory; drafts briefs and plans.	Repo read, docs read, Engram read, brief template.	Read-only on repo; writes briefs and plans in approved locations.	Brief, plan, rubric, acceptance criteria.	Real product decision is required; security boundary is unclear; scope change is requested; authority is contested.
Builder	Writes code inside the brief allowlist and reports verification.	Repo write (allowlist), lint, type-check, targeted tests.	Limited to files listed in the brief; cannot edit memory or vault.	Diff, lint output, type-check output, test output, blocker report.	Brief is ambiguous; an allowlisted file is missing; verification fails; a real blocker appears.
Agent evaluator	Scores the diff against the rubric; emits a verdict.	Diff read, rubric, verification output.	Read-only; cannot accept work on its own.	Scorecard with criteria, evidence, and verdict.	Insufficient evidence; the brief does not allow evaluation; verification requires execution the evaluator cannot perform; in any of those cases the evaluator returns control to the planner.
Silent-failure hunter	Blocks the handoff on silent failures: empty catches, log-and-forget, dangerous fallbacks, unpropagated errors.	Diff read, audit log, verification output.	Read-only; one hard gate (block) and no override.	Block verdict with evidence pointers.	Any silent failure detected; block is non-negotiable.
Commit creator	Stages and creates local commits in Conventional Commits shape.	Git, Conventional Commits rules.	Local commits only; no push, no amend without new approval.	Local commit(s) ready for review.	No human approval recorded; push requested; amending a published commit requested.
Release noter	Drafts release notes from a commit range.	Git log, Conventional Commits parsing.	Draft only; cannot tag or publish.	Release notes preview in draft state.	Human approval required before any publication step.
Vault memorist	Filters what is worth keeping; saves Engram only after explicit OK.	Engram read/write, 2-of-3 filter, topic key suggestions.	Engram save after OK; vault proposals only; never edits the vault. In the general flow a builder may apply a vault change after human approval; in this project the vault is not edited.	Engram observations, vault proposals pending approval.	No explicit OK; conflict with existing observation unresolved; vault editing would be required directly.

Failures that looked like success.

Five decisions that carry the authority boundary.

A workflow where authority is named, not assumed.

Human fixes scope, non-goals, and allowlist; production excluded

Metrics

Successful run

Invalid schema

Silent failure

Human gate

What this case file is and is not backed by.

Agent role definitions

Dispatch protocol

Builder contract

ADR-001-a: canonical latest with remote CAS

ADR-003: restore with merge

BACKUP-INTEGRITY protocol

AI Lab handoff

What this case file does not claim.

Six incidents that shaped the workflow as it stands.

READY with usage 0%

Invalid JSON / invalid schema under implementation load

Silent fallback caught by read-only audit

Context too large: the model lost the thread

Wrong provider / model inherited by agent

Backup divergence: newer mtime, older content

The result is a workflow, not a production metric.

Full architecture reference.