Supervision recipes for AI coding agents
4 ordered compositions of tools that address one or more failure modes. Recipes are the prescriptive flip side of incidents: patterns that have worked, or plausibly could, in practice.
-
context-hub (current API docs) → pydantic-ai (schema-validated outputs) → e2b (sandboxed execution) → pytest post-check. Fails fast on hallucinated symbols before any change leaves the sandbox.
case studied python -
context-hub (current, versioned API docs given to the agent before it writes code) plus semgrep rules that flag calls the project has marked as deprecated.
anecdotal typescript -
Supply-chain-hardened agent
2 toolssocket (package-risk telemetry) → snyk (CVE scan) → sandboxed execution (no maintained tool in the catalogue). Intercepts hallucinated / malicious packages before they reach the dev machine.
case studied polyglot -
Test-manipulation-resistant flow
2 toolssemgrep rules on test-file diffs plus a promptfoo LLM judge that reads the agent's session. Flags edits that weaken, skip or delete tests.
anecdotal polyglot