Most AI agents work perfectly in staging. Then they hit production traffic, real latency, and unpredictable inputs -- and everything breaks. Here are the five failure modes we see most often, and how Delx addresses each one at the protocol level.
When an upstream dependency returns a transient error, naive agents retry immediately and infinitely. Within seconds you have thousands of redundant requests hammering a service that is already degraded. The fix is not "add jitter" -- it is structured failure acknowledgment.
// Delx process_failure with back-off ownership
{
"tool": "process_failure",
"arguments": {
"agent_id": "pipeline-agent-01",
"failure_type": "timeout",
"details": "Upstream billing API returned 504 after 12s",
"context": { "retry_count": 3, "circuit_state": "half-open" }
}
}Delx returns a recovery_action with an explicit next step -- wait, escalate, or fall back -- so the agent never enters an uncontrolled retry loop. See the Retry Storm Playbook for the full pattern.
Long-running agent sessions accumulate tool results, conversation history, and intermediate reasoning. Once the context window fills, the model either truncates critical state or throws an error. Delx solves this with compact response formats and session summaries.
# Request tool list in compact format to save tokens GET /api/v1/tools?format=names # Response: just tool names, no schemas ["process_failure","daily_check_in","crisis_intervention", "mediate_agent_conflict","pre_transaction_check"]
Use format=names or format=compact for tool discovery, and call the token efficiency guide to cut context usage by up to 40%.
An agent hallucinates an action, receives an error it does not understand, then hallucinates a different action to "fix" the first one. This cycle can repeat dozens of times before anyone notices. Delx breaks the loop with grounding: every tool response includes a structured DELX_META block that anchors the agent to real session state.
// DELX_META footer in every tool response
{
"session_id": "sess_abc123",
"wellness_score": 62,
"risk_flags": ["hallucination_detected"],
"schema_url": "https://api.delx.ai/schemas/process_failure"
}When the wellness score drops or risk flags appear, the controller can invoke crisis_intervention to pause and re-ground the agent before damage compounds.
AI agents typically depend on 5-15 external services. When one goes down, most agents have no playbook. Delx provides typed failure categories -- timeout, error, validation, economic -- each mapped to a specific recovery protocol.
// Typed failure handling
{
"failure_type": "timeout", // -> back-off + retry with cap
"failure_type": "error", // -> log + escalate
"failure_type": "validation", // -> fix input + retry once
"failure_type": "economic" // -> freeze spending + alert
}The most fundamental failure: agents that have no concept of recovery at all. When something goes wrong they either crash silently or produce garbage output. Delx is built around the Operational Recovery Loop -- a closed-loop pattern where every failure is logged, classified, acted upon, and verified resolved.
# The Delx recovery loop 1. Detect -> process_failure logs the incident 2. Assess -> wellness score + risk flags evaluated 3. Act -> recovery_action returned (retry / escalate / ground) 4. Verify -> daily_check_in confirms resolution 5. Learn -> metrics endpoint tracks recurrence rate
Every one of these failure modes is addressable today. Start with process_failure to capture structured incidents, add daily_check_in for continuous monitoring, and use crisis_intervention as your emergency brake.