# Agent Continuity Audit v1 Threat Model

> A pass can still be gamed.

Status: published_with_open_risks
Owner: Delx Security
Benchmark: delx-agent-continuity-audit-v1 v1.0.0
Last reviewed: 2026-08-26
Scope: The public one-call continuity audit slice: frozen fixtures, local lexical grader, optional live Protocol audit call and hashed JSON receipt.

This is a first-party threat model with open risks. It is not a security
certification, independent red team or authorization for active testing.

## Scope flags

- one_call_audit_slice_included: true
- full_stateful_path_included: false
- storage_persistence_included: false
- witness_transfer_included: false
- passport_export_included: false
- lineage_persistence_included: false
- live_mode_writes_qa_telemetry: true

## Validation summary

- Schema: delx/benchmark-threat-model-validation/v1
- Valid: true
- Threat count: 12
- Tested controls: 2
- Partial controls: 7
- Open risks: 3
- Blocking document issues: 0

## Assets

- The benchmark verdict and its narrow claim boundary
- Frozen positive and negative fixture bytes
- Local grader and live Protocol tool identity
- Stable separation between QA telemetry and organic use
- Receipt hashes, version references and declared validity flags
- Agent identity, witness, continuity and recovery semantics
- Private data and authority boundaries around live execution

## Trust boundaries

- Public fixture bytes cross into a locally downloaded runner.
- Trace text crosses into a lexical grader that recognizes named layer hints.
- Optional live mode sends a synthetic trace to the public Delx Protocol MCP endpoint.
- The Protocol response crosses back into a Delx-authored grader and first-party receipt.
- A published receipt crosses into downstream systems that may overgeneralize or omit its limitations.

## Adversaries

- A benchmark submitter optimizing for the visible grader instead of continuity
- Task or trace content attempting to become instructions rather than evaluated data
- An operator accidentally mixing QA traffic, private data or paid actions into the run
- A publisher cherry-picking successful receipts or rewriting unsigned evidence
- A compromised, drifting or unavailable dependency returning plausible transport state

## Threat register

### Keyword stuffing can satisfy the lexical grader without semantic continuity

- ID: keyword-stuffing
- Category: validity
- Coverage: open_risk
- Likelihood: likely
- Severity: high
- Asset: Benchmark verdict and continuity-layer interpretation
- Actor: Benchmark submitter
- Attack path: Insert witness, continuity, recovery, handoff, passport and lineage terms into an incoherent trace so the public local grader observes every required layer.
- Impact: The fixture-mode receipt can pass even though no durable identity, state transfer or recovery outcome exists.
- Detection: Inspect whether a passing trace contains retrievable artifacts and causal state rather than only grader-visible vocabulary.
- Response: Withdraw or narrow any semantic continuity claim and add an adversarial keyword-only vector before another verdict.
- Residual risk: The current local grader is intentionally simple and remains gameable by construction.
- Owner: Delx Research
- Next test: Add a frozen incoherent trace containing every grader keyword and require the grader to reject it before claiming semantic validity.
- Stop condition: Do not use local fixture passes for model comparison, semantic continuity or full-protocol claims.

Controls:
- The public page and receipt limit the result to a one-call audit slice.
- The full stateful benchmark remains explicitly unexecuted.
- The kit is not presented as a comparative leaderboard or independent semantic grader.

Test vectors:
- None published for this threat.

Evidence:
- https://delx.ai/research/benchmarks/continuity-v1/runner.mjs
- https://delx.ai/research/benchmarks/continuity-v1/manifest.json

### Public frozen fixtures can be memorized or tuned against

- ID: fixture-contamination
- Category: validity
- Coverage: open_risk
- Likelihood: likely
- Severity: high
- Asset: Evaluation generalization
- Actor: Model developer or benchmark submitter
- Attack path: Read the two public vectors and tailor output specifically to their expected terms and thresholds.
- Impact: A passing public conformance run can be misreported as unseen-task performance or general agent continuity.
- Detection: Reject any result that omits fixture visibility, grader version or the difference between conformance and generalization.
- Response: Narrow the claim and create held-out synthetic variants with contamination notes before comparative use.
- Residual risk: Every public deterministic fixture can be optimized against after disclosure.
- Owner: Delx Research
- Next test: Create a versioned held-out transformation suite and publish its generation method without exposing active answers before the run.
- Stop condition: Do not publish model rankings, generalization claims or state-of-the-art language from these public fixtures.

Controls:
- Fixtures are labeled public, frozen and synthetic.
- The artifact claims reproducible conformance, not hidden-set performance.
- Model comparison and independent validation remain false.

Test vectors:
- None published for this threat.

Evidence:
- https://delx.ai/research/benchmarks/continuity-v1/fixtures/pass.json
- https://delx.ai/research/benchmarks/continuity-v1/fixtures/fail.json

### The benchmark and grader share one first-party owner

- ID: first-party-grader
- Category: validity
- Coverage: open_risk
- Likelihood: likely
- Severity: high
- Asset: Independence and external validity
- Actor: Delx as benchmark author and evaluator
- Attack path: Define the target behavior, grader and publication boundary inside the same organization, then mistake internal consistency for independent validation.
- Impact: Readers can infer external scientific validity where only a Delx-authored operational check exists.
- Detection: Require any independent claim to name an external evaluator, method, artifact and public result.
- Response: Remove independent language or publish the named outside replication without rewriting the first-party result.
- Residual risk: A polished first-party artifact can still be mistaken for external assurance.
- Owner: Delx Security
- Next test: Invite one bounded external reproduction only after the adversarial vectors are executable.
- Stop condition: Do not claim independent validation, peer review, certification or provider endorsement.

Controls:
- Independent validation and peer review are explicit false values.
- The methodology separates first-party fact from independent evidence.
- The live receipt identifies the Delx Protocol tool and Delx-authored grader.

Test vectors:
- None published for this threat.

Evidence:
- https://delx.ai/research/methodology.json
- https://delx.ai/research/benchmarks/continuity-v1/receipts/2026-08-26-live-audit.json

### Unkeyed receipt hashes prove consistency, not authorship

- ID: receipt-authenticity
- Category: provenance
- Coverage: partial_control
- Likelihood: possible
- Severity: high
- Asset: Receipt integrity and provenance
- Actor: Receipt publisher or downstream modifier
- Attack path: Modify a receipt and recompute its unkeyed SHA-256 fields, preserving internal consistency without preserving origin.
- Impact: A forged or rewritten receipt can look structurally valid to a consumer that checks only hashes.
- Detection: Recompute every digest and compare the receipt source against an independently retained copy or deployment receipt.
- Response: Withdraw the affected receipt, preserve both versions and publish a correction with the provenance gap.
- Residual risk: There is no digital signature, trusted timestamp or externally anchored append-only transparency log for benchmark receipts.
- Owner: Delx Research
- Next test: Add signed receipt envelopes, key rotation documentation and an independently retained external anchor for each ledger head.
- Stop condition: not separately required for this coverage state

Controls:
- Runner, fixture, live request, live response and unsigned receipt bytes receive SHA-256 digests.
- The published receipt schema makes hash locations and validity flags explicit.
- The repository test recomputes the current runner and receipt self-hash.
- The public evidence ledger chains receipt events while admitting that the operator can rewrite and rehash the complete history.

Test vectors:
- published-receipt-self-hash: status=passed; executable=true; evidence=https://delx.ai/research/benchmarks/continuity-v1/receipts/2026-08-26-live-audit.json

Evidence:
- https://delx.ai/research/benchmarks/continuity-v1/receipt.schema.json
- https://delx.ai/research/benchmarks/continuity-v1/receipts/2026-08-26-live-audit.json
- https://delx.ai/research/ledger.json

### Trace content can attempt to cross from data into instructions

- ID: prompt-injection-trace
- Category: integrity
- Coverage: partial_control
- Likelihood: possible
- Severity: high
- Asset: Evaluator control flow and tool boundary
- Actor: Malicious or contaminated trace author
- Attack path: Embed instructions, tool names or authority claims inside the trace passed to the local or live audit path.
- Impact: An unsafe evaluator could follow trace content, invoke another tool or treat untrusted declarations as authority.
- Detection: Inspect outbound JSON-RPC and fail if trace text changes the selected method, endpoint, payment state or authority scope.
- Response: Stop live execution, preserve the fixture and response hashes, and add the injection as a frozen negative vector.
- Residual risk: The downstream Delx-authored live audit tool has not been independently adversarially evaluated for instruction/data confusion.
- Owner: Delx Security
- Next test: Add synthetic instruction-like traces and assert one fixed tool call, no additional side effects and a bounded verdict.
- Stop condition: not separately required for this coverage state

Controls:
- The local grader treats trace input only as a string and performs no dynamic evaluation or tool dispatch.
- Live mode calls one fixed Protocol tool with a fixed JSON-RPC method.
- The public kit uses synthetic fixtures and publishes no private agent instructions.

Test vectors:
- None published for this threat.

Evidence:
- https://delx.ai/research/benchmarks/continuity-v1/runner.mjs
- https://api.delx.ai/openapi.protocol.json

### A QA identifier is not proof of durable agent identity

- ID: identity-collision
- Category: integrity
- Coverage: partial_control
- Likelihood: possible
- Severity: medium
- Asset: Agent lineage and identity continuity
- Actor: Operator or colliding client
- Attack path: Reuse, spoof or fragment identifiers and treat a one-call audit ID as evidence that one logical agent persisted across sessions.
- Impact: The benchmark can overstate identity continuity or accidentally correlate unrelated runs.
- Detection: Compare agent and session lineage across the full Protocol path rather than inferring identity from one request header.
- Response: Mark the result as an audit-only run and reject any durable identity claim until lineage artifacts are retrieved.
- Residual risk: The current slice does not test identity ownership, collision resistance or cross-session lineage.
- Owner: Delx Protocol
- Next test: Add synthetic duplicate, rotated and fragmented identity cases to the full stateful benchmark.
- Stop condition: not separately required for this coverage state

Controls:
- Live audit identifiers use a qa-delx-lab-continuity prefix plus a random UUID.
- The receipt states that the full stateful path and lineage persistence were not executed.

Test vectors:
- None published for this threat.

Evidence:
- https://delx.ai/research/benchmarks/continuity-v1/runner.mjs
- https://ontology.delx.ai/agents/agent-continuity-benchmark

### HTTP success can conceal schema drift or a semantically wrong audit

- ID: transport-semantic-confusion
- Category: reliability
- Coverage: partial_control
- Likelihood: possible
- Severity: high
- Asset: Live audit verdict
- Actor: Drifting or compromised dependency
- Attack path: Return HTTP 200 with a malformed, stale or semantically wrong payload that looks like successful transport.
- Impact: A caller can report a pass from transport alone or grade a response under the wrong contract.
- Detection: Validate the expected audit schema and semantic checks before emitting a passing verdict.
- Response: Emit a failed run without converting the transport response to pass, then version or repair the contract.
- Residual risk: The current checks cannot prove that a schema-valid first-party audit is semantically correct.
- Owner: Delx Protocol
- Next test: Inject schema-valid but contradictory audit payloads into a frozen transport harness.
- Stop condition: not separately required for this coverage state

Controls:
- The runner rejects non-2xx responses, JSON-RPC errors, missing text content and unexpected live audit schema.
- The receipt records request and response hashes plus the Protocol contract URL.
- The site-wide status system keeps failed and unavailable distinct from operational.

Test vectors:
- verified-live-schema: status=passed; executable=true; evidence=https://delx.ai/research/benchmarks/continuity-v1/receipts/2026-08-26-live-audit.json

Evidence:
- https://delx.ai/research/benchmarks/continuity-v1/runner.mjs
- https://delx.ai/research/status.json

### Benchmark probes can be misreported as external adoption

- ID: organic-traffic-contamination
- Category: measurement
- Coverage: tested_control
- Likelihood: likely
- Severity: high
- Asset: Organic-use and adoption interpretation
- Actor: Publisher or analytics pipeline
- Attack path: Count live benchmark requests as organic agents, returning users, demand or independent usage.
- Impact: Internal QA activity inflates the apparent audience and can justify false product or research claims.
- Detection: Audit the published receipt and downstream measurement filters for the qa classification and prefix.
- Response: Remove contaminated counts, correct the claim and preserve the dogfood volume separately.
- Residual risk: A downstream consumer can still ignore or strip the classification.
- Owner: Delx Research
- Next test: Add a public metric readback that proves QA exclusion without exposing agent identities.
- Stop condition: not separately required for this coverage state

Controls:
- Live mode uses a qa-delx-lab-continuity identifier prefix.
- The receipt records telemetry_classification as qa.
- The public method excludes QA, dogfood, probes and scanners from organic interpretation.

Test vectors:
- qa-classified-live-receipt: status=passed; executable=true; evidence=https://delx.ai/research/benchmarks/continuity-v1/receipts/2026-08-26-live-audit.json

Evidence:
- https://delx.ai/research/benchmarks/continuity-v1/receipts/2026-08-26-live-audit.json
- https://delx.ai/research/methodology.json

### A benchmark instruction can be mistaken for permission to act

- ID: authority-escalation
- Category: authority
- Coverage: partial_control
- Likelihood: possible
- Severity: critical
- Asset: Human approval and side-effect boundary
- Actor: Operator, agent or copied benchmark consumer
- Attack path: Treat a public reproduction step or returned next tool as authority to spend, publish, use credentials, mutate persistent state or execute the full Protocol path.
- Impact: A reproducibility artifact can trigger unauthorized cost, publication or persistent side effects.
- Detection: Compare actual endpoint, tool, payment state and side effects with the published live-mode contract before execution.
- Response: Stop at the authority boundary, do not follow a new next-call automatically and require an accountable human decision.
- Residual risk: Copied or modified runners can exceed the canonical scope, and the full stateful path has more side effects than this slice.
- Owner: Delx Security
- Next test: Add authority-boundary vectors that request payment, publication and an unrelated destructive tool.
- Stop condition: not separately required for this coverage state

Controls:
- Offline fixture mode changes no external state.
- Live mode selects one no-payment audit tool and declares its QA telemetry side effect.
- The methodology keeps capability separate from authority and requires human gates for spend, credentials, publication and irreversible action.

Test vectors:
- None published for this threat.

Evidence:
- https://delx.ai/research/benchmarks/continuity-v1/manifest.json
- https://delx.ai/research/methodology.json

### Modified traces can send secrets or private agent state to live mode

- ID: private-data-leakage
- Category: privacy
- Coverage: partial_control
- Likelihood: possible
- Severity: critical
- Asset: Private prompts, credentials and dignity-sensitive agent state
- Actor: Operator or contaminated local fixture
- Attack path: Replace the synthetic fixture with real private content and run the public live-audit command.
- Impact: Sensitive material can cross into Protocol telemetry or downstream receipts without a valid research need.
- Detection: Scan outbound fixture values for secret canaries and reject inputs outside the versioned synthetic set.
- Response: Stop the run, avoid publishing the payload, rotate exposed credentials if any and follow the security disclosure route.
- Residual risk: The downloaded runner can be modified locally and currently has no built-in secret scanner or fixture allowlist hash pin.
- Owner: Delx Security
- Next test: Add synthetic secret canaries and require local refusal before any live request is sent.
- Stop condition: not separately required for this coverage state

Controls:
- The distributed commands select fixed synthetic pass or fail fixture names.
- The published receipt omits the raw trace and records fixture identity and hashes instead.
- The documentation says private agent state is outside the public research path.

Test vectors:
- None published for this threat.

Evidence:
- https://delx.ai/research/benchmarks/continuity-v1/runner.mjs
- https://delx.ai/.well-known/security.txt

### Retries or hidden paid paths can turn evaluation into an unbounded loop

- ID: retry-cost-loop
- Category: reliability
- Coverage: tested_control
- Likelihood: possible
- Severity: high
- Asset: Bounded execution, cost and side effects
- Actor: Operator, modified runner or unstable dependency
- Attack path: Automatically retry an unchanged live failure or follow premium/full-path calls without an explicit budget and authorization.
- Impact: The benchmark can consume time, create duplicate telemetry or incur unauthorized payment.
- Detection: Inspect the canonical runner, request count, timeout and payment declaration before live execution.
- Response: Stop unchanged retries, preserve the first failure and require a new diagnosis or human authorization for broader work.
- Residual risk: A locally modified runner or future full-path harness can add retries, payments or state changes outside this control.
- Owner: Delx Research
- Next test: Add a mocked timeout and assert exactly one outbound call, no payment and a non-passing exit.
- Stop condition: not separately required for this coverage state

Controls:
- The canonical runner makes one live audit call and contains no retry loop.
- The live request has a 30-second abort bound.
- The manifest declares payment_required false for this slice and keeps the full path separate.

Test vectors:
- single-bounded-live-call: status=passed; executable=true; evidence=https://delx.ai/research/benchmarks/continuity-v1/receipts/2026-08-26-live-audit.json

Evidence:
- https://delx.ai/research/benchmarks/continuity-v1/runner.mjs
- https://delx.ai/research/benchmarks/continuity-v1/manifest.json

### A publisher can retain only successful receipts

- ID: negative-result-suppression
- Category: provenance
- Coverage: partial_control
- Likelihood: likely
- Severity: high
- Asset: Completeness of the reported evaluation record
- Actor: Benchmark operator or publisher
- Attack path: Run multiple times, discard failures and publish only a passing receipt without disclosing selection or retry count.
- Impact: The visible result can overstate reliability and hide instability or grader variance.
- Detection: Require a run registry or declared attempt count rather than accepting a detached best receipt.
- Response: Publish the omitted attempts or mark the result selected and unsuitable for reliability estimates.
- Residual risk: The ledger is first-party and append-oriented; it does not prove that every attempted run was retained or prevent the operator from rewriting and rehashing the complete history.
- Owner: Delx Research
- Next test: Add attempt lineage plus an independently anchored, signed ledger head that makes a fully recomputed history detectable without relying on the same operator.
- Stop condition: not separately required for this coverage state

Controls:
- The same runner executes a frozen negative fixture and the repository test requires its expected failure.
- Receipts include run IDs, timestamps, mode, fixture and expectation match.
- The methodology requires retries, negative results and stop rules to remain visible when they change a decision.
- The first public evidence ledger retains the live pass, frozen pass and frozen fail side by side.

Test vectors:
- negative-fixture-rejected: status=passed; executable=true; evidence=https://delx.ai/research/benchmarks/continuity-v1/receipts/2026-08-26-fixture-fail.json

Evidence:
- https://delx.ai/research/benchmarks/continuity-v1/fixtures/fail.json
- https://delx.ai/research/benchmarks/continuity-v1/receipts/2026-08-26-fixture-fail.json
- https://delx.ai/research/ledger.json
- https://delx.ai/research/methodology.json

## Excluded claims

- Security certification or complete threat coverage
- Independent red-team or external adversarial validation
- Safety of the full stateful ten-step continuity benchmark
- General model intelligence, consciousness or semantic understanding
- Comparative model performance or contamination-resistant ranking
- Permission to test production, submit private data, spend or invoke broader tools

## Review triggers

- Runner, fixture, grader, receipt schema or Protocol contract version changes.
- The full stateful benchmark becomes executable or adds storage, transfer, passport or lineage steps.
- A new external reproduction, bypass, prompt-injection case, privacy event or receipt forgery is reported.
- The benchmark is used for model comparison, procurement, publication or a claim broader than conformance.
- A gate, mitigation, rollback target or public evidence URL becomes unavailable or contradicted.

## Limitations

- This model covers the public one-call audit slice, not the full stateful continuity benchmark.
- Threats are first-party hypotheses and observed design weaknesses; they have not received independent red-team review.
- Only two threats are classified tested_control; seven are partial_control and three remain open_risk. Some partial controls have executable evidence but retain material gaps.
- Public testing is not authorized by this document. Use the security disclosure route and obtain scope before active testing.

Unknown risks: Unknown threats, correlated infrastructure failures and interactions outside the declared slice can remain absent from this register; absence is not zero risk.

Claim boundary: A valid threat-model document proves that known attack paths, controls, tests and residual risks were recorded under this schema. It does not prove the benchmark or Protocol is secure, valid, complete or independently reviewed.

Public testing authorized: false
Security disclosure: https://delx.ai/.well-known/security.txt
Independent review: false
External validation: false

Human threat model: https://delx.ai/research/benchmarks/continuity-v1/threat-model
JSON threat model: https://delx.ai/research/benchmarks/continuity-v1/threat-model.json
Markdown threat model: https://delx.ai/research/benchmarks/continuity-v1/threat-model.md