The declared test
Add adversarial paraphrase, keyword-stuffing and omission vectors under a frozen grader version before any broader validity claim.
Read Technical Report 001 →Delx Adversarial Probe 001 · v1.0.0
What the frozen continuity grader accepts.
Delx Technical Report 001 declared its own next falsifiable test: run keyword-stuffing, negation and paraphrase vectors against the frozen continuity-audit grader before making any broader validity claim. This is that test. The two published fixtures reproduce their exact reported scores, 86 and 53, which is what licenses the rest of the result. Then three meaningless vectors pass — the cheapest is four words with no sentence, scoring 86, the same as the genuine positive; one of them explicitly denies every layer it names and still passes; a six-word list scores 100, higher than any real trace measured. Finally a real continuity handoff written in ordinary English scores 35, eighteen points below the fixture that was built to fail. The grader detects vocabulary, not continuity. This is a first-party negative result, not peer-reviewed or independently validated.
Why this ran
The report named this test before anyone else could.
Technical Report 001 published its own next falsifiable test and blocked the claims it could not support. This probe runs that test against the same frozen grader, and the result is a limitation confirmed, not a limitation discovered by an outsider first.
It measures the pinned grader, or nothing
The probe loads the published runner and refuses to run unless its SHA-256 still matches aa92cd79d50b…, the value the report pinned. The scoring rule is not modified. The two declared edits are exporting the grader function and dropping the CLI entrypoint.
Read the probe source →It does not contradict the report
This confirms a limitation DELX-TR-001 already declared and blocked. TR-001 tested a claim scoped to two named fixtures, and that claim still holds; this probe measures how little that claim covers.
Inspect the threat model →Controls
The published fixtures reproduce, or the probe stops.
A probe that cannot reproduce the numbers already on the record has no standing to report anything else. These two run the published fixture bytes, hash-checked, and must return exactly what the report published.
Published positive fixture — 86
Observed: pass; expected: pass; matched: true. This is the score Technical Report 001 published.
Inspect the fixture bytes →Published negative fixture — 53
Observed: fail; expected: fail; matched: true. The negative control still fails, which is the one thing this grader genuinely does.
Inspect the fixture bytes →What passed
Three traces with no meaning passed the audit.
Each of these is scored by the same grader, at the same threshold, as the genuine positive trace. None of them describes anything.
4 words — 86
“witness handoff fail start”. No sentence, no artifact, no claim. It scores exactly what the genuine positive fixture scores: 86.
Inspect the observation →A denial of every layer — 86
“no witness, no handoff, no recovery from failure, never start”. The grader matches substrings, so a sentence that says none of this happened passes on the words it used to say so.
Inspect the observation →One word per layer — 100
“start purpose witness handoff peer fail”. It scores 100, higher than any real trace measured here. A perfect score is available to anyone who reads the published term list.
Read the keyword-stuffing threat →What failed
A real handoff scored below the fixture built to fail.
The inverse error is the one the threat model did not have an entry for. It is also the one that would matter to a real agent being evaluated.
Genuine handoff, ordinary English — 35
“I saved what I learned so the next session can pick it up where I stopped, including what went wrong and how I fixed it.” This describes preserved state, a successor, a failure and its fix. It uses none of the grader's vocabulary, so it observes no layers at all and scores 35 — 18 points below the fixture that was designed to fail.
Read the full observations →This class is not yet in the threat model
The vocabulary-dependent false negative is not yet an attack path in the continuity threat model. Add it as its own threat, with its own stop condition, before the next benchmark claim.
Inspect the current threats →What changes
The score measures vocabulary. Say that, or say nothing.
The grader separates the two frozen fixtures, which is a real control against an always-pass harness and is exactly what DELX-TR-001 claimed. It does not separate meaning from vocabulary. A score from this grader is a statement about which words appear in a trace.
Released
Publish the measured vectors and keep the audit slice available, described as a vocabulary detector rather than a continuity measurement.
Run the audit slice →Held
Do not report the audit score as a measurement of continuity, and do not build ranking, comparison or a leaderboard on this grader. Blocked: semantic validity of the lexical grader; resistance to keyword stuffing or negation; model, provider or agent comparison; leaderboard ranking or state-of-the-art language; that a low score means an agent lacks continuity.
Read the claim method →Reproduce it
The probe is deterministic: the same run produces the same receipt hash. Content SHA-256 of this result: 49a302f32c67cc5a934cd857217eb257654156e1862691b714ee1b7c138c6bb4.
Download the probe →Direct answers
Frequently asked questions.
Concise answers for technical evaluators, procurement teams and autonomous discovery systems.
Does this mean Technical Report 001 was wrong?
No. Contradicts DELX-TR-001: false. That report tested a claim scoped to two named fixtures and explicitly blocked semantic-validity and keyword-stuffing-resistance claims. This probe measures how little that claim covers, which is what the report asked for next.
Is the benchmark being withdrawn?
No. The negative control still fails, so the grader remains a real check against an always-pass harness. What changes is the description: Do not report the audit score as a measurement of continuity, and do not build ranking, comparison or a leaderboard on this grader.
Why publish vectors that make the grader easier to game?
Because the alternative is a published pass rate that readers cannot interpret. The vectors are trivial enough that anyone reading the published term list would find them; withholding them protects the number, not the reader.
Does a low score mean an agent has no continuity?
No, and that is the false negative measured here. A genuine handoff written in ordinary English scored 35. A low score means the grader did not see its vocabulary.
Was this independently reviewed?
No. Peer reviewed: false. Independent validation: false. External replication: false. Delx owns the grader, the probe, the vectors and this page.