Retrieval used as the task record failed almost every episode through omissions; verbatim top-k recall added to the full transcript helped but stayed below directives, Mem0 augmentation did not help, and an unverified agent-written ledger outperformed the always-accurate displayed checklist while still leaking re-executions.

Reported and observed measurements

Assessments (0)

No immutable Assessment has been published for this Claim yet.

Runs (0)

No execution has been linked to this Claim yet.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
arm: raw transcript · brief: amended · model: Qwen3-235B · steps: 10 · density: 0.15 · reasoning: no-think	0.38 proportion	—	Not assessed
arm: raw transcript · brief: amended · model: Qwen3-235B · steps: 10 · density: 0.15 · reasoning: no-think	0.5 proportion	—	Not assessed
arm: raw transcript · brief: amended · model: Qwen3-235B · steps: 10 · density: 0.15 · reasoning: no-think	35 events per 128 episodes	—	Not assessed
arm: raw transcript · brief: amended · model: Qwen3-235B · steps: 15 · density: 0.3 · reasoning: no-think	0.5 proportion	—	Not assessed
arm: raw transcript · brief: amended · model: Qwen3-235B · steps: 15 · density: 0.3 · reasoning: no-think	0.66 proportion	—	Not assessed
arm: raw transcript · brief: amended · model: Qwen3-235B · steps: 15 · density: 0.3 · reasoning: no-think	24 events per 128 episodes	—	Not assessed
arm: top-k retrieval, replace · brief: amended · model: Qwen3-235B · steps: 10 · density: 0.15 · reasoning: no-think	0 proportion	—	Not assessed
arm: top-k retrieval, replace · brief: amended · model: Qwen3-235B · steps: 10 · density: 0.15 · reasoning: no-think	0 proportion	—	Not assessed
arm: top-k retrieval, replace · brief: amended · model: Qwen3-235B · steps: 10 · density: 0.15 · reasoning: no-think	157 events per 128 episodes	—	Not assessed
arm: top-k retrieval, replace · brief: amended · model: Qwen3-235B · steps: 10 · density: 0.15 · reasoning: no-think	0 p-value	—	Not assessed
arm: top-k retrieval, replace · brief: amended · model: Qwen3-235B · steps: 15 · density: 0.3 · reasoning: no-think	0.02 proportion	—	Not assessed
arm: top-k retrieval, replace · brief: amended · model: Qwen3-235B · steps: 15 · density: 0.3 · reasoning: no-think	0.02 proportion	—	Not assessed

Metric and conditions

Reported

Latest observation

Evidence judgment

arm: raw transcript · brief: amended · model: Qwen3-235B · steps: 10 · density: 0.15 · reasoning: no-think

0.38 proportion

—

Not assessed

arm: raw transcript · brief: amended · model: Qwen3-235B · steps: 10 · density: 0.15 · reasoning: no-think

0.5 proportion

—

Not assessed

arm: raw transcript · brief: amended · model: Qwen3-235B · steps: 10 · density: 0.15 · reasoning: no-think

35 events per 128 episodes

—

Not assessed

arm: raw transcript · brief: amended · model: Qwen3-235B · steps: 15 · density: 0.3 · reasoning: no-think

0.5 proportion

—

Not assessed

arm: raw transcript · brief: amended · model: Qwen3-235B · steps: 15 · density: 0.3 · reasoning: no-think

0.66 proportion

—

Not assessed

arm: raw transcript · brief: amended · model: Qwen3-235B · steps: 15 · density: 0.3 · reasoning: no-think

24 events per 128 episodes

—

Not assessed

arm: top-k retrieval, replace · brief: amended · model: Qwen3-235B · steps: 10 · density: 0.15 · reasoning: no-think

0 proportion

—

Not assessed

arm: top-k retrieval, replace · brief: amended · model: Qwen3-235B · steps: 10 · density: 0.15 · reasoning: no-think

0 proportion

—

Not assessed

arm: top-k retrieval, replace · brief: amended · model: Qwen3-235B · steps: 10 · density: 0.15 · reasoning: no-think

157 events per 128 episodes

—

Not assessed

arm: top-k retrieval, replace · brief: amended · model: Qwen3-235B · steps: 10 · density: 0.15 · reasoning: no-think

0 p-value

—

Not assessed

arm: top-k retrieval, replace · brief: amended · model: Qwen3-235B · steps: 15 · density: 0.3 · reasoning: no-think

0.02 proportion

—

Not assessed

arm: top-k retrieval, replace · brief: amended · model: Qwen3-235B · steps: 15 · density: 0.3 · reasoning: no-think

0.02 proportion

—

Not assessed

Reported and observed measurements

Assessments (0)

Runs (0)

Measurement results