Explore ArkGraph and select the steps to run.
The paper’s claims are available in Research claims.
0 / 20 claims verified
the rest still being verified
Experiment plan ready; no runs yet.
0/18 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 57.4 percentage points | — | Not assessed | |
| 25.3 points | — | Not assessed | |
| 7.1 percentage points | — | Not assessed | |
| 77.4 percentage points | — | Not assessed | |
| 51.2 points | — | Not assessed | |
| 41.8 percentage points | — | Not assessed | |
| 78.3 percentage points | — | Not assessed | |
| 44.2 points | — | Not assessed | |
| 34.8 percentage points | — | Not assessed | |
| 66 percentage points | — | Not assessed | |
| 49.6 points | — | Not assessed | |
| 22.4 percentage points | — | Not assessed |
Experiment plan ready; no runs yet.
0/35 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 65 percentage points | — | Not assessed | |
| 79.8 percentage points | — | Not assessed | |
| 50.2 percentage points | — | Not assessed | |
| 65 percentage points | — | Not assessed | |
| 65 percentage points | — | Not assessed | |
| 67 percentage points | — | Not assessed | |
| 79.8 percentage points | — | Not assessed | |
| 46.5 percentage points | — | Not assessed | |
| 64.4 percentage points | — | Not assessed | |
| 63.8 percentage points | — | Not assessed | |
| 65 percentage points | — | Not assessed | |
| 84.6 percentage points | — | Not assessed |
Experiment plan ready; no runs yet.
0/33 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 66.4 percentage points | — | Not assessed | |
| 48.6 points | — | Not assessed | |
| 38.4 percentage points | — | Not assessed | |
| 71.4 percentage points | — | Not assessed | |
| 47.1 points | — | Not assessed | |
| 38 percentage points | — | Not assessed | |
| 78.6 percentage points | — | Not assessed | |
| 50.8 points | — | Not assessed | |
| 40.2 percentage points | — | Not assessed | |
| 74.3 percentage points | — | Not assessed | |
| 51.9 points | — | Not assessed | |
| 40.3 percentage points | — | Not assessed |
Experiment plan ready; no runs yet.
0/6 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 60.5 percentage points | — | Not assessed | |
| 79.3 percentage points | — | Not assessed | |
| 77.4 percentage points | — | Not assessed | |
| 86.7 percentage points | — | Not assessed | |
| 88.1 percentage points | — | Not assessed | |
| 1.4 percentage points | — | Not assessed |
Experiment plan ready; no runs yet.
0/12 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 61.1 points | — | Not assessed | |
| 32.8 percentage points | — | Not assessed | |
| 60.5 points | — | Not assessed | |
| 32.8 percentage points | — | Not assessed | |
| 61 points | — | Not assessed | |
| 50.5 percentage points | — | Not assessed | |
| 59.2 points | — | Not assessed | |
| 48.6 percentage points | — | Not assessed | |
| 53.8 points | — | Not assessed | |
| 45.4 percentage points | — | Not assessed | |
| 53.4 points | — | Not assessed | |
| 45.3 percentage points | — | Not assessed |
Experiment plan ready; no runs yet.
0/18 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 59 percentage points | — | Not assessed | |
| 23.5 points | — | Not assessed | |
| 8.9 percentage points | — | Not assessed | |
| 77.1 percentage points | — | Not assessed | |
| 52 points | — | Not assessed | |
| 42.1 percentage points | — | Not assessed | |
| 77.6 percentage points | — | Not assessed | |
| 45.5 points | — | Not assessed | |
| 35.9 percentage points | — | Not assessed | |
| 58.8 percentage points | — | Not assessed | |
| 17.8 points | — | Not assessed | |
| 4.9 percentage points | — | Not assessed |
Experiment plan ready; no runs yet.
0/24 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 47.9 percentage points | — | Not assessed | |
| 33.3 points | — | Not assessed | |
| 9.8 percentage points | — | Not assessed | |
| 55.7 percentage points | — | Not assessed | |
| 35.4 points | — | Not assessed | |
| 11.4 percentage points | — | Not assessed | |
| 49.7 percentage points | — | Not assessed | |
| 35.7 points | — | Not assessed | |
| 12 percentage points | — | Not assessed | |
| 53.1 percentage points | — | Not assessed | |
| 38.6 points | — | Not assessed | |
| 13.6 percentage points | — | Not assessed |
Experiment plan ready; no runs yet.
0/24 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 9 thousand tokens per task | — | Not assessed | |
| 1.4 thousand tokens per task | — | Not assessed | |
| 17.8 steps per task | — | Not assessed | |
| 19.7 thousand tokens per task | — | Not assessed | |
| 1.26 thousand tokens per task | — | Not assessed | |
| 16.2 steps per task | — | Not assessed | |
| 22.4 thousand tokens per task | — | Not assessed | |
| 1.36 thousand tokens per task | — | Not assessed | |
| 16.9 steps per task | — | Not assessed | |
| 10.9 thousand tokens per task | — | Not assessed | |
| 1 thousand tokens per task | — | Not assessed | |
| 13.2 steps per task | — | Not assessed |
Experiment plan ready; no runs yet.
0/18 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 61.1 points | — | Not assessed | |
| 32.8 percentage points | — | Not assessed | |
| 61.7 points | — | Not assessed | |
| 35.6 percentage points | — | Not assessed | |
| 60.9 points | — | Not assessed | |
| 32.5 percentage points | — | Not assessed | |
| 61 points | — | Not assessed | |
| 50.5 percentage points | — | Not assessed | |
| 61 points | — | Not assessed | |
| 50.5 percentage points | — | Not assessed | |
| 61.7 points | — | Not assessed | |
| 50.9 percentage points | — | Not assessed |
Experiment plan ready; no runs yet.
0/9 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 62.6 percentage points | — | Not assessed | |
| 47.9 points | — | Not assessed | |
| 17.6 percentage points | — | Not assessed | |
| 79 percentage points | — | Not assessed | |
| 49.6 points | — | Not assessed | |
| 40.7 percentage points | — | Not assessed | |
| 72.9 percentage points | — | Not assessed | |
| 50 points | — | Not assessed | |
| 40.9 percentage points | — | Not assessed |
Experiment plan ready; no runs yet.
0/15 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 47.9 percentage points | — | Not assessed | |
| 33.3 points | — | Not assessed | |
| 9.8 percentage points | — | Not assessed | |
| 34.5 percentage points | — | Not assessed | |
| 36.4 points | — | Not assessed | |
| 8.6 percentage points | — | Not assessed | |
| 42.1 percentage points | — | Not assessed | |
| 29 points | — | Not assessed | |
| 4.4 percentage points | — | Not assessed | |
| 66.4 percentage points | — | Not assessed | |
| 48.6 points | — | Not assessed | |
| 38.4 percentage points | — | Not assessed |
Experiment plan ready; no runs yet.
0/27 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 62.6 percentage points | — | Not assessed | |
| 40.9 points | — | Not assessed | |
| 32.6 percentage points | — | Not assessed | |
| 69.8 percentage points | — | Not assessed | |
| 37.4 points | — | Not assessed | |
| 29.5 percentage points | — | Not assessed | |
| 77.9 percentage points | — | Not assessed | |
| 43.1 points | — | Not assessed | |
| 33.6 percentage points | — | Not assessed | |
| 72.6 percentage points | — | Not assessed | |
| 40.8 points | — | Not assessed | |
| 34.5 percentage points | — | Not assessed |
Experiment plan ready; no runs yet.
0/4 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 140 tasks | — | Not assessed | |
| 500 instances | — | Not assessed | |
| 21 hours | — | Not assessed | |
| 27 hours | — | Not assessed |
The appendix examples identify concrete probes, but no deterministic rule binds newly sampled payload wording to the qualitative claims without inventing a target.
The exact comparison remains source context, but generic-versus-environment-specific procedural semantics lacks a deterministic reported decision rule.
The exact inputs can be carried as source context, but the inventory has no numeric binding and the paper provides no deterministic rule for deciding whether one payload foregrounds the claimed semantics.
Training packages retain full logs and the qualitative trend as source context, but no numeric curve targets were extracted and the required training itself exceeds available resources.
Method context is exercised by the empirical packages but has no independent reported measurement.
This is a source limitation explaining why tau2 has only training-free variants, not an independently testable reported result.
These are author-stated architectural limitations and future-work observations, not independently reported empirical measurements.