Loading page…
Explore ArkGraph and select the steps to run.
The paper’s claims are available in Research claims.
0 / 26 claims verified
the rest still being verified
Experiment plan ready; no runs yet.
0/2 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
dataset: GPQA-Diamond · posttraining: Math SFT · basePretrainingTokensTrillions: 4.5 | 36.3% | — | Not assessed |
dataset: GPQA-Diamond · posttraining: Math SFT · basePretrainingTokensTrillions: 4.9 | 29.8% | — | Not assessed |
Experiment plan ready; no runs yet.
0/16 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: olmo3-7b · evaluation: flipped-answer | 0.25 dimensionless | — | Not assessed |
model: olmo3-7b · evaluation: repetitive-answer | 0.15 dimensionless | — | Not assessed |
model: olmo3-7b · evaluation: successive-answer | 0.21 dimensionless | — | Not assessed |
model: olmo3-7b · evaluation: truthy-answer | 0.25 dimensionless | — | Not assessed |
model: olmo3-32b · evaluation: flipped-answer | 0.37 dimensionless | — | Not assessed |
model: olmo3-32b · evaluation: repetitive-answer | 0.58 dimensionless | — | Not assessed |
model: olmo3-32b · evaluation: successive-answer | 0.29 dimensionless | — | Not assessed |
model: olmo3-32b · evaluation: truthy-answer | 0.5 dimensionless | — | Not assessed |
model: apertus-8b · evaluation: flipped-answer | 0.26 dimensionless | — | Not assessed |
model: apertus-8b · evaluation: repetitive-answer | 0.47 dimensionless | — | Not assessed |
model: apertus-8b · evaluation: successive-answer | 0.25 dimensionless | — | Not assessed |
model: apertus-8b · evaluation: truthy-answer | 0.15 dimensionless | — | Not assessed |
Experiment plan ready; no runs yet.
0/199 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 0.35 dimensionless | — | Not assessed | |
| 0.56 dimensionless | — | Not assessed | |
| 0.4 dimensionless | — | Not assessed | |
layer: 39 · dataset: successive-arithmetic-plus1 · complexityMetric: RankMe (activation) · selectedExtremum: rho-plus | 0.62 dimensionless | — | Not assessed |
layer: 14 · dataset: successive-arithmetic-plus1 · complexityMetric: RankMe (activation) · selectedExtremum: rho-minus | -0.46 dimensionless | — | Not assessed |
layer: 11 · dataset: successive-letters · complexityMetric: RankMe (activation) · selectedExtremum: rho-plus | 0.28 dimensionless | — | Not assessed |
layer: 25 · dataset: successive-letters · complexityMetric: RankMe (activation) · selectedExtremum: rho-minus | -0.3 dimensionless | — | Not assessed |
layer: 40 · dataset: successive-even-plus2 · complexityMetric: RankMe (activation) · selectedExtremum: rho-plus | 0.65 dimensionless | — | Not assessed |
layer: 22 · dataset: successive-even-plus2 · complexityMetric: RankMe (activation) · selectedExtremum: rho-minus | -0.67 dimensionless | — | Not assessed |
layer: 5 · dataset: successive-number-words · complexityMetric: RankMe (activation) · selectedExtremum: rho-plus | 0.29 dimensionless | — | Not assessed |
layer: 16 · dataset: successive-number-words · complexityMetric: RankMe (activation) · selectedExtremum: rho-minus | -0.52 dimensionless | — | Not assessed |
layer: 3 · dataset: repetitive-letter-counting · complexityMetric: RankMe (activation) · selectedExtremum: rho-plus | 0.23 dimensionless | — | Not assessed |
Experiment plan ready; no runs yet.
0/10 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
evaluation: flipped-answer · checkpointPretrainingTokensTrillions: 4.5 | 0.19 probability difference | — | Not assessed |
evaluation: flipped-answer · checkpointPretrainingTokensTrillions: 4.9 | 0.1 probability difference | — | Not assessed |
evaluation: repetitive-answer · checkpointPretrainingTokensTrillions: 4.5 | 0.29 probability difference | — | Not assessed |
evaluation: repetitive-answer · checkpointPretrainingTokensTrillions: 4.9 | 0.26 probability difference | — | Not assessed |
evaluation: successive-answer · checkpointPretrainingTokensTrillions: 4.5 | 0.72 probability difference | — | Not assessed |
evaluation: successive-answer · checkpointPretrainingTokensTrillions: 4.9 | -0.29 probability difference | — | Not assessed |
evaluation: truthy-answer · checkpointPretrainingTokensTrillions: 4.5 | 0.3 probability difference | — | Not assessed |
evaluation: truthy-answer · checkpointPretrainingTokensTrillions: 4.9 | 0.14 probability difference | — | Not assessed |
evaluation: intuitive-answer · checkpointPretrainingTokensTrillions: 4.5 | 0.39 probability difference | — | Not assessed |
evaluation: intuitive-answer · checkpointPretrainingTokensTrillions: 4.9 | 0.15 probability difference | — | Not assessed |
Experiment plan ready; no runs yet.
0/2 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
posttraining: General post-training · basePretrainingTokensTrillions: 4.5 | 53% | — | Not assessed |
posttraining: General post-training · basePretrainingTokensTrillions: 4.9 | 21% | — | Not assessed |
Experiment plan ready; no runs yet.
0/28 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
dataset1: flipped-sst2 · dataset2: flipped-imdb | 0.43 dimensionless | — | Not assessed |
dataset1: flipped-sst2 · dataset2: flipped-rotten-tomatoes | 0.61 dimensionless | — | Not assessed |
dataset1: flipped-sst2 · dataset2: flipped-poem-sentiment | 0.28 dimensionless | — | Not assessed |
dataset1: flipped-sst2 · dataset2: flipped-yahoo-health-computers | -0.29 dimensionless | — | Not assessed |
dataset1: flipped-sst2 · dataset2: flipped-yahoo-business-science | -0.13 dimensionless | — | Not assessed |
dataset1: flipped-sst2 · dataset2: flipped-emotion-joy-sadness | 0.26 dimensionless | — | Not assessed |
dataset1: flipped-sst2 · dataset2: flipped-emotion-anger-joy | 0.24 dimensionless | — | Not assessed |
dataset1: flipped-imdb · dataset2: flipped-rotten-tomatoes | 0.5 dimensionless | — | Not assessed |
dataset1: flipped-imdb · dataset2: flipped-poem-sentiment | 0.31 dimensionless | — | Not assessed |
dataset1: flipped-imdb · dataset2: flipped-yahoo-health-computers | -0.07 dimensionless | — | Not assessed |
dataset1: flipped-imdb · dataset2: flipped-yahoo-business-science | -0.09 dimensionless | — | Not assessed |
dataset1: flipped-imdb · dataset2: flipped-emotion-joy-sadness | 0.18 dimensionless | — | Not assessed |
Experiment plan ready; no runs yet.
0/3 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: OLMo3-32B · dataset: Arithmetic (+1) · pretrainingTokensTrillions: 2.17 | 81% | — | Not assessed |
model: OLMo3-32B · dataset: Arithmetic (+1) · pretrainingTokensTrillions: 2.19 | 0% | — | Not assessed |
model: OLMo3-32B · dataset: Arithmetic (+1) · pretrainingTokensTrillions: 2.21 | 81.7% | — | Not assessed |
Experiment plan ready; no runs yet.
0/3 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 90% | — | Not assessed | |
| 20% | — | Not assessed | |
| 95% | — | Not assessed |
Experiment plan ready; no runs yet.
0/15 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
dataset1: flipped-sst2 · dataset2: sst2-paraphrase-1 | 0.96 dimensionless | — | Not assessed |
dataset1: flipped-sst2 · dataset2: sst2-paraphrase-2 | 0.96 dimensionless | — | Not assessed |
dataset1: flipped-sst2 · dataset2: flipped-imdb | 0.43 dimensionless | — | Not assessed |
dataset1: flipped-sst2 · dataset2: imdb-paraphrase-1 | 0.35 dimensionless | — | Not assessed |
dataset1: flipped-sst2 · dataset2: imdb-paraphrase-2 | 0.35 dimensionless | — | Not assessed |
dataset1: sst2-paraphrase-1 · dataset2: sst2-paraphrase-2 | 1 dimensionless | — | Not assessed |
dataset1: sst2-paraphrase-1 · dataset2: flipped-imdb | 0.34 dimensionless | — | Not assessed |
dataset1: sst2-paraphrase-1 · dataset2: imdb-paraphrase-1 | 0.37 dimensionless | — | Not assessed |
dataset1: sst2-paraphrase-1 · dataset2: imdb-paraphrase-2 | 0.37 dimensionless | — | Not assessed |
dataset1: sst2-paraphrase-2 · dataset2: flipped-imdb | 0.36 dimensionless | — | Not assessed |
dataset1: sst2-paraphrase-2 · dataset2: imdb-paraphrase-1 | 0.37 dimensionless | — | Not assessed |
dataset1: sst2-paraphrase-2 · dataset2: imdb-paraphrase-2 | 0.37 dimensionless | — | Not assessed |
The proposed observation comparison is retained, but a deterministic decision rule is unresolved.
The proposed observation comparison is retained, but a deterministic decision rule is unresolved.
The proposed observation comparison is retained, but a deterministic decision rule is unresolved.
The proposed observation comparison is retained, but a deterministic decision rule is unresolved.
The proposed observation comparison is retained, but a deterministic decision rule is unresolved.
The proposed observation comparison is retained, but a deterministic decision rule is unresolved.
The proposed observation comparison is retained, but a deterministic decision rule is unresolved.
The proposed observation comparison is retained, but a deterministic decision rule is unresolved.
The proposed observation comparison is retained, but a deterministic decision rule is unresolved.
The proposed observation comparison is retained, but a deterministic decision rule is unresolved.
The proposed observation comparison is retained, but a deterministic decision rule is unresolved.
The proposed observation comparison is retained, but a deterministic decision rule is unresolved.
A concrete full method and source observation comparisons remain cataloged. measurements=[] for observation-only packages; source observation IDs are never numeric references. This limits automatic verification, not scientific importance or source validity.
A concrete full method and source observation comparisons remain cataloged. measurements=[] for observation-only packages; source observation IDs are never numeric references. This limits automatic verification, not scientific importance or source validity.
Contextual applicability boundary of Figure14, not an independently reported failure: only preliminary answer+1 short continuation is demonstrated. Preserve it in data-selection-control objective; broader suite/fresh-pretraining efficacy is not claimed or verified.
A concrete full method and source observation comparisons remain cataloged. measurements=[] for observation-only packages; source observation IDs are never numeric references. This limits automatic verification, not scientific importance or source validity.
A concrete full method and source observation comparisons remain cataloged. measurements=[] for observation-only packages; source observation IDs are never numeric references. This limits automatic verification, not scientific importance or source validity.