Loading page…
Explore ArkGraph and select the steps to run.
The paper’s claims are available in Research claims.
0 / 29 claims verified
the rest still being verified
Experiment plan ready; no runs yet.
0/81 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: Qwen3-1.7B · method: ASPIRE · dataset: LB[16k-18k] | 35 requests | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: LB[16k-18k] | 14.78 tokens/verification | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: LB[16k-18k] | 97.2% | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: LB[16k-18k] | 202 tokens/request | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: LB[16k-18k] | 5.93 ms/forward step | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB[16k-18k] | 35 requests | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB[16k-18k] | 5.78 tokens/verification | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB[16k-18k] | 93.4% | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB[16k-18k] | 71 tokens/request | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB[16k-18k] | 6.16 ms/forward step | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB[16k-18k] | 29.21 ms/forward step | — | Not assessed |
model: Qwen3-1.7B · method: Vegas · dataset: LB[16k-18k] | 35 requests | — | Not assessed |
Experiment plan ready; no runs yet.
0/66 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: Qwen3-8B · method: AutoRegressive · dataset: AIME25 | 1,377.5 output token/s | — | Not assessed |
model: Qwen3-8B · method: AutoRegressive · dataset: AIME25 | 1 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · method: AutoRegressive · dataset: CodeElo | 1,336.7 output token/s | — | Not assessed |
model: Qwen3-8B · method: AutoRegressive · dataset: CodeElo | 1 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · method: AutoRegressive · dataset: LB[16k-18k] | 813.1 output token/s | — | Not assessed |
model: Qwen3-8B · method: AutoRegressive · dataset: LB[16k-18k] | 1 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · method: AutoRegressive · dataset: LB-v2[30k-40k] | 357 output token/s | — | Not assessed |
model: Qwen3-8B · method: AutoRegressive · dataset: LB-v2[30k-40k] | 1 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · method: AutoRegressive · dataset: LB-v2[80k-100k] | 135.7 output token/s | — | Not assessed |
model: Qwen3-8B · method: AutoRegressive · dataset: LB-v2[80k-100k] | 1 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · method: AutoRegressive · dataset: all five workloads | 1 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · method: MagicDec · dataset: AIME25 | 1,830.7 output token/s | — | Not assessed |
Experiment plan ready; no runs yet.
0/5 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
refresh: enabled · qualifier: approximately · fixedDraftLength: 1 | 95% | — | Not assessed |
refresh: disabled · qualifier: approximately · fixedDraftLength: 1 | 95% | — | Not assessed |
refresh: disabled · fixedDraftLength: 10 | 60.3% | — | Not assessed |
refresh: enabled · fixedDraftLength: 10 | 76.6% | — | Not assessed |
comparison: refresh minus no refresh · fixedDraftLength: 10 | 16.4 percentage points | — | Not assessed |
Experiment plan ready; no runs yet.
0/36 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: Qwen3-1.7B (N=28) · method: ASPIRE-Fixed · dataset: AIME25 · fixedDraftLength: 5 · refreshLayerPosition: 0 | 61.4% | — | Not assessed |
model: Qwen3-1.7B (N=28) · method: ASPIRE-Fixed · dataset: AIME25 · fixedDraftLength: 5 · refreshLayerPosition: N/4 | 82.1% | — | Not assessed |
model: Qwen3-1.7B (N=28) · method: ASPIRE-Fixed · dataset: AIME25 · fixedDraftLength: 5 · refreshLayerPosition: N/2 | 84.9% | — | Not assessed |
model: Qwen3-1.7B (N=28) · method: ASPIRE-Fixed · dataset: AIME25 · fixedDraftLength: 5 · refreshLayerPosition: 3N/4 | 87.5% | — | Not assessed |
model: Qwen3-1.7B (N=28) · method: ASPIRE-Fixed · dataset: AIME25 · fixedDraftLength: 5 · refreshLayerPosition: N-2 | 85.6% | — | Not assessed |
model: Qwen3-1.7B (N=28) · method: ASPIRE-Fixed · dataset: AIME25 · fixedDraftLength: 5 · refreshLayerPosition: N-1 | 82.6% | — | Not assessed |
model: Qwen3-1.7B (N=28) · method: ASPIRE-Fixed · dataset: LB[16k-18k] · fixedDraftLength: 5 · refreshLayerPosition: 0 | 47.8% | — | Not assessed |
model: Qwen3-1.7B (N=28) · method: ASPIRE-Fixed · dataset: LB[16k-18k] · fixedDraftLength: 5 · refreshLayerPosition: N/4 | 81.5% | — | Not assessed |
model: Qwen3-1.7B (N=28) · method: ASPIRE-Fixed · dataset: LB[16k-18k] · fixedDraftLength: 5 · refreshLayerPosition: N/2 | 92.7% | — | Not assessed |
model: Qwen3-1.7B (N=28) · method: ASPIRE-Fixed · dataset: LB[16k-18k] · fixedDraftLength: 5 · refreshLayerPosition: 3N/4 | 94.7% | — | Not assessed |
model: Qwen3-1.7B (N=28) · method: ASPIRE-Fixed · dataset: LB[16k-18k] · fixedDraftLength: 5 · refreshLayerPosition: N-2 | 93.4% | — | Not assessed |
model: Qwen3-1.7B (N=28) · method: ASPIRE-Fixed · dataset: LB[16k-18k] · fixedDraftLength: 5 · refreshLayerPosition: N-1 | 76.8% | — | Not assessed |
Experiment plan ready; no runs yet.
0/81 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: Qwen3-1.7B · method: ASPIRE · dataset: LB-v2[80k-100k] | 7 requests | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: LB-v2[80k-100k] | 13.89 tokens/verification | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: LB-v2[80k-100k] | 97.3% | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: LB-v2[80k-100k] | 2,056 tokens/request | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: LB-v2[80k-100k] | 5.22 ms/forward step | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB-v2[80k-100k] | 7 requests | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB-v2[80k-100k] | 5.86 tokens/verification | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB-v2[80k-100k] | 97.2% | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB-v2[80k-100k] | 2,050 tokens/request | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB-v2[80k-100k] | 5.95 ms/forward step | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB-v2[80k-100k] | 27.46 ms/forward step | — | Not assessed |
model: Qwen3-1.7B · method: Vegas · dataset: LB-v2[80k-100k] | 7 requests | — | Not assessed |
Experiment plan ready; no runs yet.
0/4 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: Qwen3-8B · contextLengthTokens: 16000 | 1.218 ms/cycle | — | Not assessed |
model: Qwen3-8B · contextLengthTokens: 16000 | 1.03% | — | Not assessed |
model: Qwen3-8B · contextLengthTokens: 32000 | 2.113 ms/cycle | — | Not assessed |
model: Qwen3-8B · contextLengthTokens: 32000 | 1.06% | — | Not assessed |
Experiment plan ready; no runs yet.
0/26 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: Qwen3-1.7B · dataset: AIME25 | 42 requests | — | Not assessed |
model: Qwen3-1.7B · dataset: AIME25 | 2.59 × AutoRegressive | — | Not assessed |
model: Qwen3-1.7B · dataset: AIME25 | 2.06 × AutoRegressive | — | Not assessed |
model: Qwen3-1.7B · dataset: AIME25 | 20.2% | — | Not assessed |
model: Qwen3-1.7B · dataset: LB-v2[30k-40k] | 18 requests | — | Not assessed |
model: Qwen3-1.7B · dataset: LB-v2[30k-40k] | 4.46 × AutoRegressive | — | Not assessed |
model: Qwen3-1.7B · dataset: LB-v2[30k-40k] | 4.17 × AutoRegressive | — | Not assessed |
model: Qwen3-1.7B · dataset: LB-v2[30k-40k] | 6.5% | — | Not assessed |
model: Qwen3-8B · dataset: AIME25 | 30 requests | — | Not assessed |
model: Qwen3-8B · dataset: AIME25 | 1.95 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · dataset: AIME25 | 1.86 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · dataset: AIME25 | 4.7% | — | Not assessed |
Experiment plan ready; no runs yet.
0/12 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: Qwen3-1.7B | 8.24 tokens | — | Not assessed |
model: Qwen3-1.7B | 14.96 tokens | — | Not assessed |
model: Qwen3-1.7B | 11.63 tokens | — | Not assessed |
model: Qwen3-1.7B | 11.6 tokens | — | Not assessed |
model: Qwen3-8B | 9.24 tokens | — | Not assessed |
model: Qwen3-8B | 16.46 tokens | — | Not assessed |
model: Qwen3-8B | 12.03 tokens | — | Not assessed |
model: Qwen3-8B | 12.02 tokens | — | Not assessed |
model: DS-Llama-8B | 8.05 tokens | — | Not assessed |
model: DS-Llama-8B | 17.64 tokens | — | Not assessed |
model: DS-Llama-8B | 12.22 tokens | — | Not assessed |
model: DS-Llama-8B | 12.25 tokens | — | Not assessed |
Experiment plan ready; no runs yet.
0/12 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
pageSizeTokens: 1 | 1,430 output token/s | — | Not assessed |
pageSizeTokens: 1 | 86.8% | — | Not assessed |
pageSizeTokens: 4 | 1,389 output token/s | — | Not assessed |
pageSizeTokens: 4 | 84.7% | — | Not assessed |
pageSizeTokens: 8 | 1,451 output token/s | — | Not assessed |
pageSizeTokens: 8 | 88.3% | — | Not assessed |
pageSizeTokens: 16 | 1,437 output token/s | — | Not assessed |
pageSizeTokens: 16 | 86.8% | — | Not assessed |
pageSizeTokens: 32 | 1,334 output token/s | — | Not assessed |
pageSizeTokens: 32 | 81.2% | — | Not assessed |
pageSizeTokens: 64 | 1,394 output token/s | — | Not assessed |
pageSizeTokens: 64 | 84.1% | — | Not assessed |
Experiment plan ready; no runs yet.
0/81 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: Qwen3-1.7B · method: ASPIRE · dataset: AIME25 | 42 requests | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: AIME25 | 6.01 tokens/verification | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: AIME25 | 80.3% | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: AIME25 | 14,707 tokens/request | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: AIME25 | 4.89 ms/forward step | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: AIME25 | 42 requests | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: AIME25 | 5.27 tokens/verification | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: AIME25 | 85.4% | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: AIME25 | 14,059 tokens/request | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: AIME25 | 5.06 ms/forward step | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: AIME25 | 18.52 ms/forward step | — | Not assessed |
model: Qwen3-1.7B · method: Vegas · dataset: AIME25 | 42 requests | — | Not assessed |
Experiment plan ready; no runs yet.
0/3 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 2.56 tokens | — | Not assessed | |
| 20 tokens | — | Not assessed | |
| 11.51 tokens | — | Not assessed |
Experiment plan ready; no runs yet.
0/36 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
task: HotpotQA · model: Qwen3-8B · decoding: greedy | 200 examples | — | Not assessed |
task: HotpotQA · model: Qwen3-8B · method: AutoRegressive · decoding: greedy | 62.98% | — | Not assessed |
task: HotpotQA · model: Qwen3-8B · method: ASPIRE · decoding: greedy | 62.95% | — | Not assessed |
task: HotpotQA · model: Qwen3-8B · decoding: greedy | -0.03 percentage points | — | Not assessed |
task: TriviaQA · model: Qwen3-8B · decoding: greedy | 200 examples | — | Not assessed |
task: TriviaQA · model: Qwen3-8B · method: AutoRegressive · decoding: greedy | 90.01% | — | Not assessed |
task: TriviaQA · model: Qwen3-8B · method: ASPIRE · decoding: greedy | 90.01% | — | Not assessed |
task: TriviaQA · model: Qwen3-8B · decoding: greedy | 0 percentage points | — | Not assessed |
task: NarrativeQA · model: Qwen3-8B · decoding: greedy | 80 examples | — | Not assessed |
task: NarrativeQA · model: Qwen3-8B · method: AutoRegressive · decoding: greedy | 26.76% | — | Not assessed |
task: NarrativeQA · model: Qwen3-8B · method: ASPIRE · decoding: greedy | 26.81% | — | Not assessed |
task: NarrativeQA · model: Qwen3-8B · decoding: greedy | 0.05 percentage points | — | Not assessed |
Experiment plan ready; no runs yet.
0/66 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: Qwen3-1.7B · method: AutoRegressive · dataset: AIME25 | 2,454.4 output token/s | — | Not assessed |
model: Qwen3-1.7B · method: AutoRegressive · dataset: AIME25 | 1 × AutoRegressive | — | Not assessed |
model: Qwen3-1.7B · method: AutoRegressive · dataset: CodeElo | 2,044 output token/s | — | Not assessed |
model: Qwen3-1.7B · method: AutoRegressive · dataset: CodeElo | 1 × AutoRegressive | — | Not assessed |
model: Qwen3-1.7B · method: AutoRegressive · dataset: LB[16k-18k] | 1,195.1 output token/s | — | Not assessed |
model: Qwen3-1.7B · method: AutoRegressive · dataset: LB[16k-18k] | 1 × AutoRegressive | — | Not assessed |
model: Qwen3-1.7B · method: AutoRegressive · dataset: LB-v2[30k-40k] | 544.7 output token/s | — | Not assessed |
model: Qwen3-1.7B · method: AutoRegressive · dataset: LB-v2[30k-40k] | 1 × AutoRegressive | — | Not assessed |
model: Qwen3-1.7B · method: AutoRegressive · dataset: LB-v2[80k-100k] | 210.5 output token/s | — | Not assessed |
model: Qwen3-1.7B · method: AutoRegressive · dataset: LB-v2[80k-100k] | 1 × AutoRegressive | — | Not assessed |
model: Qwen3-1.7B · method: AutoRegressive · dataset: all five workloads | 1 × AutoRegressive | — | Not assessed |
model: Qwen3-1.7B · method: MagicDec · dataset: AIME25 | 4,726.4 output token/s | — | Not assessed |
Experiment plan ready; no runs yet.
0/81 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: Qwen3-1.7B · method: ASPIRE · dataset: CodeElo | 42 requests | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: CodeElo | 4.82 tokens/verification | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: CodeElo | 76.5% | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: CodeElo | 14,060 tokens/request | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: CodeElo | 5.23 ms/forward step | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: CodeElo | 42 requests | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: CodeElo | 4.88 tokens/verification | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: CodeElo | 77.6% | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: CodeElo | 13,385 tokens/request | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: CodeElo | 5.21 ms/forward step | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: CodeElo | 19.8 ms/forward step | — | Not assessed |
model: Qwen3-1.7B · method: Vegas · dataset: CodeElo | 42 requests | — | Not assessed |
Experiment plan ready; no runs yet.
0/16 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: Qwen3-8B · method: ASPIRE · dataset: AIME25 | 1.86 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · method: ASPIRE-FSM · dataset: AIME25 | 1.61 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · method: ASPIRE · dataset: CodeElo | 1.7 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · method: ASPIRE-FSM · dataset: CodeElo | 1.45 × AutoRegressive | — | Not assessed |
model: DS-Llama-8B · method: ASPIRE · dataset: AIME25 | 1.8 × AutoRegressive | — | Not assessed |
model: DS-Llama-8B · method: ASPIRE-FSM · dataset: AIME25 | 1.6 × AutoRegressive | — | Not assessed |
model: DS-Llama-8B · method: ASPIRE · dataset: CodeElo | 1.8 × AutoRegressive | — | Not assessed |
model: DS-Llama-8B · method: ASPIRE-FSM · dataset: CodeElo | 1.55 × AutoRegressive | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: LB-v2[30k-40k] | 4.17 × AutoRegressive | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-FSM · dataset: LB-v2[30k-40k] | 4.18 × AutoRegressive | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: overall | 3.34 × AutoRegressive | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-FSM · dataset: overall | 3.26 × AutoRegressive | — | Not assessed |
Experiment plan ready; no runs yet.
0/10 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: Qwen3-8B · method: ASPIRE · dataset: LB[16k-18k] | 1.81 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · method: ASPIRE-Fixed · dataset: LB[16k-18k] | 1.73 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · method: ASPIRE · dataset: LB-v2[30k-40k] | 2.3 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · method: ASPIRE-Fixed · dataset: LB-v2[30k-40k] | 1.98 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · method: ASPIRE · dataset: LB-v2[80k-100k] | 2.75 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · method: ASPIRE-Fixed · dataset: LB-v2[80k-100k] | 2.22 × AutoRegressive | — | Not assessed |
model: DS-Llama-8B · method: ASPIRE · dataset: LB-v2[30k-40k] | 2.49 × AutoRegressive | — | Not assessed |
model: DS-Llama-8B · method: ASPIRE-Fixed · dataset: LB-v2[30k-40k] | 2.12 × AutoRegressive | — | Not assessed |
model: DS-Llama-8B · method: ASPIRE · dataset: LB-v2[80k-100k] | 2.66 × AutoRegressive | — | Not assessed |
model: DS-Llama-8B · method: ASPIRE-Fixed · dataset: LB-v2[80k-100k] | 2.21 × AutoRegressive | — | Not assessed |
Experiment plan ready; no runs yet.
0/66 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: DS-Llama-8B · method: AutoRegressive · dataset: AIME25 | 1,520.1 output token/s | — | Not assessed |
model: DS-Llama-8B · method: AutoRegressive · dataset: AIME25 | 1 × AutoRegressive | — | Not assessed |
model: DS-Llama-8B · method: AutoRegressive · dataset: CodeElo | 1,480.9 output token/s | — | Not assessed |
model: DS-Llama-8B · method: AutoRegressive · dataset: CodeElo | 1 × AutoRegressive | — | Not assessed |
model: DS-Llama-8B · method: AutoRegressive · dataset: LB[16k-18k] | 785.3 output token/s | — | Not assessed |
model: DS-Llama-8B · method: AutoRegressive · dataset: LB[16k-18k] | 1 × AutoRegressive | — | Not assessed |
model: DS-Llama-8B · method: AutoRegressive · dataset: LB-v2[30k-40k] | 386.1 output token/s | — | Not assessed |
model: DS-Llama-8B · method: AutoRegressive · dataset: LB-v2[30k-40k] | 1 × AutoRegressive | — | Not assessed |
model: DS-Llama-8B · method: AutoRegressive · dataset: LB-v2[80k-100k] | 150.6 output token/s | — | Not assessed |
model: DS-Llama-8B · method: AutoRegressive · dataset: LB-v2[80k-100k] | 1 × AutoRegressive | — | Not assessed |
model: DS-Llama-8B · method: AutoRegressive · dataset: all five workloads | 1 × AutoRegressive | — | Not assessed |
model: DS-Llama-8B · method: MagicDec · dataset: AIME25 | 1,990.6 output token/s | — | Not assessed |
Experiment plan ready; no runs yet.
0/28 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: Qwen3-8B · dataset: AIME25 · calibration: fitted for this TP degree · tensorParallelism: 1 | 30 requests | — | Not assessed |
model: Qwen3-8B · dataset: AIME25 · calibration: fitted for this TP degree · tensorParallelism: 1 | 1,353 output token/s | — | Not assessed |
model: Qwen3-8B · dataset: AIME25 · calibration: fitted for this TP degree · tensorParallelism: 1 | 2,553 output token/s | — | Not assessed |
model: Qwen3-8B · dataset: AIME25 · calibration: fitted for this TP degree · tensorParallelism: 1 | 1.89 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · dataset: AIME25 · calibration: fitted for this TP degree · tensorParallelism: 2 | 64 requests | — | Not assessed |
model: Qwen3-8B · dataset: AIME25 · calibration: fitted for this TP degree · tensorParallelism: 2 | 2,494 output token/s | — | Not assessed |
model: Qwen3-8B · dataset: AIME25 · calibration: fitted for this TP degree · tensorParallelism: 2 | 5,392 output token/s | — | Not assessed |
model: Qwen3-8B · dataset: AIME25 · calibration: fitted for this TP degree · tensorParallelism: 2 | 2.16 × AutoRegressive | — | Not assessed |
model: Qwen3-8B · dataset: AIME25 · calibration: TP=1 coefficients reused at TP=2 · tensorParallelism: 2 stale | 64 requests | — | Not assessed |
model: Qwen3-8B · dataset: AIME25 · calibration: TP=1 coefficients reused at TP=2 · tensorParallelism: 2 stale | 2,494 output token/s | — | Not assessed |
model: Qwen3-8B · dataset: AIME25 · calibration: TP=1 coefficients reused at TP=2 · tensorParallelism: 2 stale | 5,509 output token/s | — | Not assessed |
model: Qwen3-8B · dataset: AIME25 · calibration: TP=1 coefficients reused at TP=2 · tensorParallelism: 2 stale | 2.21 × AutoRegressive | — | Not assessed |
Experiment plan ready; no runs yet.
0/81 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: Qwen3-1.7B · method: ASPIRE · dataset: LB-v2[30k-40k] | 18 requests | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: LB-v2[30k-40k] | 13.58 tokens/verification | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: LB-v2[30k-40k] | 96.3% | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: LB-v2[30k-40k] | 1,805 tokens/request | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE · dataset: LB-v2[30k-40k] | 5.84 ms/forward step | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB-v2[30k-40k] | 18 requests | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB-v2[30k-40k] | 5.85 tokens/verification | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB-v2[30k-40k] | 97% | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB-v2[30k-40k] | 1,816 tokens/request | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB-v2[30k-40k] | 6.18 ms/forward step | — | Not assessed |
model: Qwen3-1.7B · method: ASPIRE-Fixed · dataset: LB-v2[30k-40k] | 28.2 ms/forward step | — | Not assessed |
model: Qwen3-1.7B · method: Vegas · dataset: LB-v2[30k-40k] | 17 requests | — | Not assessed |
Experiment plan ready; no runs yet.
0/10 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
parameterValue: 0.4 · sweptParameter: smoothing weight ω | 1,650 output token/s | — | Not assessed |
parameterValue: 0.4 · sweptParameter: acceptance prior α₀ | 1,429 output token/s | — | Not assessed |
parameterValue: 0.6 · sweptParameter: smoothing weight ω | 1,599 output token/s | — | Not assessed |
parameterValue: 0.6 · sweptParameter: acceptance prior α₀ | 1,419 output token/s | — | Not assessed |
parameterValue: 0.8 · sweptParameter: smoothing weight ω | 1,501 output token/s | — | Not assessed |
parameterValue: 0.8 · sweptParameter: acceptance prior α₀ | 1,582 output token/s | — | Not assessed |
parameterValue: 0.9 · sweptParameter: smoothing weight ω | 1,731 output token/s | — | Not assessed |
parameterValue: 0.9 · sweptParameter: acceptance prior α₀ | 1,501 output token/s | — | Not assessed |
parameterValue: 0.95 · sweptParameter: smoothing weight ω | 1,628 output token/s | — | Not assessed |
parameterValue: 0.95 · sweptParameter: acceptance prior α₀ | 1,598 output token/s | — | Not assessed |
Experiment plan ready; no runs yet.
0/3 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
| 1.7 × AutoRegressive | — | Not assessed | |
| 4.58 × AutoRegressive | — | Not assessed | |
| 27% | — | Not assessed |
The refresh-position package carries this source context and may retain recall traces, but the existing deterministic binding cannot express the paper's qualitative bar ordering without inventing coordinates or a threshold.
The LongBench variability package retains the full round matrix for qualitative review. No scalar target is fabricated because the source defines no threshold for substantial fluctuation.
The ranking is a deterministic relation over the represented Table 1 throughput packages.
This is method context, not an independent empirical result; implementation fidelity is exercised by every ASPIRE package.
This claim defines shared experimental conditions rather than an independent result; those conditions are bound into the relevant packages.
This qualitative deployment limitation is tested by the represented fitted-versus-stale TP experiment and has no separate reported scalar.
This is a scope limitation carried into every performance objective: prefill is excluded and no end-to-end result is inferred.
This is an explicit future-work limitation, not a reported implemented result.