Source: paper-fixed:PDF p. 16, Appendix C.2 and Table 9
No immutable Assessment has been published for this Claim yet.
No execution has been linked to this Claim yet.
0/37 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
model: SPG · dataset: MATH-500 · generationLengthTokens: 128 | 34.6% | — | Not assessed |
model: SPG · dataset: MATH-500 · generationLengthTokens: 256 | 39.6% | — | Not assessed |
model: SPG · dataset: MATH-500 · generationLengthTokens: 512 | 42.6% | — | Not assessed |
model: SPG · dataset: Countdown · generationLengthTokens: 128 | 68.8% | — | Not assessed |
model: SPG · dataset: Countdown · generationLengthTokens: 256 | 70.7% | — | Not assessed |
model: SPG · dataset: Countdown · generationLengthTokens: 512 | 70.3% | — | Not assessed |
model: SPG+IM · dataset: MATH-500 · generationLengthTokens: 128 | 34.6% | — | Not assessed |
model: SPG+IM · dataset: MATH-500 · generationLengthTokens: 256 | 40.4% | — | Not assessed |
model: SPG+IM · dataset: MATH-500 · generationLengthTokens: 512 | 45.2% | — | Not assessed |
model: SPG+IM · dataset: Countdown · generationLengthTokens: 128 | 81.6% | — | Not assessed |
model: SPG+IM · dataset: Countdown · generationLengthTokens: 256 | 81.6% | — | Not assessed |
model: SPG+IM · dataset: Countdown · generationLengthTokens: 512 | 80.9% | — | Not assessed |