With Qwen3-8B greedy decoding on eight LongBench-v1 tasks, four tasks agreed within 0.05 points, the largest task deviation was 0.50 points, and the task-macro average was 51.92 for AutoRegressive versus 51.72 for ASPIRE (−0.20 points) across 1,460 examples. The authors attribute the small differences to floating-point nondeterminism in batched-kernel tie-breaking.

With Qwen3-8B greedy decoding on eight LongBench-v1 tasks, four tasks agreed within 0.05 points, the largest task deviation was 0.50 points, and the task-macro average was 51.92 for AutoRegressive versus 51.72 for ASPIRE (−0.20 points) across 1,460 examples. The authors attribute the small differences to floating-point nondeterminism in batched-kernel tie-breaking. · CiteArk

Measurement results

0/36 assessed

0 supported0 disputed or conflicting36 not assessed

The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
task: HotpotQA · model: Qwen3-8B · decoding: greedy	200 examples	—	Not assessed
task: HotpotQA · model: Qwen3-8B · method: AutoRegressive · decoding: greedy	62.98%	—	Not assessed
task: HotpotQA · model: Qwen3-8B · method: ASPIRE · decoding: greedy	62.95%	—	Not assessed
task: HotpotQA · model: Qwen3-8B · decoding: greedy	-0.03 percentage points	—	Not assessed
task: TriviaQA · model: Qwen3-8B · decoding: greedy	200 examples	—	Not assessed
task: TriviaQA · model: Qwen3-8B · method: AutoRegressive · decoding: greedy	90.01%	—	Not assessed
task: TriviaQA · model: Qwen3-8B · method: ASPIRE · decoding: greedy	90.01%	—	Not assessed
task: TriviaQA · model: Qwen3-8B · decoding: greedy	0 percentage points	—	Not assessed
task: NarrativeQA · model: Qwen3-8B · decoding: greedy	80 examples	—	Not assessed
task: NarrativeQA · model: Qwen3-8B · method: AutoRegressive · decoding: greedy	26.76%	—	Not assessed
task: NarrativeQA · model: Qwen3-8B · method: ASPIRE · decoding: greedy	26.81%	—	Not assessed
task: NarrativeQA · model: Qwen3-8B · decoding: greedy	0.05 percentage points	—	Not assessed

Reported and observed measurements

Assessments (0)

Runs (0)

Measurement results