On RULER-4k retrieval, attention-KL reordering outperformed the MSE ordering at 0.5 and 1 bpd for every 7–8B model, with its largest objective separation among the paper's benchmarks; at 2 bpd it came within 0.7–3.6 points of FP16.

Reported and observed measurements

Assessments (0)

No immutable Assessment has been published for this Claim yet.

Runs (0)

No execution has been linked to this Claim yet.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
bpd: 0.5 · side: K-only · model: LLaMA-3.1-8B · objective: MSE · contextLength: 4096	20.9%	—	Not assessed
bpd: 0.5 · side: K-only · model: LLaMA-3.1-8B · objective: KL · contextLength: 4096	44.3%	—	Not assessed
bpd: 1 · side: K-only · model: LLaMA-3.1-8B · objective: MSE · contextLength: 4096	67.1%	—	Not assessed
bpd: 1 · side: K-only · model: LLaMA-3.1-8B · objective: KL · contextLength: 4096	76%	—	Not assessed
bpd: 2 · side: K-only · model: LLaMA-3.1-8B · objective: KL · contextLength: 4096	92.5%	—	Not assessed
model: LLaMA-3.1-8B · configuration: FP16 · contextLength: 4096	93.5%	—	Not assessed
bpd: 0.5 · side: K-only · model: Mistral-7B-v0.3 · objective: MSE · contextLength: 4096	36.7%	—	Not assessed
bpd: 0.5 · side: K-only · model: Mistral-7B-v0.3 · objective: KL · contextLength: 4096	52%	—	Not assessed
bpd: 1 · side: K-only · model: Mistral-7B-v0.3 · objective: MSE · contextLength: 4096	69.1%	—	Not assessed
bpd: 1 · side: K-only · model: Mistral-7B-v0.3 · objective: KL · contextLength: 4096	79.3%	—	Not assessed
bpd: 2 · side: K-only · model: Mistral-7B-v0.3 · objective: KL · contextLength: 4096	90.6%	—	Not assessed
model: Mistral-7B-v0.3 · configuration: FP16 · contextLength: 4096	91.3%	—	Not assessed

Metric and conditions

Reported

Latest observation

Evidence judgment

bpd: 0.5 · side: K-only · model: LLaMA-3.1-8B · objective: MSE · contextLength: 4096

20.9%

—

Not assessed

bpd: 0.5 · side: K-only · model: LLaMA-3.1-8B · objective: KL · contextLength: 4096

44.3%

—

Not assessed

bpd: 1 · side: K-only · model: LLaMA-3.1-8B · objective: MSE · contextLength: 4096

67.1%

—

Not assessed

bpd: 1 · side: K-only · model: LLaMA-3.1-8B · objective: KL · contextLength: 4096

76%

—

Not assessed

bpd: 2 · side: K-only · model: LLaMA-3.1-8B · objective: KL · contextLength: 4096

92.5%

—

Not assessed

model: LLaMA-3.1-8B · configuration: FP16 · contextLength: 4096

93.5%

—

Not assessed

bpd: 0.5 · side: K-only · model: Mistral-7B-v0.3 · objective: MSE · contextLength: 4096

36.7%

—

Not assessed

bpd: 0.5 · side: K-only · model: Mistral-7B-v0.3 · objective: KL · contextLength: 4096

52%

—

Not assessed

bpd: 1 · side: K-only · model: Mistral-7B-v0.3 · objective: MSE · contextLength: 4096

69.1%

—

Not assessed

bpd: 1 · side: K-only · model: Mistral-7B-v0.3 · objective: KL · contextLength: 4096

79.3%

—

Not assessed

bpd: 2 · side: K-only · model: Mistral-7B-v0.3 · objective: KL · contextLength: 4096

90.6%

—

Not assessed

model: Mistral-7B-v0.3 · configuration: FP16 · contextLength: 4096

91.3%

—

Not assessed

Reported and observed measurements

Assessments (0)

Runs (0)

Measurement results