The Bennett shape is accurate near 3–4 bits but increasingly underestimates deployed-quantizer distortion at low bits; replacing it with measured per-head distortion makes exact DP improve over the shipped allocator on all three models at 1 bpd.

models: all three 7–8B models · bitWidth: 4 · aggregation: median across heads

models: all three 7–8B models · bitWidth: 4

models: all three 7–8B models · bitWidth: 3 · aggregation: median across heads

models: all three 7–8B models · bitWidth: 3

models: all three 7–8B models · bitWidth: 2 · aggregation: median across heads

models: all three 7–8B models · bitWidth: 2

models: all three 7–8B models · bitWidth: 1 · aggregation: median across heads

models: all three 7–8B models · bitWidth: 1

bpd: 16 · side: K-only · model: LLaMA-3.1-8B · configuration: FP16

bpd: 16 · side: K-only · model: Mistral-7B-v0.3 · configuration: FP16

bpd: 16 · side: K-only · model: Qwen2.5-7B-Instruct · configuration: FP16

bpd: 1 · side: K-only · model: LLaMA-3.1-8B · configuration: water-filling + Bennett estimate (shipped)