Every tested compressor produced lower perplexity when keys were compressed pre-RoPE rather than post-RoPE at the canonical bit rates; the paper attributes this to position-stationary pre-RoPE covariance and the loss of a useful shared low-rank basis after position-dependent rotations. · CiteArk
Every tested compressor produced lower perplexity when keys were compressed pre-RoPE rather than post-RoPE at the canonical bit rates; the paper attributes this to position-stationary pre-RoPE covariance and the loss of a useful shared low-rank basis after position-dependent rotations.
Source: paper:PDF p. 22; Appendix G, Figure A6
Reported and observed measurements
No structured measurement is attached to this Claim.
Assessments (0)
No immutable Assessment has been published for this Claim yet.
