Generalization behavior is locally stable under a single Adam update on sampled OLMo3 data: suite probability changes are negligible across the tested checkpoints, including a learning rate of 10^-2 and a batch of 8K tokens. This analysis argues against explaining mode-hopping solely by one-step edge-of-stability oscillations. · CiteArk
Not assessedPlan blockedFindinggeneralization-locally-stable
Generalization behavior is locally stable under a single Adam update on sampled OLMo3 data: suite probability changes are negligible across the tested checkpoints, including a learning rate of 10^-2 and a batch of 8K tokens. This analysis argues against explaining mode-hopping solely by one-step edge-of-stability oscillations.
Source: paper:§4.3, Figure 10, p.7
Reported and observed measurements
No structured measurement is attached to this Claim.
Assessments (0)
No immutable Assessment has been published for this Claim yet.
