With repeated demonstration answers, models fluctuate between copying the repeated pattern and solving the task. OLMo3-7B has a strong tendency toward pattern matching, including letter-counting accuracy near zero throughout pretraining; OLMo3-32B generalizes more often but can suddenly fail. Apertus also exhibits fluctuations in this evaluation. · CiteArk
Not assessedPlan blockedFindingrepetitive-mode-hopping
With repeated demonstration answers, models fluctuate between copying the repeated pattern and solving the task. OLMo3-7B has a strong tendency toward pattern matching, including letter-counting accuracy near zero throughout pretraining; OLMo3-32B generalizes more often but can suddenly fail. Apertus also exhibits fluctuations in this evaluation.
