In the anisotropic two-feature toy problem, vanilla GD rapidly fits the training data but can retain chance-level or very poor test performance for a long plateau. With distribution shift, the reported plateau scales as order 1/(eta epsilon) for large initialization and order (1/eta) log(1/(tau epsilon)) for small initialization; with identical train and test distributions it again scales as order 1/(eta epsilon). · CiteArk
In the anisotropic two-feature toy problem, vanilla GD rapidly fits the training data but can retain chance-level or very poor test performance for a long plateau. With distribution shift, the reported plateau scales as order 1/(eta epsilon) for large initialization and order (1/eta) log(1/(tau epsilon)) for small initialization; with identical train and test distributions it again scales as order 1/(eta epsilon).
来源：paper:PDF pp. 5-6, Theorem 1, Figure 6, Corollary 1, and same-distribution paragraph
