For sparse parity at k=2, 3, and 4, EGD and vanilla SGD initially diverge, after which the paper's accompanying prose says vanilla SGD moves toward the vicinity where EGD began grokking. EGD overshoots its distance from initialization, while vanilla SGD approaches gradually; both ultimately maintain approximately the same fixed distance. · CiteArk
For sparse parity at k=2, 3, and 4, EGD and vanilla SGD initially diverge, after which the paper's accompanying prose says vanilla SGD moves toward the vicinity where EGD began grokking. EGD overshoots its distance from initialization, while vanilla SGD approaches gradually; both ultimately maintain approximately the same fixed distance.
来源：paper:PDF pp. 25 and 27-28, Appendix F, Figures 21-22 and accompanying observations
