For sparse parity at k=2, 3, and 4, EGD and vanilla SGD initially diverge, after which the paper's accompanying prose says vanilla SGD moves toward the vicinity where EGD began grokking. EGD overshoots its distance from initialization, while vanilla SGD approaches gradually; both ultimately maintain approximately the same fixed distance.

For sparse parity at k=2, 3, and 4, EGD and vanilla SGD initially diverge, after which the paper's accompanying prose says vanilla SGD moves toward the vicinity where EGD began grokking. EGD overshoots its distance from initialization, while vanilla SGD approaches gradually; both ultimately maintain approximately the same fixed distance. · CiteArk

Reported and observed measurements

Assessments (0)

Runs (0)