The MNIST learning curves show EGD and its RSVD variants grokking earlier than Grokfast. Grokfast exhibits a sudden accuracy drop and recovery, and both Grokfast and vanilla AdamW have noisy training loss, whereas exact-SVD EGD and all plotted RSVD ranks have smoother training-loss curves; some RSVD ranks grok faster than exact SVD. · CiteArk
The MNIST learning curves show EGD and its RSVD variants grokking earlier than Grokfast. Grokfast exhibits a sudden accuracy drop and recovery, and both Grokfast and vanilla AdamW have noisy training loss, whereas exact-SVD EGD and all plotted RSVD ranks have smoother training-loss curves; some RSVD ranks grok faster than exact SVD.
来源：paper:PDF pp. 22-23, Figures 12-13 and accompanying MNIST paragraph
