Exact SVD at every EGD update adds substantial per-step computation. The paper mitigates this by switching off EGD after a chosen validation threshold or using truncated randomized SVD; it nevertheless claims EGD requires no additional stored gradient history, unlike the compared Grokfast EMA/filtering setup. · CiteArk
Exact SVD at every EGD update adds substantial per-step computation. The paper mitigates this by switching off EGD after a chosen validation threshold or using truncated randomized SVD; it nevertheless claims EGD requires no additional stored gradient history, unlike the compared Grokfast EMA/filtering setup.
来源：paper:PDF pp. 8, 18, and 22-24, Practical Considerations and Appendices C-D
