Exact SVD at every EGD update adds substantial per-step computation. The paper mitigates this by switching off EGD after a chosen validation threshold or using truncated randomized SVD; it nevertheless claims EGD requires no additional stored gradient history, unlike the compared Grokfast EMA/filtering setup.

Exact SVD at every EGD update adds substantial per-step computation. The paper mitigates this by switching off EGD after a chosen validation threshold or using truncated randomized SVD; it nevertheless claims EGD requires no additional stored gradient history, unlike the compared Grokfast EMA/filtering setup. · CiteArk

Reported and observed measurements

Assessments (0)

Runs (0)