Loading page…
Randomized-SVD EGD is not hyperparameter-free: its rank and inner-loop iteration count must be chosen, and reducing rank can trade faster updates for delayed or absent grokking. The paper fixes two inner iterations in Appendix C, omits oversampling from its simplified RSVD implementation, leaves the effect of oversampling for future work, and states that rank tuning is needed for the best gain. · CiteArk