The formal grokking and convergence analysis is confined to a simplified two-dimensional linear model with squared loss and controlled Gaussian anisotropy. A convergence analysis for complex models such as MLPs and transformers is explicitly left for future work. · CiteArk
The formal grokking and convergence analysis is confined to a simplified two-dimensional linear model with squared loss and controlled Gaussian anisotropy. A convergence analysis for complex models such as MLPs and transformers is explicitly left for future work.
来源:paper:PDF pp. 4-7 and 10, Section 3 and Future Work