On the distribution-shift toy experiment, EGD reaches perfect generalization after only a few iterations and appears insensitive to initialization scale, whereas large initialization lengthens the vanilla-GD plateau and small initialization attenuates it. · CiteArk
On the distribution-shift toy experiment, EGD reaches perfect generalization after only a few iterations and appears insensitive to initialization scale, whereas large initialization lengthens the vanilla-GD plateau and small initialization attenuates it.
