CIFAR-10 residual networks have generally smaller standard deviations of post-BN, pre-nonlinearity 3x3-layer responses than plain counterparts, and response magnitude decreases across ResNet-20, ResNet-56, and ResNet-110 as depth increases. The authors interpret this as evidence that learned residual functions are closer to zero and individual layers modify the signal less.

CIFAR-10 residual networks have generally smaller standard deviations of post-BN, pre-nonlinearity 3x3-layer responses than plain counterparts, and response magnitude decreases across ResNet-20, ResNet-56, and ResNet-110 as depth increases. The authors interpret this as evidence that learned residual functions are closer to zero and individual layers modify the signal less. · CiteArk

Reported and observed measurements

Assessments (0)

Runs (0)