CIFAR-10 residual networks have generally smaller standard deviations of post-BN, pre-nonlinearity 3x3-layer responses than plain counterparts, and response magnitude decreases across ResNet-20, ResNet-56, and ResNet-110 as depth increases. The authors interpret this as evidence that learned residual functions are closer to zero and individual layers modify the signal less. · CiteArk
CIFAR-10 residual networks have generally smaller standard deviations of post-BN, pre-nonlinearity 3x3-layer responses than plain counterparts, and response magnitude decreases across ResNet-20, ResNet-56, and ResNet-110 as depth increases. The authors interpret this as evidence that learned residual functions are closer to zero and individual layers modify the signal less.
来源：paper:PDF page 8, Figure 7 and 'Analysis of Layer Responses' paragraph
