研究仓库

This paper studies why coordinate-based multilayer perceptrons struggle to fit high-frequency signals in low-dimensional domains. It analyzes training through neural tangent kernels and shows how sinusoidal Fourier features produce a stationary effective kernel whose bandwidth can be adjusted. Experiments examine convergence, generalization, feature sampling distributions, network depth, joint feature optimization, translation sensitivity, and directional bias. Direct image and shape regression and indirectly supervised CT, MRI, and simplified NeRF reconstruction compare unembedded inputs with basic, positional, and Gaussian mappings. Gaussian features provide the strongest reported results among the main mappings, while bandwidth selection balances underfitting and overfitting. Appendix studies identify limitations of feature optimization and axis-aligned encodings.

This paper introduces residual learning for training substantially deeper neural networks. Instead of directly fitting a desired mapping, stacked layers learn a residual that is added to an identity shortcut. The authors compare plain and residual networks on ImageNet and CIFAR-10, examine shortcut variants and residual-response magnitudes, and scale residual networks to 152 layers on ImageNet and 1202 layers on CIFAR-10. Reported results show reduced optimization degradation and improved classification accuracy with depth. The learned representations also improve Faster R-CNN detection on PASCAL VOC and COCO and support competitive ImageNet detection and localization systems.