Deep Residual Learning for Image Recognition (ResNet)PDF Download

ResNet introduces residual connections that let a network learn a change to its input representation. The paper studies why making a network deeper can otherwise make training more difficult.

The distinction between optimization difficulty and overfitting is worth slowing down for. Once that problem is clear, the seemingly simple skip connection becomes a much more interesting design choice.