The authors report that the degradation problem of deeper plain networks is also observed with bottleneck designs, while deeper non-bottleneck ResNets gain accuracy with depth but are less economical than bottleneck ResNets.

Source: paper:PDF page 6, footnote 4 in the deeper-bottleneck architecture discussion