Loading page…
In the Mamba-block ablation with SSD as the inner sequence-mixing layer, parallel ABCX projections use fewer parameters and slightly lower perplexity than sequential projections, while the extra normalization used in Mamba-2 slightly improves perplexity. The paper also reports preliminary larger-scale evidence that the extra normalization improves training stability. · CiteArk