In the Chinchilla-style Pile scaling-law plots, Mamba-2 matches or exceeds Mamba and the Transformer++ recipe across approximately 125M to 1.3B parameters. Relative to the authors' Transformer baseline, the paper describes Mamba-2 as Pareto dominant in perplexity, theoretical FLOPs, and actual wall-clock time. · CiteArk
In the Chinchilla-style Pile scaling-law plots, Mamba-2 matches or exceeds Mamba and the Transformer++ recipe across approximately 125M to 1.3B parameters. Relative to the authors' Transformer baseline, the paper describes Mamba-2 as Pareto dominant in perplexity, theoretical FLOPs, and actual wall-clock time.
来源:paper_markdown:Figure 9, Sections 9.2.1 and 9.2, PDF pages 29–30