Across FineWeb-Edu tiers, the paired validation-loss benefit of the sink logit shrinks with scale, while the additional benefit of a gate when combined with the sink logit grows overall; combo2 has the lowest mean validation loss at every tested tier.

Across FineWeb-Edu tiers, the paired validation-loss benefit of the sink logit shrinks with scale, while the additional benefit of a gate when combined with the sink logit grows overall; combo2 has the lowest mean validation loss at every tested tier. · CiteArk

测量结果

已评估 0/69 项

获得支持 0有争议或冲突 0尚未评估 69

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
arm: baseline · tier: 10M	4.2119 nats	—	尚未评估
arm: baseline · tier: 50M	3.5399 nats	—	尚未评估
arm: baseline · tier: 124M	3.371 nats	—	尚未评估
arm: baseline · tier: 350M	3.0338 nats	—	尚未评估
arm: off-by-one · tier: 10M · comparison: baseline	-0.016 nats	—	尚未评估
arm: off-by-one · tier: 10M · comparison: baseline	0.0027 nats	—	尚未评估
arm: sink token · tier: 10M · comparison: baseline	-0.0025 nats	—	尚未评估
arm: sink token · tier: 10M · comparison: baseline	0.0041 nats	—	尚未评估
arm: sink logit · tier: 10M · comparison: baseline	-0.0185 nats	—	尚未评估
arm: sink logit · tier: 10M · comparison: baseline	0.0033 nats	—	尚未评估
arm: sink logit · tier: 50M · comparison: baseline	-0.0068 nats	—	尚未评估
arm: sink logit · tier: 50M · comparison: baseline	0.0003 nats	—	尚未评估

Across FineWeb-Edu tiers, the paired validation-loss benefit of the sink logit shrinks with scale, while the additional benefit of a gate when combined with the sink logit grows overall; combo2 has the lowest mean validation loss at every tested tier.

报告指标与观测值

Assessment（0）

关联运行（0）

测量结果