Table 1 reports per-GPU training throughput for GPT3-1.3B at 8k context on 8×A100 GPUs for the baseline without FlashAttention, FlashAttention, and FlashAttention-2.

Table 1 reports per-GPU training throughput for GPT3-1.3B at 8k context on 8×A100 GPUs for the baseline without FlashAttention, FlashAttention, and FlashAttention-2. · CiteArk

测量结果

已评估 0/3 项

获得支持 0有争议或冲突 0尚未评估 3

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
model: GPT3-1.3B · method: Without FlashAttention · context: 8k	72 TFLOPs/s/GPU	—	尚未评估
model: GPT3-1.3B · method: FlashAttention · context: 8k	170 TFLOPs/s/GPU	—	尚未评估
model: GPT3-1.3B · method: FlashAttention-2 · context: 8k	220 TFLOPs/s/GPU	—	尚未评估

Table 1 reports per-GPU training throughput for GPT3-1.3B at 8k context on 8×A100 GPUs for the baseline without FlashAttention, FlashAttention, and FlashAttention-2.

报告指标与观测值

Assessment（0）

关联运行（0）

测量结果