Table 1 reports per-GPU training throughput for GPT3-2.7B at 8k context on 8×A100 GPUs for the baseline without FlashAttention, FlashAttention, and FlashAttention-2.

Table 1 reports per-GPU training throughput for GPT3-2.7B at 8k context on 8×A100 GPUs for the baseline without FlashAttention, FlashAttention, and FlashAttention-2. · CiteArk

Measurement results

0/3 assessed

0 supported0 disputed or conflicting3 not assessed

The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
model: GPT3-2.7B · method: Without FlashAttention · context: 8k	80 TFLOPs/s/GPU	—	Not assessed
model: GPT3-2.7B · method: FlashAttention · context: 8k	175 TFLOPs/s/GPU	—	Not assessed
model: GPT3-2.7B · method: FlashAttention-2 · context: 8k	225 TFLOPs/s/GPU	—	Not assessed

Table 1 reports per-GPU training throughput for GPT3-2.7B at 8k context on 8×A100 GPUs for the baseline without FlashAttention, FlashAttention, and FlashAttention-2.

Reported and observed measurements

Assessments (0)

Runs (0)

Measurement results