For 1.3B- and 2.7B-parameter GPT-style models at 2k and 8k context, FlashAttention-2 is summarized as up to 1.3× faster than FlashAttention and 2.8× faster than a baseline without FlashAttention, reaching 225 TFLOPs/s/GPU and 72% model-FLOPs utilization.

For 1.3B- and 2.7B-parameter GPT-style models at 2k and 8k context, FlashAttention-2 is summarized as up to 1.3× faster than FlashAttention and 2.8× faster than a baseline without FlashAttention, reaching 225 TFLOPs/s/GPU and 72% model-FLOPs utilization. · CiteArk

Measurement results

0/4 assessed

0 supported0 disputed or conflicting4 not assessed

The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
method: FlashAttention-2 · baseline: FlashAttention	1.3 ×	—	Not assessed
method: FlashAttention-2 · baseline: Without FlashAttention	2.8 ×	—	Not assessed
model: GPT3-2.7B · method: FlashAttention-2 · context: 8k	225 TFLOPs/s/GPU	—	Not assessed
gpu: A100 · method: FlashAttention-2	72%	—	Not assessed

For 1.3B- and 2.7B-parameter GPT-style models at 2k and 8k context, FlashAttention-2 is summarized as up to 1.3× faster than FlashAttention and 2.8× faster than a baseline without FlashAttention, reaching 225 TFLOPs/s/GPU and 72% model-FLOPs utilization.

Reported and observed measurements

Assessments (0)

Runs (0)

Measurement results