McNemar tests and bootstrap confidence intervals report statistically significant targeted-intervention gains for the seven MuSiQue and HotpotQA predictor/retriever/intervention combinations listed in Appendix Table 13.

Reported and observed measurements

Assessments (0)

No immutable Assessment has been published for this Claim yet.

Runs (0)

No execution has been linked to this Claim yet.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Reranking	8.5 percentage points	—	Not assessed
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Reranking	0 probability	—	Not assessed
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Reranking	6.8 percentage points	—	Not assessed
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Reranking	10.2 percentage points	—	Not assessed
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Augmentation	3.7 percentage points	—	Not assessed
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Augmentation	0 probability	—	Not assessed
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Augmentation	2.2 percentage points	—	Not assessed
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Augmentation	5.2 percentage points	—	Not assessed
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: Contriever · intervention: Augmentation	3.5 percentage points	—	Not assessed
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: Contriever · intervention: Augmentation	0 probability	—	Not assessed
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: Contriever · intervention: Augmentation	2.1 percentage points	—	Not assessed
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: Contriever · intervention: Augmentation	4.8 percentage points	—	Not assessed

Metric and conditions

Reported

Latest observation

Evidence judgment

dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Reranking

8.5 percentage points

—

Not assessed

dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Reranking

0 probability

—

Not assessed

dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Reranking

6.8 percentage points

—

Not assessed

dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Reranking

10.2 percentage points

—

Not assessed

dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Augmentation

3.7 percentage points

—

Not assessed

dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Augmentation

0 probability

—

Not assessed

dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Augmentation

2.2 percentage points

—

Not assessed

dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Augmentation

5.2 percentage points

—

Not assessed

dataset: MuSiQue · predictor: MuSiQue-trained · retriever: Contriever · intervention: Augmentation

3.5 percentage points

—

Not assessed

dataset: MuSiQue · predictor: MuSiQue-trained · retriever: Contriever · intervention: Augmentation

0 probability

—

Not assessed

dataset: MuSiQue · predictor: MuSiQue-trained · retriever: Contriever · intervention: Augmentation

2.1 percentage points

—

Not assessed

dataset: MuSiQue · predictor: MuSiQue-trained · retriever: Contriever · intervention: Augmentation

4.8 percentage points

—

Not assessed

Reported and observed measurements

Assessments (0)

Runs (0)

Measurement results