Diagnosing the Fact-Grounding Gap in Multi-Hop Question Answering · CiteArk

ArkGraph

浏览 ArkGraph，选择本次要执行的步骤。

Diagnosing the Fact-Grounding Gap in Multi-Hop Question Answering

公开

作者：Kevin Mo、Nathan Mo、Richard Zhu

arXiv 2026 EMNLP 2026 计算与语言人工智能信息检索

复现结果

全部结论 ↗

尚无复现结果

论文中的结论可在「研究结论」中查看。

完整研究主张目录

0 / 34 条结论已通过验证

其余仍在验证中

关键指标论文报告值 → 复现观测值 · 点击行展开详情

The paper reports strong agreement between GPT-4.1-mini judging and human annotation, while noting a conservative error skew for fact-presence decisions.整组结论：等待复现0/5 项指标获得支持 · 0 项已有评估5项测量结果展开结果表

实验方案已生成，还没有运行记录

测量结果

已评估 0/5 项

获得支持 0有争议或冲突 0尚未评估 5

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
论文报告	最近可用实测	证据判断
92%	—	尚未评估
0.84 coefficient	—	尚未评估
200 examples	—	尚未评估
12 cases	—	尚未评估
16 cases	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
dataset: MuSiQue · flaggedHops: 0	68.7%	—	尚未评估
dataset: MuSiQue · flaggedHops: 0	69.1%	—	尚未评估
dataset: MuSiQue · flaggedHops: 1	52.9%	—	尚未评估
dataset: MuSiQue · flaggedHops: 1	56.6%	—	尚未评估
dataset: MuSiQue · flaggedHops: 2	44.9%	—	尚未评估
dataset: MuSiQue · flaggedHops: 2	49.6%	—	尚未评估
dataset: MuSiQue · flaggedHops: 3	38.8%	—	尚未评估
dataset: MuSiQue · flaggedHops: 3	44.7%	—	尚未评估
dataset: MuSiQue · flaggedHops: 4	8.5%	—	尚未评估
dataset: MuSiQue · flaggedHops: 4	25.5%	—	尚未评估
dataset: MuSiQue · flaggedHops: 0	0.4 percentage points	—	尚未评估
dataset: MuSiQue · flaggedHops: 4	17 percentage points	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
condition: Baseline	78.8%	—	尚未评估
condition: Baseline	0%	—	尚未评估
condition: Baseline	0 calls/question	—	尚未评估
condition: Always	78.7%	—	尚未评估
condition: Always	100%	—	尚未评估
condition: Always	2.15 calls/question	—	尚未评估
condition: DeBERTa targeted	78.9%	—	尚未评估
condition: DeBERTa targeted	54.1%	—	尚未评估
condition: DeBERTa targeted	1.16 calls/question	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
condition: Baseline	42.2%	—	尚未评估
condition: Baseline	0%	—	尚未评估
condition: Baseline	0 calls/question	—	尚未评估
condition: DeBERTa targeted	52.2%	—	尚未评估
condition: DeBERTa targeted	53.4%	—	尚未评估
condition: DeBERTa targeted	1.07 calls/question	—	尚未评估
condition: Oracle	52.3%	—	尚未评估
condition: Oracle	54.1%	—	尚未评估
condition: Oracle	1.08 calls/question	—	尚未评估
	10 percentage points	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
model: BERT-base	110 millions	—	尚未评估
model: BERT-base	0.753 score	—	尚未评估
model: BERT-base	0.737 score	—	尚未评估
model: BERT-base	0.745 score	—	尚未评估
model: RoBERTa-large	355 millions	—	尚未评估
model: RoBERTa-large	0.84 score	—	尚未评估
model: RoBERTa-large	0.683 score	—	尚未评估
model: RoBERTa-large	0.754 score	—	尚未评估
model: DeBERTa-v3-large	435 millions	—	尚未评估
model: DeBERTa-v3-large	0.805 score	—	尚未评估
model: DeBERTa-v3-large	0.765 score	—	尚未评估
model: DeBERTa-v3-large	0.785 score	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
flaggedHops: 0	556 questions	—	尚未评估
flaggedHops: 0	68.7%	—	尚未评估
flaggedHops: 1	868 questions	—	尚未评估
flaggedHops: 1	52.9%	—	尚未评估
flaggedHops: 2	690 questions	—	尚未评估
flaggedHops: 2	44.9%	—	尚未评估
flaggedHops: 3	255 questions	—	尚未评估
flaggedHops: 3	38.8%	—	尚未评估
flaggedHops: 4	47 questions	—	尚未评估
flaggedHops: 4	8.5%	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
intervention: augmentation	8.6 percentage points	—	尚未评估
intervention: reranking	25.3 percentage points	—	尚未评估
intervention: re-retrieval; intervention subtype not separately reported for this value	-0.5 percentage points	—	尚未评估
dataset: MuSiQue	97.7%	—	尚未评估
dataset: HotpotQA	100%	—	尚未评估
dataset: 2WikiMultihopQA	97.6%	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
questionType: comparison	2.7%	—	尚未评估
questionType: multi-step	16.6%	—	尚未评估
	97.4%	—	尚未评估
	0.983 score	—	尚未评估
	0%	—	尚未评估
predictor: 2WikiMultihopQA-retrained	78.9%	—	尚未评估
	78.8%	—	尚未评估
	0 probability	—	尚未评估
	0.999 probability	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
condition: Baseline	51.8%	—	尚未评估
condition: Baseline	0%	—	尚未评估
condition: Baseline	0 calls/question	—	尚未评估
condition: Always	60.9%	—	尚未评估
condition: Always	100%	—	尚未评估
condition: Always	2.65 calls/question	—	尚未评估
condition: DeBERTa targeted	60.3%	—	尚未评估
condition: DeBERTa targeted	50%	—	尚未评估
condition: DeBERTa targeted	1.33 calls/question	—	尚未评估
	8.5 percentage points	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
questionHops: 3	-2.1 percentage points	—	尚未评估
questionHops: 3	2.1 percentage points	—	尚未评估
questionHops: 3	4.2 percentage points	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
	0.449 coefficient	—	尚未评估
	0.042 coefficient	—	尚未评估
	5,283 hops	—	尚未评估
method: Majority	50.9%	—	尚未评估
method: Majority	0 score	—	尚未评估
method: Hop number	64.9%	—	尚未评估
method: Hop number	0.596 score	—	尚未评估
method: Retrieval recall	68.1%	—	尚未评估
method: Retrieval recall	0.716 score	—	尚未评估
method: String match · usesGold: true	80.8%	—	尚未评估
method: String match · usesGold: true	0.823 score	—	尚未评估
method: Gold paragraph present · usesGold: true	83.4%	—	尚未评估

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
labels: string match · passages: excluded	54.7%	—	尚未评估
labels: string match · passages: excluded	0.547 score	—	尚未评估
labels: LLM judge · passages: excluded	63.6%	—	尚未评估
labels: LLM judge · passages: excluded	0.615 score	—	尚未评估
labels: LLM judge · passages: included	79.4%	—	尚未评估
labels: LLM judge · passages: included	0.785 score	—	尚未评估
	8.9 percentage points	—	尚未评估
	15.8 percentage points	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
hop: 1	68.5%	—	尚未评估
hop: 1	2,417 hops	—	尚未评估
hop: 2	43.7%	—	尚未评估
hop: 2	2,417 hops	—	尚未评估
hop: 3	30.7%	—	尚未评估
hop: 3	1,165 hops	—	尚未评估
hop: 4	18.5%	—	尚未评估
hop: 4	405 hops	—	尚未评估
hop: all	49.1%	—	尚未评估
hop: all	6,404 hops	—	尚未评估

论文报告	最近可用实测	证据判断
1,962 hops	—	尚未评估
30.7%	—	尚未评估
1,749 hops	—	尚未评估
27.3%	—	尚未评估
2,690 hops	—	尚未评估
42%	—	尚未评估
47%	—	尚未评估
9%	—	尚未评估
3 hops	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
condition: Baseline	48.1%	—	尚未评估
condition: Baseline	0%	—	尚未评估
condition: Baseline	0 calls/question	—	尚未评估
condition: Always	57.1%	—	尚未评估
condition: Always	100%	—	尚未评估
condition: Always	2 calls/question	—	尚未评估
condition: DeBERTa targeted	53.2%	—	尚未评估
condition: DeBERTa targeted	30.7%	—	尚未评估
condition: DeBERTa targeted	0.61 calls/question	—	尚未评估
condition: Oracle	55.5%	—	尚未评估
condition: Oracle	46.9%	—	尚未评估
condition: Oracle	0.94 calls/question	—	尚未评估

论文报告	最近可用实测	证据判断
81.7%	—	尚未评估
2,677 hops	—	尚未评估
18.3%	—	尚未评估
15%	—	尚未评估
468 hops	—	尚未评估

论文报告	最近可用实测	证据判断
6,404 sub-questions	—	尚未评估
4,100 sub-questions	—	尚未评估
2,304 sub-questions	—	尚未评估
1,315 patterns	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
condition: Baseline	51.9%	—	尚未评估
condition: Baseline	0%	—	尚未评估
condition: Baseline	0 calls/question	—	尚未评估
condition: Always	54.6%	—	尚未评估
condition: Always	100%	—	尚未评估
condition: Always	2.65 calls/question	—	尚未评估
condition: DeBERTa targeted	55.6%	—	尚未评估
condition: DeBERTa targeted	50%	—	尚未评估
condition: DeBERTa targeted	1.32 calls/question	—	尚未评估
condition: Oracle	54.9%	—	尚未评估
condition: Oracle	48.9%	—	尚未评估
condition: Oracle	1.29 calls/question	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
dataset: MuSiQue train	46,610 hops	—	尚未评估
	53.9%	—	尚未评估
	46.1%	—	尚未评估
dataset: MuSiQue dev	6,404 hops	—	尚未评估
	3 epoch	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
dataset: MuSiQue	139,416 passages	—	尚未评估
dataset: HotpotQA	507,494 passages	—	尚未评估
dataset: 2WikiMultihopQA	136,009 passages	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
condition: Baseline	51.2%	—	尚未评估
condition: Baseline	0%	—	尚未评估
condition: Baseline	0 calls/question	—	尚未评估
condition: DeBERTa targeted	54.7%	—	尚未评估
condition: DeBERTa targeted	45.2%	—	尚未评估
condition: DeBERTa targeted	1.2 calls/question	—	尚未评估
condition: Oracle	54.8%	—	尚未评估
condition: Oracle	50.9%	—	尚未评估
condition: Oracle	1.35 calls/question	—	尚未评估
	3.5 percentage points	—	尚未评估

论文报告	最近可用实测	证据判断
654 traces	—	尚未评估
48.1%	—	尚未评估
51.6%	—	尚未评估
3.5 percentage points	—	尚未评估
2,417 questions	—	尚未评估
3.7 percentage points	—	尚未评估

ArkGraph

Diagnosing the Fact-Grounding Gap in Multi-Hop Question Answering

研究结论

尚无复现结果

关键指标论文报告值 → 复现观测值 · 点击行展开详情

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

定性结论

论文报告	最近可用实测	证据判断
3,201 hops	—	尚未评估
50%	—	尚未评估
50.9%	—	尚未评估

论文报告	最近可用实测	证据判断
59.8%	—	尚未评估
49.1%	—	尚未评估
10.7 percentage points	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Reranking	8.5 percentage points	—	尚未评估
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Reranking	0 probability	—	尚未评估
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Reranking	6.8 percentage points	—	尚未评估
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Reranking	10.2 percentage points	—	尚未评估
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Augmentation	3.7 percentage points	—	尚未评估
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Augmentation	0 probability	—	尚未评估
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Augmentation	2.2 percentage points	—	尚未评估
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: BM25 · intervention: Augmentation	5.2 percentage points	—	尚未评估
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: Contriever · intervention: Augmentation	3.5 percentage points	—	尚未评估
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: Contriever · intervention: Augmentation	0 probability	—	尚未评估
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: Contriever · intervention: Augmentation	2.1 percentage points	—	尚未评估
dataset: MuSiQue · predictor: MuSiQue-trained · retriever: Contriever · intervention: Augmentation	4.8 percentage points	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
predictor: Random	-0.066 coefficient	—	尚未评估
predictor: Random	0.0011 probability	—	尚未评估
predictor: DeBERTa	-0.216 coefficient	—	尚未评估
predictor: DeBERTa	0 probability	—	尚未评估
predictor: BM25 threshold	-0.254 coefficient	—	尚未评估
predictor: BM25 threshold	0 probability	—	尚未评估
predictor: DeBERTa intersection BM25 threshold	-0.282 coefficient	—	尚未评估
predictor: DeBERTa intersection BM25 threshold	0 probability	—	尚未评估
	42.9%	—	尚未评估
flagCategory: Both flag	1,971 hops	—	尚未评估
flagCategory: Both flag	30.8%	—	尚未评估
flagCategory: DeBERTa only	1,230 hops	—	尚未评估

指标与条件	论文报告	最近可用实测	证据判断
dataset: MuSiQue dev · retriever: BM25	598 hops	—	尚未评估
dataset: MuSiQue dev · retriever: Contriever	725 hops	—	尚未评估
	77%	—	尚未评估
dataset: MuSiQue · retriever: Contriever	51.2%	—	尚未评估
dataset: MuSiQue · retriever: BM25	51.9%	—	尚未评估
dataset: HotpotQA · retriever: BM25	3.1%	—	尚未评估
dataset: HotpotQA · retriever: Contriever	4.5%	—	尚未评估