Source: paper:pp. 8–9, §4.4, Table 4; pp. 18–19, B.3, Table 8
No immutable Assessment has been published for this Claim yet.
No execution has been linked to this Claim yet.
0/20 assessed
The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.
| Metric and conditions | Reported | Latest observation | Evidence judgment |
|---|---|---|---|
variant: w/o Memorizer · category: overall · benchmark: locomo | 49.06 F1 points (0–100) | — | Not assessed |
variant: w/o Memorizer · category: overall · benchmark: longmemeval | 64% | — | Not assessed |
variant: w/o Memorizer · category: overall · benchmark: narrativeqa | 38.87 F1 points (0–100) | — | Not assessed |
variant: w/o Memorizer · category: overall · benchmark: hotpotqa | 57.72 F1 points (0–100) | — | Not assessed |
variant: w/o Researcher · category: overall · benchmark: locomo | 43.3 F1 points (0–100) | — | Not assessed |
variant: w/o Researcher · category: overall · benchmark: longmemeval | 58.4% | — | Not assessed |
variant: w/o Researcher · category: overall · benchmark: narrativeqa | 32.51 F1 points (0–100) | — | Not assessed |
variant: w/o Researcher · category: overall · benchmark: hotpotqa | 39.1 F1 points (0–100) | — | Not assessed |
setting: Flat Raw Store · benchmark: LoCoMo | 0 tokens/workspace | — | Not assessed |
setting: Flat Raw Store · benchmark: LoCoMo | 0 s/workspace | — | Not assessed |
setting: Flat Raw Store · benchmark: LoCoMo | 8,174.29 tokens/query | — | Not assessed |
setting: Flat Raw Store · benchmark: LoCoMo | 17.11 s/query | — | Not assessed |