Larger or alternative Memorizer backbones do not consistently improve LoCoMo F1: Qwen3.5-122B-A10B lowers it from 52.09 to 51.30, while GPT-5.5 raises it modestly to 52.62. Both increase workspace construction time; online latency and action rounds change only slightly.

Larger or alternative Memorizer backbones do not consistently improve LoCoMo F1: Qwen3.5-122B-A10B lowers it from 52.09 to 51.30, while GPT-5.5 raises it modestly to 52.62. Both increase workspace construction time; online latency and action rounds change only slightly. · CiteArk

Measurement results

0/12 assessed

0 supported0 disputed or conflicting12 not assessed

The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
researcher: trained Qwen3.5-4B · memorizerBackbone: Qwen3.5-4B (default)	79.53 s/workspace	—	Not assessed
researcher: trained Qwen3.5-4B · memorizerBackbone: Qwen3.5-4B (default)	13.81 s/query	—	Not assessed
researcher: trained Qwen3.5-4B · memorizerBackbone: Qwen3.5-4B (default)	3.16 rounds/query	—	Not assessed
researcher: trained Qwen3.5-4B · memorizerBackbone: Qwen3.5-4B (default)	52.09 F1 points (0–100)	—	Not assessed
researcher: trained Qwen3.5-4B · memorizerBackbone: Qwen3.5-122B-A10B	132.43 s/workspace	—	Not assessed
researcher: trained Qwen3.5-4B · memorizerBackbone: Qwen3.5-122B-A10B	14.36 s/query	—	Not assessed
researcher: trained Qwen3.5-4B · memorizerBackbone: Qwen3.5-122B-A10B	3.25 rounds/query	—	Not assessed
researcher: trained Qwen3.5-4B · memorizerBackbone: Qwen3.5-122B-A10B	51.3 F1 points (0–100)	—	Not assessed
researcher: trained Qwen3.5-4B · memorizerBackbone: GPT-5.5	117.65 s/workspace	—	Not assessed
researcher: trained Qwen3.5-4B · memorizerBackbone: GPT-5.5	13.19 s/query	—	Not assessed
researcher: trained Qwen3.5-4B · memorizerBackbone: GPT-5.5	3.03 rounds/query	—	Not assessed
researcher: trained Qwen3.5-4B · memorizerBackbone: GPT-5.5	52.62 F1 points (0–100)	—	Not assessed

Larger or alternative Memorizer backbones do not consistently improve LoCoMo F1: Qwen3.5-122B-A10B lowers it from 52.09 to 51.30, while GPT-5.5 raises it modestly to 52.62. Both increase workspace construction time; online latency and action rounds change only slightly.

Reported and observed measurements

Assessments (0)

Runs (0)

Measurement results