KITE: KV-Invariant Transformer Expansion for Efficient Agentic LLM Scaling · CiteArk

Recent activity

—Analysis made new progress
—Analysis environment started
—Analysis made new progress
—Research inventory and experiment plan prepared
—Waiting for experiment selection

ArkGraph

Explore ArkGraph and select the steps to run.

KITE: KV-Invariant Transformer Expansion for Efficient Agentic LLM Scaling

Public

Authors:Zhiheng Hu, Yixun Wei, Jian Zhou, Yizhuang Zhou, Ji Li, Xing Chen, Yang Li, Bojun Wang, Yibo Zhu, Xiangyu Zhang, Daxin Jiang

arXiv 2026 Machine Learning (cs.LG)Artificial Intelligence

Reproduction results

All claims ↗

No reproduction results yet

The paper’s claims are available in Research claims.

Full research claim catalog

0 / 13 claims verified

the rest still being verified

Key metricsPaper-reported → reproduced · click a row to expand

At the same final checkpoints used for the matched-compute comparison, 67B SST scores higher than both classic baselines on all seven reported downstream tasks and has the lowest held-out ARXIV token NLL: 1.4421 versus 1.4424 for 63B classic and 1.4521 for 47B classic.Whole claim: Awaiting reproduction0/24 measurements supported · 0 assessed24measurementsExplore results

Experiment plan ready; no runs yet.

Measurement results

0/24 assessed

0 supported0 disputed or conflicting24 not assessed

The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
model: 47B classic · split: validation + test	68.5%	—	Not assessed
model: 63B classic · split: validation + test	71%	—	Not assessed
model: 67B SST · split: validation + test	75%	—	Not assessed
model: 47B classic · split: test	58.44%	—	Not assessed
model: 63B classic · split: test	59.39%	—	Not assessed
model: 67B SST · split: test	60.54%	—	Not assessed
model: 47B classic	56.56%	—	Not assessed
model: 63B classic	55.04%	—	Not assessed
model: 67B SST	58.38%	—	Not assessed
model: 47B classic · split: full test	33.06%	—	Not assessed
model: 63B classic · split: full test	33.14%	—	Not assessed
model: 67B SST · split: full test	34.36%	—	Not assessed

Open claim details

At final checkpoints with approximately matched theoretical cumulative training FLOPs, 67B SST has the lowest reported EMA-200 training LM loss: 1.5900 versus 1.6006 for 47B classic and 1.5921 for 63B classic, improvements of 0.0106 and 0.0021 respectively.Whole claim: Awaiting reproduction0/12 measurements supported · 0 assessed12measurementsExplore results

Experiment plan ready; no runs yet.

Measurement results

0/12 assessed

0 supported0 disputed or conflicting12 not assessed

The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
model: 67B SST · checkpoint: final · training_tokens_billion: 390.54	1.59 loss	—	Not assessed
model: 47B classic · checkpoint: final · training_tokens_billion: 443.16	1.6006 loss	—	Not assessed
model: 63B classic · checkpoint: final · training_tokens_billion: 334.62	1.5921 loss	—	Not assessed
model: 67B SST · comparison: 47B classic	-0.0106 loss	—	Not assessed
model: 67B SST · comparison: 63B classic	-0.0021 loss	—	Not assessed
model: 63B classic · comparison: 47B classic	-0.0085 loss	—	Not assessed
model: 47B classic	443.16 billion tokens	—	Not assessed
model: 63B classic	334.62 billion tokens	—	Not assessed
model: 47B classic	1 fraction of 47B reference	—	Not assessed
model: 63B classic · qualifier: approximately	1 fraction of 47B reference	—	Not assessed
model: 67B SST · qualifier: approximately · includes_source_stage: true	1 fraction of 47B reference	—	Not assessed
model: 47B classic · qualifier: approximately	5,168,360,000,000,000,000,000 FLOPs	—	Not assessed

Open claim details

For each of six selected OpenRouter models, total input tokens and uncached input tokens exceed output tokens; estimated uncached-input charges account for 67.4% to 76.7% of uncached-input-plus-output charges.Whole claim: Awaiting reproduction0/18 measurements supported · 0 assessed18measurementsExplore results

Experiment plan ready; no runs yet.

Measurement results

0/18 assessed

0 supported0 disputed or conflicting18 not assessed

The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
model: GPT-6 Astra	106.43 ratio	—	Not assessed
model: GPT-6 Astra	14.64 ratio	—	Not assessed
model: GPT-6 Astra	74.6%	—	Not assessed
model: GPT-6 Astra Pro	63.15 ratio	—	Not assessed
model: GPT-6 Astra Pro	14.01 ratio	—	Not assessed
model: GPT-6 Astra Pro	73.8%	—	Not assessed
model: Claude Fable 5	63.38 ratio	—	Not assessed
model: Claude Fable 5	16.29 ratio	—	Not assessed
model: Claude Fable 5	76.7%	—	Not assessed
model: Claude Sonnet 5	59.88 ratio	—	Not assessed
model: Claude Sonnet 5	12.12 ratio	—	Not assessed
model: Claude Sonnet 5	70.8%	—	Not assessed

Open claim details

SST trades lower bulk-prefill active-body count for higher decode active-body count relative to 47B classic. Under the paper's common analytical proxy, SST becomes cheaper than 47B once prefill weight exceeds 65.5% and cheaper than 63B above 13.4%; at a 75:25 prefill:decode mix it is estimated 6.7% below 47B and 31.6% below 63B.Whole claim: Awaiting reproduction0/18 measurements supported · 0 assessed18measurementsExplore results

Experiment plan ready; no runs yet.

Measurement results

0/18 assessed

0 supported0 disputed or conflicting18 not assessed

The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
model: 67B SST · component: Prefiller only	1.12 billion parameters per token	—	Not assessed
model: 67B SST · components: Prefiller + Decoder	2.155 billion parameters per token	—	Not assessed
model: 47B classic	1.477 billion parameters per token	—	Not assessed
model: 63B classic	2.016 billion parameters per token	—	Not assessed
model: 67B SST · decode_weight_percent: 25 · prefill_weight_percent: 75	0.933 fraction of 47B classic	—	Not assessed
model: 47B classic · decode_weight_percent: 25 · prefill_weight_percent: 75	1 fraction of 47B classic	—	Not assessed
model: 63B classic · decode_weight_percent: 25 · prefill_weight_percent: 75	1.365 fraction of 47B classic	—	Not assessed
model: 63B classic · comparison: 47B classic · prefill_decode_mix: 75:25	36.5%	—	Not assessed
model: 67B SST · comparison: 47B classic · prefill_decode_mix: 75:25	6.7%	—	Not assessed
model: 67B SST · comparison: 63B classic · prefill_decode_mix: 75:25	31.6%	—	Not assessed
models: 67B SST and 47B classic · decode_weight_percent: 34.5	65.5%	—	Not assessed
models: 67B SST and 63B classic · decode_weight_percent: 86.6	13.4%	—	Not assessed

Open claim details

The matched-compute result is based on an effective-architecture theoretical FLOP estimator whose reported forward coefficients differ by model and whose training estimate multiplies forward computation by three to approximate forward plus backward work.Whole claim: Awaiting reproduction0/8 measurements supported · 0 assessed8measurementsExplore results

Experiment plan ready; no runs yet.

Measurement results

0/8 assessed

0 supported0 disputed or conflicting8 not assessed

The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
model: 47B classic · qualifier: approximately	3.8875 billion FLOPs per token	—	Not assessed
model: 63B classic · qualifier: approximately	5.1485 billion FLOPs per token	—	Not assessed
model: source · qualifier: approximately	3.0873 billion FLOPs per token	—	Not assessed
model: 67B SST · qualifier: approximately	5.4107 billion FLOPs per token	—	Not assessed
qualifier: approximation	3 factor	—	Not assessed
	4,096 tokens	—	Not assessed
	512 tokens	—	Not assessed
	128,896 rows	—	Not assessed

Open claim details

The evaluated SST route expands a 33.819B-parameter, 18-layer source into a 66.959B two-tower model by copying source layers into an 18-layer Prefiller and an 18-layer Decoder; the Prefiller produces aligned layer-wise KV, the Decoder reads it, and both towers are jointly trained after conversion.Whole claim: Awaiting reproduction0/21 measurements supported · 0 assessed21measurementsExplore results

Experiment plan ready; no runs yet.

Measurement results

0/21 assessed

0 supported0 disputed or conflicting21 not assessed

The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
model: source	33.819 billion parameters	—	Not assessed
model: 67B SST	66.959 billion parameters	—	Not assessed
model: 47B classic	46.727 billion parameters	—	Not assessed
model: 63B classic	62.691 billion parameters	—	Not assessed
model: source	18 layers	—	Not assessed
model: 67B SST	18 layers	—	Not assessed
model: 67B SST	18 layers	—	Not assessed
model: 47B classic	20 layers	—	Not assessed
model: 63B classic	22 layers	—	Not assessed
model: source	2,304 dimensions	—	Not assessed
model: 67B SST	2,304 dimensions	—	Not assessed
model: 47B classic	2,560 dimensions	—	Not assessed

Open claim details

SST's training loss rises sharply at source-to-SST conversion and then falls during joint continuation; the paper reports the direction from Figure 4 but does not assign an exact conversion-jump magnitude.Whole claim: Awaiting reproduction0/2 measurements supported · 0 assessed2measurementsExplore results

Experiment plan ready; no runs yet.

Measurement results

0/2 assessed

0 supported0 disputed or conflicting2 not assessed

The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
	167.98 billion cumulative training tokens	—	Not assessed
	0.301 fraction of B_ref	—	Not assessed

Open claim details

Appendix A reports the token-count and estimated-charge operands underlying the six OpenRouter ratios and uncached-input charge shares, with model-specific standard rates and half-price Batch rates.Whole claim: Awaiting reproduction0/37 measurements supported · 0 assessed37measurementsExplore results

Experiment plan ready; no runs yet.

Measurement results

0/37 assessed

0 supported0 disputed or conflicting37 not assessed

The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
model: GPT-6 Astra	356.199 billion tokens	—	Not assessed
model: GPT-6 Astra	2,232.435 billion tokens	—	Not assessed
model: GPT-6 Astra	24.323 billion tokens	—	Not assessed
model: GPT-6 Astra	3.56 million USD	—	Not assessed
model: GPT-6 Astra	1.2135 million USD	—	Not assessed
model: GPT-6 Astra Pro	45.611 billion tokens	—	Not assessed
model: GPT-6 Astra Pro	159.946 billion tokens	—	Not assessed
model: GPT-6 Astra Pro	3.255 billion tokens	—	Not assessed
model: GPT-6 Astra Pro	0.4554 million USD	—	Not assessed
model: GPT-6 Astra Pro	0.1613 million USD	—	Not assessed
model: Claude Fable 5	159.115 billion tokens	—	Not assessed
model: Claude Fable 5	460.103 billion tokens	—	Not assessed

Open claim details

The OpenRouter sample is selected, has incomplete and unequal day coverage, and is not separately labeled as agentic traffic; its charge estimates use a fixed price snapshot and omit several pricing effects, so it motivates rather than measures an agentic GPU-cost distribution.Whole claim: Awaiting reproduction0/4 measurements supported · 0 assessed4measurementsExplore results

Experiment plan ready; no runs yet.

Measurement results

0/4 assessed

0 supported0 disputed or conflicting4 not assessed

The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
models: GPT-6 Astra and GPT-6 Astra Pro	17 days	—	Not assessed
model: Gemini 3.8 Flash	19 days	—	Not assessed
models: Claude Fable 5, Claude Sonnet 5, and Gemini 3.7 Flash	30 days	—	Not assessed
model: Claude Fable 5	10 days	—	Not assessed

Open claim details

The compared models use matched sparse-expert and hybrid-attention patterns but different widths, heads, and body sizes; in SST only the Prefiller contains K/V projections and K normalization, while both towers have independent Transformer-block parameters.Whole claim: Awaiting reproduction0/26 measurements supported · 0 assessed26measurementsExplore results

Experiment plan ready; no runs yet.

Measurement results

0/26 assessed

0 supported0 disputed or conflicting26 not assessed

The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
models: source and each SST tower	4,608 dimensions	—	Not assessed
model: 47B classic	5,120 dimensions	—	Not assessed
model: 63B classic	5,632 dimensions	—	Not assessed
models: source and each SST tower	576 dimensions	—	Not assessed
models: source and each SST tower	576 dimensions	—	Not assessed
model: 47B classic	640 dimensions	—	Not assessed
model: 47B classic	640 dimensions	—	Not assessed
model: 63B classic	704 dimensions	—	Not assessed
model: 63B classic	704 dimensions	—	Not assessed
models: source, SST, 47B classic, 63B classic	128 dimensions	—	Not assessed
models: source, SST, 47B classic, 63B classic	64 dimensions	—	Not assessed
models: source, SST, 47B classic, 63B classic	128 dimensions	—	Not assessed

Open claim details

The evaluated SST checkpoint includes 167.98B source-stage tokens followed by 222.55B continuation tokens, for 390.54B cumulative tokens; continuation jointly trains both towers in BF16 and retains optimizer state for Prefiller/shared parameters while initializing fresh Decoder state.Whole claim: Awaiting reproduction0/20 measurements supported · 0 assessed20measurementsExplore results

Experiment plan ready; no runs yet.

Measurement results

0/20 assessed

0 supported0 disputed or conflicting20 not assessed

The complete result group stays with its claim. Assessed rows come first; their judgments do not replace the whole-claim conclusion. Expand a row for conditions, sources, and evidence history.

Each row is a measurement with its reported value, latest available observation, and cumulative evidence state.
Metric and conditions	Reported	Latest observation	Evidence judgment
model: source stage of SST	167.98 billion tokens	—	Not assessed
model: 67B SST continuation	222.55 billion tokens	—	Not assessed
model: 67B SST	390.54 billion tokens	—	Not assessed
	13,350 updates	—	Not assessed
	17,687 updates	—	Not assessed
	31,037 updates	—	Not assessed
gpu_type: NVIDIA H800	128 GPUs	—	Not assessed
	4,096 tokens	—	Not assessed
	3,072 sequences	—	Not assessed
models: source, 67B SST, and 63B classic	12,582,912 tokens	—	Not assessed
model: 47B classic	14,680,064 tokens	—	Not assessed
	6 steps	—	Not assessed

Open claim details

Qualitative findings

Increasing the classic baseline from 47B to 63B does not improve every task: the 63B classic score is lower on GSM8K (55.04 versus 56.56) and HumanEval few-shot (32.93 versus 35.98), although it is higher on the other reported evaluation columns.Experiment blocked; see the specific reason

The GSM8K and HumanEval directions have an explicit evaluation decision rule but require the same unavailable 47B and 63B final checkpoints and evaluator definitions.

Open claim details

Other claims1

The study evaluates one selected SST configuration and does not establish an optimal KITE connectivity pattern, a scaling law, a causal benefit from each ingredient, or a measured serving speedup.Stated by the authors

This is a limitation on inference from the represented experiments, not an independent reported outcome. No unreported connectivity ablation, scaling law, causal decomposition, or serving-speed result is fabricated.

Open claim details