SkillSonar generalizes zero-shot to external safety benchmarks, achieving the lowest ASR of 52.90% on SkillSafetyBench and highest safety score of 54.00% on WildClawBench compared to prompt, permission, and allowlist baselines.整组结论：实验受阻，详见具体原因0/9 项指标获得支持 · 0 项已有评估9项测量结果展开结果表

No experiment has been scheduled for this empirical claim; feasibility remains to be established.

测量结果

已评估 0/9 项

获得支持 0有争议或冲突 0尚未评估 9

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
defense: No-guard · benchmark: SkillSafetyBench	93.3 percentage_points	—	尚未评估
defense: No-guard · benchmark: WildClawBench	38 percentage_points	—	尚未评估
defense: AcceptEdits · benchmark: WildClawBench	41 percentage_points	—	尚未评估
defense: AcceptEdits + allowlist · benchmark: SkillSafetyBench	64.3 percentage_points	—	尚未评估
defense: AcceptEdits + allowlist · benchmark: WildClawBench	45 percentage_points	—	尚未评估
defense: System prompt · benchmark: SkillSafetyBench	83.3 percentage_points	—	尚未评估
defense: System prompt · benchmark: WildClawBench	43 percentage_points	—	尚未评估
defense: SkillSonar (ours) · benchmark: SkillSafetyBench	52.9 percentage_points	—	尚未评估
defense: SkillSonar (ours) · benchmark: WildClawBench	54 percentage_points	—	尚未评估

打开 Claim 详情

On benign tasks containing no malicious skill behavior, SkillSonar preserves high utility on Claude Code (0.817 on GLM-5, 0.836 on Claude Haiku 4.5, and 0.724 on GPT-5.4), demonstrating minimal false-positive disruption of safe execution.整组结论：实验受阻，详见具体原因0/15 项指标获得支持 · 0 项已有评估15项测量结果展开结果表

No experiment has been scheduled for this empirical claim; feasibility remains to be established.

测量结果

已评估 0/15 项

获得支持 0有争议或冲突 0尚未评估 15

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
model: GLM-5 · defense: No-guard	0.815 score	—	尚未评估
model: Claude Haiku 4.5 · defense: No-guard	0.827 score	—	尚未评估
model: GPT-5.4 · defense: No-guard	0.793 score	—	尚未评估
model: GLM-5 · defense: System prompt	0.833 score	—	尚未评估
model: Claude Haiku 4.5 · defense: System prompt	0.788 score	—	尚未评估
model: GPT-5.4 · defense: System prompt	0.836 score	—	尚未评估
model: GLM-5 · defense: AcceptEdits	0.796 score	—	尚未评估
model: Claude Haiku 4.5 · defense: AcceptEdits	0.813 score	—	尚未评估
model: GPT-5.4 · defense: AcceptEdits	0.74 score	—	尚未评估
model: GLM-5 · defense: AcceptEdits + allowlist	0.792 score	—	尚未评估
model: Claude Haiku 4.5 · defense: AcceptEdits + allowlist	0.819 score	—	尚未评估
model: GPT-5.4 · defense: AcceptEdits + allowlist	0.764 score	—	尚未评估

打开 Claim 详情

Over five matched evaluation rounds, SkillSonar reduces ASR across all individual risk families, achieving reductions of 50.0 pp on Resource & Reliability, 42.9 pp on Execution Safety, 35.3 pp on Specification Integrity, 18.3 pp on Operational Side Effects, 61.5 pp on Privacy & Data Flow (OOD), and 44.8 pp on Capability Control (OOD).整组结论：实验受阻，详见具体原因0/18 项指标获得支持 · 0 项已有评估18项测量结果展开结果表

No experiment has been scheduled for this empirical claim; feasibility remains to be established.

测量结果

已评估 0/18 项

获得支持 0有争议或冲突 0尚未评估 18

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
split: id_test · defense: No Guard · risk_family: Resource & Reliability	61.3 percentage_points	—	尚未评估
split: id_test · defense: SkillSonar · risk_family: Resource & Reliability	11.3 percentage_points	—	尚未评估
split: id_test · risk_family: Resource & Reliability	50 percentage_points	—	尚未评估
split: id_test · defense: No Guard · risk_family: Execution Safety	48.6 percentage_points	—	尚未评估
split: id_test · defense: SkillSonar · risk_family: Execution Safety	5.7 percentage_points	—	尚未评估
split: id_test · risk_family: Execution Safety	42.9 percentage_points	—	尚未评估
split: id_test · defense: No Guard · risk_family: Specification Integrity	41.2 percentage_points	—	尚未评估
split: id_test · defense: SkillSonar · risk_family: Specification Integrity	5.9 percentage_points	—	尚未评估
split: id_test · risk_family: Specification Integrity	35.3 percentage_points	—	尚未评估
split: id_test · defense: No Guard · risk_family: Operational Side Effects	36.7 percentage_points	—	尚未评估
split: id_test · defense: SkillSonar · risk_family: Operational Side Effects	18.3 percentage_points	—	尚未评估
split: id_test · risk_family: Operational Side Effects	18.3 percentage_points	—	尚未评估

打开 Claim 详情

Packaging the safety policy as a modular skill-native artifact provides substantial gains over flattening the identical policy into a system prompt: SkillSonar reduces ID ASR from 0.353 to 0.104 and OOD ASR from 0.437 to 0.109, increases ID task utility from 0.655 to 0.815, and lowers token consumption from ~239K to ~188K (a ~21% reduction).整组结论：实验受阻，详见具体原因0/12 项指标获得支持 · 0 项已有评估12项测量结果展开结果表

No experiment has been scheduled for this empirical claim; feasibility remains to be established.

测量结果

已评估 0/12 项

获得支持 0有争议或冲突 0尚未评估 12

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
split: id_test · configuration: Flattened prompt	0.353 fraction	—	尚未评估
split: ood_test · configuration: Flattened prompt	0.437 fraction	—	尚未评估
split: id_test · configuration: SkillSonar	0.104 fraction	—	尚未评估
split: ood_test · configuration: SkillSonar	0.109 fraction	—	尚未评估
split: id_test · configuration: Flattened prompt	0.655 score	—	尚未评估
split: ood_test · configuration: Flattened prompt	0.672 score	—	尚未评估
split: id_test · configuration: SkillSonar	0.815 score	—	尚未评估
split: ood_test · configuration: SkillSonar	0.789 score	—	尚未评估
configuration: Flattened prompt	0.748 score	—	尚未评估
configuration: SkillSonar	0.763 score	—	尚未评估
configuration: Flattened prompt	239 thousands_of_tokens	—	尚未评估
configuration: SkillSonar	188 thousands_of_tokens	—	尚未评估

打开 Claim 详情

Compared to action-level runtime defenses AgentSpec and TS-Guard, SkillSonar achieves superior safety–utility operating points on Claude Code with GLM-5: AgentSpec achieves 0.294 ID ASR but drops task utility to 0.581, while TS-Guard achieves 0.412 ID ASR, whereas SkillSonar reaches 0.104 ID ASR with 0.815 task utility.整组结论：实验受阻，详见具体原因0/10 项指标获得支持 · 0 项已有评估10项测量结果展开结果表

No experiment has been scheduled for this empirical claim; feasibility remains to be established.

测量结果

已评估 0/10 项

获得支持 0有争议或冲突 0尚未评估 10

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
split: id_test · defense: AgentSpec	0.294 fraction	—	尚未评估
split: ood_test · defense: AgentSpec	0.576 fraction	—	尚未评估
split: id_test · defense: AgentSpec	0.581 score	—	尚未评估
defense: AgentSpec	0.642 score	—	尚未评估
split: id_test · defense: TS-Guard	0.412 fraction	—	尚未评估
split: ood_test · defense: TS-Guard	0.492 fraction	—	尚未评估
defense: TS-Guard	0.703 score	—	尚未评估
split: id_test · defense: No-guard	0.477 fraction	—	尚未评估
split: id_test · defense: No-guard	0.829 score	—	尚未评估
defense: No-guard	0.757 score	—	尚未评估

打开 Claim 详情

Human audit of the LLM judges confirms high reliability: the data construction judge achieves 94% balanced audit agreement (88% success precision, 100% failure confirmation) and the final evaluation judge achieves 98% balanced agreement (96% success precision, 100% failure confirmation).整组结论：实验受阻，详见具体原因0/6 项指标获得支持 · 0 项已有评估6项测量结果展开结果表

No experiment has been scheduled for this empirical claim; feasibility remains to be established.

测量结果

已评估 0/6 项

获得支持 0有争议或冲突 0尚未评估 6

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
stage: Data construction	88 percentage_points	—	尚未评估
stage: Data construction	100 percentage_points	—	尚未评估
stage: Data construction	94 percentage_points	—	尚未评估
stage: Final evaluation	96 percentage_points	—	尚未评估
stage: Final evaluation	100 percentage_points	—	尚未评估
stage: Final evaluation	98 percentage_points	—	尚未评估

打开 Claim 详情

Under an adaptive attacker performing three iterations of targeted adversarial skill refinement against the final guard, SkillSonar reduces overall ASR from 45.1% to 22.9% (-22.2 percentage points), with reductions across all six individual risk families.整组结论：实验受阻，详见具体原因0/21 项指标获得支持 · 0 项已有评估21项测量结果展开结果表

No experiment has been scheduled for this empirical claim; feasibility remains to be established.

测量结果

已评估 0/21 项

获得支持 0有争议或冲突 0尚未评估 21

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
defense: No Guard · risk_family: Capability Control	55 percentage_points	—	尚未评估
defense: SkillSonar · risk_family: Capability Control	26.7 percentage_points	—	尚未评估
risk_family: Capability Control	-28.3 percentage_points	—	尚未评估
defense: No Guard · risk_family: Specification Integrity	53.3 percentage_points	—	尚未评估
defense: SkillSonar · risk_family: Specification Integrity	43.3 percentage_points	—	尚未评估
risk_family: Specification Integrity	-10 percentage_points	—	尚未评估
defense: No Guard · risk_family: Resource & Reliability	45 percentage_points	—	尚未评估
defense: SkillSonar · risk_family: Resource & Reliability	26.7 percentage_points	—	尚未评估
risk_family: Resource & Reliability	-18.3 percentage_points	—	尚未评估
defense: No Guard · risk_family: Privacy & Data Flow	43.3 percentage_points	—	尚未评估
defense: SkillSonar · risk_family: Privacy & Data Flow	8.3 percentage_points	—	尚未评估
risk_family: Privacy & Data Flow	-35 percentage_points	—	尚未评估

打开 Claim 详情

Optimized SkillSonar guards exhibit bidirectional zero-shot transfer across distinct model families without target-model fine-tuning: the GLM-5-optimized guard reduces Kimi-K2.6 ASR to 0.199 (ID) and 0.161 (OOD), while the Kimi-K2.6-optimized guard achieves 0.112 ID ASR on GLM-5, 0.059 on Claude Haiku 4.5, and 0.019 on GPT-5.4.整组结论：实验受阻，详见具体原因0/7 项指标获得支持 · 0 项已有评估7项测量结果展开结果表

No experiment has been scheduled for this empirical claim; feasibility remains to be established.

测量结果

已评估 0/7 项

获得支持 0有争议或冲突 0尚未评估 7

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
split: id_test · source_model: GLM-5 · target_model: Kimi-K2.6	0.199 fraction	—	尚未评估
split: ood_test · source_model: GLM-5 · target_model: Kimi-K2.6	0.161 fraction	—	尚未评估
split: id_test · source_model: Kimi-K2.6 · target_model: GLM-5	0.112 fraction	—	尚未评估
split: id_test · source_model: Kimi-K2.6 · target_model: Claude Haiku 4.5	0.059 fraction	—	尚未评估
split: id_test · source_model: Kimi-K2.6 · target_model: GPT-5.4	0.019 fraction	—	尚未评估
split: id_test · source_model: GLM-5 · target_model: Kimi-K2.6	0.799 score	—	尚未评估
split: ood_test · source_model: GLM-5 · target_model: Kimi-K2.6	0.799 score	—	尚未评估

打开 Claim 详情

In the OpenClaw agent environment, SkillSonar operates without modifying host runtime mechanics, reducing GLM-5 ASR to 0.245 (ID) and 0.232 (OOD), and Claude Haiku 4.5 ASR to 0.265 (ID) and 0.259 (OOD), demonstrating cross-harness portability.整组结论：实验受阻，详见具体原因0/30 项指标获得支持 · 0 项已有评估30项测量结果展开结果表

No experiment has been scheduled for this empirical claim; feasibility remains to be established.

测量结果

已评估 0/30 项

获得支持 0有争议或冲突 0尚未评估 30

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
model: GLM-5 · split: id_test · defense: No-guard · environment: OpenClaw	0.596 fraction	—	尚未评估
model: GLM-5 · split: ood_test · defense: No-guard · environment: OpenClaw	0.621 fraction	—	尚未评估
model: GLM-5 · split: id_test · defense: No-guard · environment: OpenClaw	0.893 score	—	尚未评估
model: GLM-5 · split: ood_test · defense: No-guard · environment: OpenClaw	0.92 score	—	尚未评估
model: GLM-5 · defense: No-guard · environment: OpenClaw	102,398 tokens	—	尚未评估
model: GLM-5 · split: id_test · defense: System prompt · environment: OpenClaw	0.577 fraction	—	尚未评估
model: GLM-5 · split: ood_test · defense: System prompt · environment: OpenClaw	0.453 fraction	—	尚未评估
model: GLM-5 · split: id_test · defense: System prompt · environment: OpenClaw	0.807 score	—	尚未评估
model: GLM-5 · split: ood_test · defense: System prompt · environment: OpenClaw	0.835 score	—	尚未评估
model: GLM-5 · defense: System prompt · environment: OpenClaw	93,352 tokens	—	尚未评估
model: GLM-5 · split: id_test · defense: SkillSonar (ours) · environment: OpenClaw	0.245 fraction	—	尚未评估
model: GLM-5 · split: ood_test · defense: SkillSonar (ours) · environment: OpenClaw	0.232 fraction	—	尚未评估

打开 Claim 详情

Explicit safety responsibility assignment is necessary for effective runtime guarding: without explicit invocation, ordinary skill selection yields high ASR (0.400 ID, 0.582 OOD), whereas explicit invocation reduces ASR to 0.104 ID and 0.109 OOD while maintaining comparable benign utility (0.756 vs. 0.767).整组结论：实验受阻，详见具体原因0/6 项指标获得支持 · 0 项已有评估6项测量结果展开结果表

No experiment has been scheduled for this empirical claim; feasibility remains to be established.

测量结果

已评估 0/6 项

获得支持 0有争议或冲突 0尚未评估 6

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
split: id_test · configuration: No explicit invocation	0.4 fraction	—	尚未评估
split: ood_test · configuration: No explicit invocation	0.582 fraction	—	尚未评估
split: id_test · configuration: Explicit invocation	0.104 fraction	—	尚未评估
split: ood_test · configuration: Explicit invocation	0.109 fraction	—	尚未评估
configuration: No explicit invocation	0.756 score	—	尚未评估
configuration: Explicit invocation	0.767 score	—	尚未评估

打开 Claim 详情

During Monte Carlo Tree Search guard evolution, full-evaluation malicious ASR decreases from 0.300 at iteration 1 to 0.078 by iteration 9, demonstrating that feedback-driven refinement effectively optimizes guard policy efficacy.整组结论：实验受阻，详见具体原因0/2 项指标获得支持 · 0 项已有评估2项测量结果展开结果表

No experiment has been scheduled for this empirical claim; feasibility remains to be established.

测量结果

已评估 0/2 项

获得支持 0有争议或冲突 0尚未评估 2

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
iteration: 1	0.3 fraction	—	尚未评估
iteration: 9	0.078 fraction	—	尚未评估

打开 Claim 详情

On Claude Haiku 4.5 and GPT-5.4, SkillSonar generalizes zero-shot across model architectures, reducing ID/OOD ASR to 0.096/0.254 on Claude Haiku 4.5 and 0.019/0.034 on GPT-5.4, while preserving higher utility than default AcceptEdits.整组结论：实验受阻，详见具体原因0/50 项指标获得支持 · 0 项已有评估50项测量结果展开结果表

No experiment has been scheduled for this empirical claim; feasibility remains to be established.

测量结果

已评估 0/50 项

获得支持 0有争议或冲突 0尚未评估 50

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
model: Claude Haiku 4.5 · split: id_test · defense: No-guard	0.481 fraction	—	尚未评估
model: Claude Haiku 4.5 · split: ood_test · defense: No-guard	0.707 fraction	—	尚未评估
model: Claude Haiku 4.5 · split: id_test · defense: No-guard	0.764 score	—	尚未评估
model: Claude Haiku 4.5 · split: ood_test · defense: No-guard	0.737 score	—	尚未评估
model: Claude Haiku 4.5 · defense: No-guard	179,918 tokens	—	尚未评估
model: Claude Haiku 4.5 · split: id_test · defense: System prompt	0.451 fraction	—	尚未评估
model: Claude Haiku 4.5 · split: ood_test · defense: System prompt	0.508 fraction	—	尚未评估
model: Claude Haiku 4.5 · split: id_test · defense: System prompt	0.661 score	—	尚未评估
model: Claude Haiku 4.5 · split: ood_test · defense: System prompt	0.662 score	—	尚未评估
model: Claude Haiku 4.5 · defense: System prompt	160,672 tokens	—	尚未评估
model: Claude Haiku 4.5 · split: id_test · defense: AcceptEdits	0.118 fraction	—	尚未评估
model: Claude Haiku 4.5 · split: ood_test · defense: AcceptEdits	0.051 fraction	—	尚未评估

打开 Claim 详情

On repeated runtime evaluation over N=10 runs with GLM-5 on Claude Code, SkillSonar reduces ID ASR from 0.482 to 0.104 and OOD ASR from 0.606 to 0.115 while maintaining 0.779 ID and 0.715 OOD task utility, significantly outperforming system-prompt and permission-based baselines.整组结论：实验受阻，详见具体原因0/25 项指标获得支持 · 0 项已有评估25项测量结果展开结果表

No experiment has been scheduled for this empirical claim; feasibility remains to be established.

测量结果

已评估 0/25 项

获得支持 0有争议或冲突 0尚未评估 25

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
split: id_test · defense: No-guard	0.482 fraction	—	尚未评估
split: ood_test · defense: No-guard	0.606 fraction	—	尚未评估
split: id_test · defense: No-guard	0.811 score	—	尚未评估
split: ood_test · defense: No-guard	0.839 score	—	尚未评估
defense: No-guard	179.3 thousands_of_tokens	—	尚未评估
split: id_test · defense: System prompt	0.414 fraction	—	尚未评估
split: ood_test · defense: System prompt	0.489 fraction	—	尚未评估
split: id_test · defense: System prompt	0.783 score	—	尚未评估
split: ood_test · defense: System prompt	0.807 score	—	尚未评估
defense: System prompt	175.6 thousands_of_tokens	—	尚未评估
split: id_test · defense: AcceptEdits	0.122 fraction	—	尚未评估
split: ood_test · defense: AcceptEdits	0.073 fraction	—	尚未评估

打开 Claim 详情

SCOPE-R contains 206 attack-confirmed malicious skill instances partitioned into 95 train instances (46.1%), 52 ID-test instances (25.2%), and 59 OOD-test instances (28.6%), with Capability Control and Privacy & Data Flow entirely held out from training.整组结论：实验受阻，详见具体原因0/8 项指标获得支持 · 0 项已有评估8项测量结果展开结果表

No experiment has been scheduled for this empirical claim; feasibility remains to be established.

测量结果

已评估 0/8 项

获得支持 0有争议或冲突 0尚未评估 8

保留同一结论下的完整结果组。已有评估排在前面；单项判断不替代整组结论。展开行查看条件、来源与历次证据。

每行是一项测量，分别显示论文报告、最近可用实测和累计证据状态。
指标与条件	论文报告	最近可用实测	证据判断
split: train	95 count	—	尚未评估
split: train	46.1 percentage_points	—	尚未评估
split: id_test	52 count	—	尚未评估
split: id_test	25.2 percentage_points	—	尚未评估
split: ood_test	59 count	—	尚未评估
split: ood_test	28.6 percentage_points	—	尚未评估
split: total	206 count	—	尚未评估
split: total	100 percentage_points	—	尚未评估

打开 Claim 详情

Defense-as-Skill: Evolving Runtime Guard Skill for Skill-Augmented Agents

研究结论

复现结果

尚无复现结果

关键指标论文报告值 → 复现观测值 · 点击行展开详情

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果

测量结果