跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2609.28559 · v2

Who Is Behind the Harness? Verifying Model Identity in Coding Agents through Behavioral Fingerprinting

首次提交 2026-09-23 17:12
版本更新 2026-11-18 21:37
雷达观测 2026-11-19 15:04
arXiv ↗PDF ↗

摘要

LLMs increasingly operate through coding-agent harnesses that inspect repositories, invoke tools, and modify files. Substituting the model behind such an agent can therefore change security-relevant decisions, including whether it verifies changes or recovers safely from failures. Existing LLM fingerprints largely infer identity from direct text or token distributions. In coding agents, these signals are mediated by system instructions, controller logic, tools, and execution feedback, limiting their transfer. We present LIDAR (LLM Identification from Decisions and Actions at Runtime), an active black-box fingerprinting method for coding-agent execution. Three coding probe pairs expose post-edit verification, transient-failure recovery, and specification--test conflict resolution under controlled changes. LIDAR represents the resulting trajectories with complementary instance-level and distribution-level features and compares them with clean references using a lightweight probabilistic identifier. It requires no access to model weights, logits, or provider internals. We evaluate 24 models from eight families on OpenCode, OpenHands, and mini-swe-agent, fitting separate references for each harness. Closed-set Top-1 accuracy ranges from 91.39\% to 98.89\% across the three harnesses, exceeding all three baselines in each setting. When the audit also includes unregistered models from the same families, balanced accuracy ranges from 79.58\% to 80.42\%. These results show that agent execution behavior provides model-identity evidence beyond final outputs.

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v2,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-18 21:37 · 最新观测版本

Who Is Behind the Harness? Verifying Model Identity in Coding Agents through Behavioral Fingerprinting

LLMs increasingly operate through coding-agent harnesses that inspect repositories, invoke tools, and modify files. Substituting the model behind such an agent can therefore change security-relevant decisions, including whether it verifies changes or recovers safely from failures. Existing LLM fingerprints largely infer identity from direct text or token distributions. In coding agents, these signals are mediated by system instructions, controller logic, tools, and execution feedback, limiting their transfer. We present LIDAR (LLM Identification from Decisions and Actions at Runtime), an active black-box fingerprinting method for coding-agent execution. Three coding probe pairs expose post-edit verification, transient-failure recovery, and specification--test conflict resolution under controlled changes. LIDAR represents the resulting trajectories with complementary instance-level and distribution-level features and compares them with clean references using a lightweight probabilistic identifier. It requires no access to model weights, logits, or provider internals. We evaluate 24 models from eight families on OpenCode, OpenHands, and mini-swe-agent, fitting separate references for each harness. Closed-set Top-1 accuracy ranges from 91.39\% to 98.89\% across the three harnesses, exceeding all three baselines in each setting. When the audit also includes unregistered models from the same families, balanced accuracy ranges from 79.58\% to 80.42\%. These results show that agent execution behavior provides model-identity evidence beyond final outputs.

查看此版本解读 →查看此版本原文 ↗