跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.22434 · v1

Encoding, Decodability, and Use: A Five-Layer Framework for Benchmark Measurement Audits with Evidence from UAV Intent Recognition

首次提交 2026-11-17 17:08
版本更新 2026-11-17 17:08
雷达观测 2026-11-18 19:42
arXiv ↗PDF ↗

摘要

Benchmark accuracy is routinely read as evidence that a model understands the construct the benchmark measures. We propose and instantiate on AeroIntent a five-layer framework that decomposes that reading into observation, representation, decoding, use, and behaviour. The framework separates encoding, decodability, and use as distinct levels. In the instantiation, the observation layer is unreliable on 7 of the 20 Drone programs and floors the direct visual correlate of radial velocity at a 0.26\% scale change. The representation layer exposes two task-designed RGB features with nominal signals. RAFT flow radial bias reaches r=+0.257r=+0.257 (p=0.030p=0.030), and bounding-box optical-flow divergence reaches r=+0.203r=+0.203 (p=0.029p=0.029). No multi-feature readout gives a fold-mean test correlation above the cluster minimum detectable effect (MDE), and the largest positive global out-of-sample correlation is r=+0.080r=+0.080. The decoding layer sets the MDE at ∣r∣=0.599|r|=0.599 at the multi-feature Bonferroni threshold (α=0.00714α=0.00714, power 0.80), and the observed r=+0.080r=+0.080 has power 0.00. Two injection experiments establish the MDE's condition from both sides: aligned injections are detected at or above the simulated power, and orthogonal injections are not detected. The use layer finds accuracy consistent with program memory (program-identity 0.937/0.597/0.210) and multi-region static cues. The behaviour layer is consistent with accuracy not requiring a radial-velocity readout (appearance 0.6552 vs bounding-box trajectory 0.4080, cluster-level paired Δ=+0.2471Δ=+0.2471, p=0.0132p=0.0132).

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

目前只收录了一个版本。收录下一版后,可以对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-17 17:08 · 最新观测版本

Encoding, Decodability, and Use: A Five-Layer Framework for Benchmark Measurement Audits with Evidence from UAV Intent Recognition

Benchmark accuracy is routinely read as evidence that a model understands the construct the benchmark measures. We propose and instantiate on AeroIntent a five-layer framework that decomposes that reading into observation, representation, decoding, use, and behaviour. The framework separates encoding, decodability, and use as distinct levels. In the instantiation, the observation layer is unreliable on 7 of the 20 Drone programs and floors the direct visual correlate of radial velocity at a 0.26\% scale change. The representation layer exposes two task-designed RGB features with nominal signals. RAFT flow radial bias reaches r=+0.257r=+0.257 (p=0.030p=0.030), and bounding-box optical-flow divergence reaches r=+0.203r=+0.203 (p=0.029p=0.029). No multi-feature readout gives a fold-mean test correlation above the cluster minimum detectable effect (MDE), and the largest positive global out-of-sample correlation is r=+0.080r=+0.080. The decoding layer sets the MDE at ∣r∣=0.599|r|=0.599 at the multi-feature Bonferroni threshold (α=0.00714α=0.00714, power 0.80), and the observed r=+0.080r=+0.080 has power 0.00. Two injection experiments establish the MDE's condition from both sides: aligned injections are detected at or above the simulated power, and orthogonal injections are not detected. The use layer finds accuracy consistent with program memory (program-identity 0.937/0.597/0.210) and multi-region static cues. The behaviour layer is consistent with accuracy not requiring a radial-velocity readout (appearance 0.6552 vs bounding-box trajectory 0.4080, cluster-level paired Δ=+0.2471Δ=+0.2471, p=0.0132p=0.0132).

查看此版本解读 →查看此版本原文 ↗