← 研踪 PaperTrailarXiv:2611.23329 · v1
A level is not a rank: an empirical audit of quantile-to-index conversion in 10 released conformal-prediction libraries
首次提交 2026-11-17 03:00
版本更新 2026-11-17 03:00
雷达观测 2026-11-19 19:09
摘要
Context. A conformal interval at 0.90 keeps that guarantee only at the right rank: a fresh score falls at or below the rth smallest of n with probability r/(n + 1). The libraries we study instead hand a level to a quantile function, which can land elsewhere. At n = 20, the upper end of statsforecast's interval sits at position 18.05 where rank 19 is required. Objective. Where does installed code resolve the wrong rank, at what cost in coverage, and what does it return when no rank reaches the level? Method. We audit 39 sites where a level becomes an index in 10 pinned packages, a named set rather than a systematic search. 12 are classified by running them, 11 through their shared numpy method, the rest from source. Coverage is paired against the required rank on the same scores. Results. All 7 tabular implementations reach the required rank. In darts, on one code path, coverage is unchanged (0.0000) where the ranks agree and 0.0280 to 0.0560 lower where they differ; a second archive repeats that split but not its range. Below the feasibility floor, , of 14 public call paths 5 silently return a finite number; the rest raise or warn. Conclusions. Quantile interfaces take only a level; statsforecast's default calibrates below its floor. 6 of 19 upstream filings have merged, one fixing a coverage shortfall. A conformance suite classifies helpers by the rank they reach, and a checklist guides review.
作者备注:34 pages, 3 figures, 7 tables. Online Resource (45 pages) included as an ancillary file. Code, probe outputs and pinned environments: doi:10.5281/zenodo.21799203
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-17 03:00 · 最新观测版本
A level is not a rank: an empirical audit of quantile-to-index conversion in 10 released conformal-prediction libraries
Context. A conformal interval at 0.90 keeps that guarantee only at the right rank: a fresh score falls at or below the rth smallest of n with probability r/(n + 1). The libraries we study instead hand a level to a quantile function, which can land elsewhere. At n = 20, the upper end of statsforecast's interval sits at position 18.05 where rank 19 is required. Objective. Where does installed code resolve the wrong rank, at what cost in coverage, and what does it return when no rank reaches the level? Method. We audit 39 sites where a level becomes an index in 10 pinned packages, a named set rather than a systematic search. 12 are classified by running them, 11 through their shared numpy method, the rest from source. Coverage is paired against the required rank on the same scores. Results. All 7 tabular implementations reach the required rank. In darts, on one code path, coverage is unchanged (0.0000) where the ranks agree and 0.0280 to 0.0560 lower where they differ; a second archive repeats that split but not its range. Below the feasibility floor, , of 14 public call paths 5 silently return a finite number; the rest raise or warn. Conclusions. Quantile interfaces take only a level; statsforecast's default calibrates below its floor. 6 of 19 upstream filings have merged, one fixing a coverage shortfall. A conformance suite classifies helpers by the rank they reach, and a checklist guides review.
查看此版本解读 →查看此版本原文 ↗