← 研踪 PaperTrailarXiv:2611.21509 · v1
Almost Controlled: The FDR of Benjamini-Hochberg on Weighted Conformal p-Values
几乎受控:加权保形p值上Benjamini-Hochberg方法的FDR
首次提交 2026-11-17 02:07
版本更新 2026-11-17 02:07
雷达观测 2026-11-17 16:10
摘要
Weighted conformal p-values generalize conformal inference to cases with covariate shift. One way to use them is to pass them to the Benjamini-Hochberg (BH) procedure, either to decide which test units to select or to detect outliers. But weighted conformal p-values may not be positively dependent, so we want to know: does BH still control the false discovery rate (FDR) in finite samples? Based on experiments, it appears to, but we prove it doesn't. We give two counterexamples, certified in exact rational arithmetic, where the FDR exceeds its target level . We also identify how large the difference between the FDR and can be. With calibration points, test units and weight ratio , the relative excess is bounded by a function of , , and that is of order when is small. The bound holds for every configuration with and every . This order can be reached when multiple non-null test units share a common threshold: in a Poisson-limit model, that configuration's excess is to first order as and then . For some fixed distributions with this structure, the excess decreases only like as grows. If every test unit is null, the excess is at fixed , and in certain configurations it grows linearly with . To assist practitioners in judging how large the FDR can be, we turn that bound, which depends only on , , and , into a lowered BH level with proved finite-sample control, and give a much less conservative rule of thumb that covers every worst case we were able to find but provides no guarantee.
作者备注:55 pages, 3 figures, 1 table. Main text 10 pages; full proofs in the appendices
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-17 02:07 · 最新观测版本
Almost Controlled: The FDR of Benjamini-Hochberg on Weighted Conformal p-Values
Weighted conformal p-values generalize conformal inference to cases with covariate shift. One way to use them is to pass them to the Benjamini-Hochberg (BH) procedure, either to decide which test units to select or to detect outliers. But weighted conformal p-values may not be positively dependent, so we want to know: does BH still control the false discovery rate (FDR) in finite samples? Based on experiments, it appears to, but we prove it doesn't. We give two counterexamples, certified in exact rational arithmetic, where the FDR exceeds its target level . We also identify how large the difference between the FDR and can be. With calibration points, test units and weight ratio , the relative excess is bounded by a function of , , and that is of order when is small. The bound holds for every configuration with and every . This order can be reached when multiple non-null test units share a common threshold: in a Poisson-limit model, that configuration's excess is to first order as and then . For some fixed distributions with this structure, the excess decreases only like as grows. If every test unit is null, the excess is at fixed , and in certain configurations it grows linearly with . To assist practitioners in judging how large the FDR can be, we turn that bound, which depends only on , , and , into a lowered BH level with proved finite-sample control, and give a much less conservative rule of thumb that covers every worst case we were able to find but provides no guarantee.
查看此版本解读 →查看此版本原文 ↗