Quantification of Tail-trimming Effects for Interval CVaR Based Nonlinear Regression
Yulei You, Junyi Liu
AI 速读 基于摘要
该文从理论上量化了In-CVaR风险度量对损失右尾的截断在非线性回归抗异常值中的必要性与有效性。
math.OC · cs.LG · stat.ML
查看原文摘要
The tail trimming strategy is widely adopted for robustness against outlier contamination in statistical learning and operations research. However, it generally lacks the thorough understanding that to what extent the tail-trimming technique is effective for nonlinear regression. In this paper, we address the robust nonlinear regression based on the so-called interval conditional value-at-risk (In-CVaR), which is introduced to guard against outliers by trimming extreme losses. We first analyze the trimming effect of the In-CVaR based estimator under distributional perturbation. Our first finding is that the In-CVaR based estimator is qualitatively robust under distributional perturbation in terms of the Prokhorov metric if and only if the largest portion of losses is trimmed. Furthermore, under sample contamination, we derive a probably approximately correct (PAC) bound for the In-CVaR based estimator, showing that the preliminary covariate filtering that is customary in the literature is not needed for In-CVaR based regression. The nonconvexity of the In-CVaR objective, together with unboundedness of covariates, responses and parameter space create the technical challenges for our developed theory, and thus necessitate the new treatment of the qualitative robustness and PAC analysis. Overall, this study rigorously analyzes robustness properties of In-CVaR based nonlinear regression models, which illustrates to what extent the right tail-trimming of the loss by In-CVaR risk measure is essential and effective for robust nonlinear regression. These theoretical findings are further supported by numerical experiments.
Medical segmentation foundation models such as SAM and MedSAM provide strong prompt-driven segmentation, but their image encoders are still too large for many clinical settings. Compression is also risky in medicine because a model can keep high Dice while losing boundary fidelity. We propose MedCore, a structured pruning framework for MedSAM. The main idea is to preserve two kinds of structures: structures that became important during SAM-to-MedSAM adaptation, and structures that have high boundary leverage. We identify the first type by a dual-intervention score comparing zeroing a group with resetting it to its original SAM weights. We identify the second type by boundary-aware Fisher estimation. We also introduce a boundary leverage principle, which relates first-order boundary displacement to boundary logit perturbation divided by the spatial gradient norm. This principle explains why boundary metrics can degrade even when Dice remains high. On polyp segmentation benchmarks, MedCore reduces parameters by 60.0% and FLOPs by 58.4% while achieving Dice 0.9549, Boundary F1 0.6388, and HD95 5.14 after post-pruning fine-tuning. It also reaches 86.6% parameter reduction and 90.4G FLOPs, retaining Dice and HD95 near the fine-tuned MedSAM reference, with lower BF1. Our analysis shows that MedSAM lies in a head-fragile boundary regime: the median 95th-percentile boundary leverage is 2.887 times as high for head-pruning as for MLP-pruning steps, consistent with BF1 and HD95 degradation. Our code is available at https://github.com/cenweizhang/MedCore.