Parameter-Free and Group Conditional Online Conformal Prediction
Beepul Bharti, Ambar Pal, Jacopo Teneggi, Jeremias Sulam
原文摘要
Quantifying the uncertainty in predictions made by machine learning models is challenging in online settings where the characteristics of the data and labels change over time. Online conformal prediction (OCP) methods tackle this problem by producing intervals for each observation and guaranteeing that a proportion of them contain their labels, a property known as marginal coverage. OCP methods, however, may systematically miscover observations belonging to groups defined by attributes like age or race, raising fairness concerns. Group-conditional OCP (G-OCP) methods address this by producing intervals that provide coverage for multiple groups. Existing approaches rely on learning rates, which control how they react to changes in the data stream, but can be hard to tune in online settings. Leveraging advancements in learning-rate-free methods, we develop a learning-rate-free G-OCP algorithm with strong group-coverage guarantees that hold under weaker assumptions than those required in prior work. Empirical results show that our method provides group coverage, produces interval lengths competitive with those of standard methods, and reacts more effectively to changes in the data than fixed-learning-rate methods.
stat.ML · cs.LG