← 研踪 PaperTrailarXiv:2608.06004 · v2
Do Tabular Foundation Models Agree with Themselves?
首次提交 2026-08-06 21:09
版本更新 2026-11-04 18:21
雷达观测 2026-11-05 11:10
摘要
Tabular Foundation Models (TFMs) are the state of the art for many tabular prediction tasks. They are transformers that approximate the Bayesian posterior predictive distribution under a pretraining prior. TFMs become multivariate predictors autoregressively, by sampling one target and appending it to the features. This paper is the first to evaluate whether this setup induces a valid stochastic process. Informally, we ask: *Do we get the same answer to the same question asked in two different ways?* The standard Kolmogorov consistency conditions cannot be checked on TFMs directly, since TFMs are unable to model certain conditional distributions. We therefore propose two adapted, testable requirements. Marginalization consistency demands that marginalized conditionals equal directly predicted marginals, and factorization consistency that both factorization orders yield the same joint. Since the data-generating distribution satisfies both, a perfect TFM would as well, and every violation lower-bounds how far the model's predictions are from the ground truth. Every TFM we evaluate violates both requirements, for classification and regression, on all datasets.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-04 18:21 · 最新观测版本
Do Tabular Foundation Models Agree with Themselves?
Tabular Foundation Models (TFMs) are the state of the art for many tabular prediction tasks. They are transformers that approximate the Bayesian posterior predictive distribution under a pretraining prior. TFMs become multivariate predictors autoregressively, by sampling one target and appending it to the features. This paper is the first to evaluate whether this setup induces a valid stochastic process. Informally, we ask: *Do we get the same answer to the same question asked in two different ways?* The standard Kolmogorov consistency conditions cannot be checked on TFMs directly, since TFMs are unable to model certain conditional distributions. We therefore propose two adapted, testable requirements. Marginalization consistency demands that marginalized conditionals equal directly predicted marginals, and factorization consistency that both factorization orders yield the same joint. Since the data-generating distribution satisfies both, a perfect TFM would as well, and every violation lower-bounds how far the model's predictions are from the ground truth. Every TFM we evaluate violates both requirements, for classification and regression, on all datasets.
查看此版本解读 →查看此版本原文 ↗