← 研踪 PaperTrailarXiv:2605.10668 · v2
A Spectral Framework for Closed-Form Relative Density Estimation
首次提交 2026-05-11 22:51
版本更新 2026-11-16 21:35
雷达观测 2026-11-17 17:11
摘要
We propose a closed-form spectral framework for relative log-density estimation in linearly parameterized probabilistic models, including unnormalized and conditional models. This is achieved by representing the Kullback-Leibler (KL) divergence as an integral of weighted chi-squared divergences, converting KL estimation into a family of least-squares problems. We derive an explicit spectral formula based only on first- and second-order feature moments, yielding closed-form estimators of both divergences and log-density potentials for fixed features. The framework extends to a broad class of -divergences and can be combined with kernelization or feature learning with neural networks (with then only a closed form for the last layer). We prove convergence guarantees for the resulting estimators and empirically compare them on synthetic data with optimization-based variational formulations, including logistic and softmax regression for normalized conditional models.
作者备注:Published at Advances in Neural Processing Systems
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-16 21:35 · 最新观测版本
A Spectral Framework for Closed-Form Relative Density Estimation
We propose a closed-form spectral framework for relative log-density estimation in linearly parameterized probabilistic models, including unnormalized and conditional models. This is achieved by representing the Kullback-Leibler (KL) divergence as an integral of weighted chi-squared divergences, converting KL estimation into a family of least-squares problems. We derive an explicit spectral formula based only on first- and second-order feature moments, yielding closed-form estimators of both divergences and log-density potentials for fixed features. The framework extends to a broad class of -divergences and can be combined with kernelization or feature learning with neural networks (with then only a closed form for the last layer). We prove convergence guarantees for the resulting estimators and empirically compare them on synthetic data with optimization-based variational formulations, including logistic and softmax regression for normalized conditional models.
查看此版本解读 →查看此版本原文 ↗