← 研踪 PaperTrailarXiv:2609.09910 · v2
Structure and Regularization of Fused Objectives in Kernel Discriminant Analysis for Small-Sample-Size Problems
首次提交 2026-09-09 17:08
版本更新 2026-10-27 21:40
雷达观测 2026-10-29 00:35
摘要
When the number of training samples is smaller than the dimension, discriminant analysis faces a singular within-class scatter matrix, and fused objectives that combine the total variance C, the between-class scatter S_b and the within-class scatter S_w are one way to handle it. Using the seven variants of Kernelized Linear Principal Component Discriminant Analysis (KLPCDA) as a platform, we derive the structure of such objectives in this small-sample-size (SSS) setting. The training projections split into a class-mean part and a within-class part, the two scatter operators satisfy WB = 0, and all between-class information lies in the null space of S_w. A single fusion coefficient therefore cannot change the solution, and the variants, KPCA, the regularized kernel Fisher discriminant and the null space methods lie on the boundary of one family with two parameters, the fusion ratio t and a ridge parameter mu. On six tasks from five application domains, the regularized members were more accurate than the best of the seven variants chosen on validation data on all six tasks, by 0.8 to 5.7 points, also under added noise, whereas the fusion ratio gave no gain, including in controlled experiments on dimension ratio, class imbalance and class separation. Selecting a variant per task could gain less than one point, while a fixed choice made on other tasks transferred to every held-out task. The analysis thus gives a principled default for SSS kernel discriminant analysis, with diagnostic quantities, the null space dimension and the class-size spectrum, that can be computed before training.
作者备注：Revised version. Supplementary material merged into the main file
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-10-27 21:40 · 最新观测版本
Structure and Regularization of Fused Objectives in Kernel Discriminant Analysis for Small-Sample-Size Problems
When the number of training samples is smaller than the dimension, discriminant analysis faces a singular within-class scatter matrix, and fused objectives that combine the total variance C, the between-class scatter S_b and the within-class scatter S_w are one way to handle it. Using the seven variants of Kernelized Linear Principal Component Discriminant Analysis (KLPCDA) as a platform, we derive the structure of such objectives in this small-sample-size (SSS) setting. The training projections split into a class-mean part and a within-class part, the two scatter operators satisfy WB = 0, and all between-class information lies in the null space of S_w. A single fusion coefficient therefore cannot change the solution, and the variants, KPCA, the regularized kernel Fisher discriminant and the null space methods lie on the boundary of one family with two parameters, the fusion ratio t and a ridge parameter mu. On six tasks from five application domains, the regularized members were more accurate than the best of the seven variants chosen on validation data on all six tasks, by 0.8 to 5.7 points, also under added noise, whereas the fusion ratio gave no gain, including in controlled experiments on dimension ratio, class imbalance and class separation. Selecting a variant per task could gain less than one point, while a fixed choice made on other tasks transferred to every held-out task. The analysis thus gives a principled default for SSS kernel discriminant analysis, with diagnostic quantities, the null space dimension and the class-size spectrum, that can be computed before training.
查看此版本解读 →查看此版本原文 ↗