Domain Generalization (DG) aims to learn representations robust to distribution shifts. Recent geometric alignment methods, such as CPCANet, extract domain-invariant structures through batch-wise Common Principal Component Analysis (CPCA). However, CPCANet suffers from rank-deficient covariance estimation due to the small-sample-size issue in mini-batch training. To address this limitation, we propose Projection Pursuit CPCANet (PP-CPCANet), a covariance-free framework that learns a global orthogonal basis on the Stiefel manifold and jointly optimizes it with network parameters via the Cayley transform. We further introduce a symmetry-breaking detached-median PP dispersion objective to extract common principal components (CPCs) with dense and robust optimization signals. Experiments on four DG benchmarks show that PP-CPCANet achieves SOTA performance while maintaining stable training.
Domain Generalization (DG) aims to learn representations that generalize to unseen target domains under out-of-distribution (OOD) shifts. Despite recent advances in invariant representation learning, explicitly capturing shared structures across source-domain distributions remains relatively underexplored. In this work, we integrate Common Principal Component Analysis (CPCA) into deep learning to identify a common orthogonal basis across multiple domains, proposing two complementary end-to-end differentiable frameworks. First, we introduce Common Principal Component Network (CPCNet), which unrolls the iterative Flury-Gautschi (FG) algorithm into fully differentiable neural layers, enabling covariance-based joint diagonalization within end-to-end representation learning. Second, to address rank deficiency arising from mini-batch covariance estimation in small-sample regimes, we develop Projection Pursuit-based Common Principal Component Network (PP-CPCNet), which directly optimizes a common orthogonal basis without explicit covariance estimation. Experiments validate the joint diagonalization capabilities of both methods, while evaluations on four standard DG benchmarks demonstrate competitive zero-shot generalization performance with favorable computational efficiency.