← 研踪 PaperTrailarXiv:2403.10232 · v3
Over-fitting Control in Deep Nonlinear Matrix Completion via Gradual Nonsmooth Regularization
首次提交 2024-03-15 20:00
版本更新 2026-11-02 00:53
雷达观测 2026-11-04 23:43
摘要
Conventional matrix completion methods exploit the underlying low-rank structure of a target matrix to recover its missing entries, but may be inadequate for capturing complex nonlinear dependencies. Deep neural networks have therefore been increasingly adopted for nonlinear matrix completion (NLMC). However, their high representational capacity, particularly in deep fully connected neural networks (FCNNs), can lead to over-fitting when training is restricted to incomplete observations. In this paper, we propose a Deep Neural Network with Nonsmooth Regularization (DNN-NSR) framework to address this issue by incorporating two complementary structural priors: sparsity of intermediate representations and low-rankness of layer-wise weight matrices. Specifically, the $\ell_1$-norm of hidden representations promotes sparse latent features, while the nuclear norm of weight matrices encourages low-rank layer-wise transformations. To optimize the resulting nonsmooth and nonconvex objective, we develop an extrapolated proximal gradient algorithm that employs soft-thresholding and singular-value thresholding to handle the two regularization terms, respectively. We further introduce a gradual regularization strategy that progressively increases the influence of these structural penalties during training. Under suitable assumptions, we establish convergence properties of the proposed algorithm toward a critical point of the training objective. The effectiveness of DNN-NSR is evaluated through extensive experiments on synthetic matrix recovery, single-image inpainting, and recommender systems using the MovieLens 100K and MovieLens 1M datasets. Comparisons with representative linear and nonlinear matrix completion methods demonstrate the potential of the proposed framework to improve recovery performance by jointly exploiting sparse latent representations and low-rank network transformations.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v3 · 2026-11-02 00:53 · 最新观测版本
Over-fitting Control in Deep Nonlinear Matrix Completion via Gradual Nonsmooth Regularization
Conventional matrix completion methods exploit the underlying low-rank structure of a target matrix to recover its missing entries, but may be inadequate for capturing complex nonlinear dependencies. Deep neural networks have therefore been increasingly adopted for nonlinear matrix completion (NLMC). However, their high representational capacity, particularly in deep fully connected neural networks (FCNNs), can lead to over-fitting when training is restricted to incomplete observations. In this paper, we propose a Deep Neural Network with Nonsmooth Regularization (DNN-NSR) framework to address this issue by incorporating two complementary structural priors: sparsity of intermediate representations and low-rankness of layer-wise weight matrices. Specifically, the $\ell_1$-norm of hidden representations promotes sparse latent features, while the nuclear norm of weight matrices encourages low-rank layer-wise transformations. To optimize the resulting nonsmooth and nonconvex objective, we develop an extrapolated proximal gradient algorithm that employs soft-thresholding and singular-value thresholding to handle the two regularization terms, respectively. We further introduce a gradual regularization strategy that progressively increases the influence of these structural penalties during training. Under suitable assumptions, we establish convergence properties of the proposed algorithm toward a critical point of the training objective. The effectiveness of DNN-NSR is evaluated through extensive experiments on synthetic matrix recovery, single-image inpainting, and recommender systems using the MovieLens 100K and MovieLens 1M datasets. Comparisons with representative linear and nonlinear matrix completion methods demonstrate the potential of the proposed framework to improve recovery performance by jointly exploiting sparse latent representations and low-rank network transformations.
查看此版本解读 →查看此版本原文 ↗