跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2608.12704 · v4

Efficient Hessian-Free Methods for Multi-Objective Bilevel Optimization with a Nonconvex Lower-Level Objective

首次提交 2026-08-13 09:37
版本更新 2026-11-18 16:15
雷达观测 2026-11-19 19:07
arXiv ↗PDF ↗

摘要

Multi-objective bilevel optimization has wide applications in the AI area such as automated learning and multi-task meta-learning. Although some recent studies have investigated multi-objective bilevel optimization, many existing methods rely on the (strongly) convex lower level problems. In fact, these multi-objective bilevel learning problems are generally nonconvex, and particularly their lower level problems are nonconvex. To fill this gap, we propose a class of Multi-Objective Moreau Envelope based Hessian-free Algorithms (MOMEHA) for the multi-objective bilevel learning problems with a nonconvex lower level. Specifically, our method uses the Moreau envelope to formulate a multi-objective single-level relaxation that enforces lower-level first-order stationarity. In particular, our method retains computational advantages of being single-loop and Hessian-free in the multi-objective setting by incorporating a smooth weighted Tchebycheff scalarization. Furthermore, we propose a momentum variant of MOMEHA (i.e., MB-MOMEHA) for the stochastic multi-objective bilevel learning problems. Under the stated assumptions, we establish finite-iteration bounds on the Moreau gap and Pareto stationarity residuals under dynamically relaxed gap constraints in both deterministic and stochastic settings. Some experiments on few-shot meta-learning and neural architecture search demonstrate that our methods outperform the existing approaches in terms of Pareto front quality, validating their effectiveness and robustness.

作者备注:55 pages

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v4,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v4 · 2026-11-18 16:15 · 最新观测版本

Efficient Hessian-Free Methods for Multi-Objective Bilevel Optimization with a Nonconvex Lower-Level Objective

Multi-objective bilevel optimization has wide applications in the AI area such as automated learning and multi-task meta-learning. Although some recent studies have investigated multi-objective bilevel optimization, many existing methods rely on the (strongly) convex lower level problems. In fact, these multi-objective bilevel learning problems are generally nonconvex, and particularly their lower level problems are nonconvex. To fill this gap, we propose a class of Multi-Objective Moreau Envelope based Hessian-free Algorithms (MOMEHA) for the multi-objective bilevel learning problems with a nonconvex lower level. Specifically, our method uses the Moreau envelope to formulate a multi-objective single-level relaxation that enforces lower-level first-order stationarity. In particular, our method retains computational advantages of being single-loop and Hessian-free in the multi-objective setting by incorporating a smooth weighted Tchebycheff scalarization. Furthermore, we propose a momentum variant of MOMEHA (i.e., MB-MOMEHA) for the stochastic multi-objective bilevel learning problems. Under the stated assumptions, we establish finite-iteration bounds on the Moreau gap and Pareto stationarity residuals under dynamically relaxed gap constraints in both deterministic and stochastic settings. Some experiments on few-shot meta-learning and neural architecture search demonstrate that our methods outperform the existing approaches in terms of Pareto front quality, validating their effectiveness and robustness.

查看此版本解读 →查看此版本原文 ↗