Exact Fixed-Point Constraints in Neural-ODEs: Injecting Prior Knowledge into the Network Parameters
Feliciano Giuseppe Pacifico, Duccio Fanelli, Lorenzo Buffoni, Lorenzo Chicchi, Diego Febbe, Raffaele Marino
原文摘要
Neural ordinary differential equations (Neural-ODEs) learn a velocity field from data. Often, however, additional information on the target field is available, such as the location of some of its equilibria or, more generally, its values at given points. We introduce a method to inject this information directly into the parameters of a Neural-ODE, so that a finite set of prescribed fixed points holds exactly throughout training, with no penalty terms and no post-hoc projection. The output matrix is parametrized through the Moore-Penrose pseudoinverse, implemented by a thin QR factorization of the hidden features at the prescribed points. This splits the output matrix into a component determined by the constraints and a free one, which shapes the field away from the prescribed points without altering its values there. As a result, the constrained Neural-ODE remains a universal approximator of the continuous vector fields compatible with the constraints. We test the method on a two-dimensional Lotka-Volterra competition model with four equilibria. In vector field regression, the constrained model keeps the fixed-point residual at single-precision round-off, and its test error is up to about five times lower than that of an unconstrained network of the same architecture. A soft penalty on the equilibria lowers the residual only at the cost of accuracy, and still leaves it at least an order of magnitude above that of the constrained model. Trained to convergence on datasets of increasing size, the unconstrained and penalized networks saturate already on small datasets, while the constrained model keeps improving over a wider range. Close to the equilibria and over long time horizons, the constrained model is the most accurate at every dataset size considered. This also holds when the models are trained on observed trajectories, by backpropagation through the ODE solver.
cond-mat.dis-nn · cs.LG · physics.bio-ph