FIELDS: Face reconstruction with accurate Inference of Expression using Learning with Direct Supervision
Chen Ling, Henglin Shi, Hedvig Kjellström
原文摘要
Monocular 3D face reconstruction estimates the parameters of a 3D morphable face model (3DMM) from a single image, providing an explicit geometric representation for facial expression analysis. Owing to the scarcity of paired 2D images and 3D facial meshes, most methods are trained in a self-supervised manner by rendering the predicted face and minimizing image-level reconstruction losses, such as photometric and landmark errors, against the input image. However, these losses enforce visual alignment but do not ensure that the predicted parameters preserve affective information. Recent methods add image-level emotion consistency loss, but it can be satisfied by exaggerating facial deformations. To address these issues, we propose FIELDS (Face reconstruction with accurate Inference of Expression using Learning with Direct Supervision), which learns affect-informative expression representations in the FLAME parameter space. Specifically, FIELDS augments 2D self-supervised reconstruction with scan-derived expression anchoring and a parameter-space emotion loss. During training, scan-fitted targets directly constrain expression-related FLAME parameters to discourage exaggerated deformations, while a jointly trained emotion head uses discrete emotion and continuous valence--arousal annotations to make the predicted parameters informative for affect analysis. Experiments show that FIELDS improves affective utility from the predicted parameters in both within- and cross-dataset settings. Notably, it also achieves lower reconstruction errors overall and across most facial regions than the comparison baselines under both rigid and similarity alignment.
cs.CV