TEMPER: Reducing Attribute Leakage from Face Recognition Embeddings
Ünsal Öztürk, Vedrana Krivokuća Hahn, Sushil Bhattacharjee, Sébastien Marcel
原文摘要
Face recognition embeddings encode soft-biometric attributes such as gender and ethnicity, which can be inferred from templates. This paper proposes TEMPlate Entropy Regularisation (TEMPER), a post-hoc method that removes an attribute. TEMPER learns a linear map of the embedding in a split-latent variational autoencoder. The map is initialised at the principal component analysis (PCA) solution of the embeddings and trained to maximise the predictive entropy of an auxiliary attribute classifier. A relational term keeps the similarities of mated pairs close to the embedding similarities. We evaluate the removal of gender and a four-group ethnicity label from ArcFace embeddings against ten prior post-hoc methods and two label-free controls. Every method is trained on VGGFace2 and tested on IJB-C and RFW at false match rates down to $10^{-4}$. Leakage is the balanced accuracy of the strongest of four attacker families trained on transformed VGGFace2 templates. TEMPER retains the highest IJB-C true match rate at a false match rate of $10^{-3}$ at every leakage threshold for both attributes. TEMPER retains a true match rate of 82.4% at a gender leakage of at most 52.5%, close to chance, against 76.7% for the best prior method. In the ablation, this improvement comes from the PCA initialisation and the relational term.
cs.CV