Developing a machine learning-based vehicular safety system that is effective and generalizes well, capable of coping with all the different scenarios in real traffic is a challenge that requires large amounts of data. Especially visual data for when you want an autonomous vehicle to make decisions based on peoples’ possible intent revealed by the facial expression and eye gaze of nearby pedestrians. The problem with collecting this kind of data is the privacy issues and conflict with current laws like General Data Protection Regulation (GDPR). To deal with this problem we can anonymise faces with current identity and face swapping techniques. To evaluate the performance and interpretation of the anonymization process, there is a need for a metric to measure how well these faces are anonymized that takes identity leakage into consideration. To our knowledge, there is currently no such investigation for this problem. However, our method is based on current facial recognition methods and how recent face swapping work determines identity transfer performance. Our suggestion is to utilize state-of-the-art identity encoders like FaceNet and ArcFace to make use of the embedding vectors to measure anonymity. We provide qualitative results that show the applicability of publicly available identity encoders for measuring anonymity. We further strengthen the applicability of how these encoders behave on the VGGFace2 dataset compared to samples that have had their identity changed by Faceshifter, along with a survey regarding the anonymization procedure to pinpoint how strong facial anonymization is compared the vector distance measurements.

Measuring and comparing facial expression have several practical applications. One such application is to measure the facial expression embedding, and to compare distances between those expressions embeddings in order to determine the identity- and face swapping algorithms' capabilities in preserving the facial expression information. One useful aspect is to present how well the expressions are preserved while anonymizing facial data during privacy aware data collection. We show that a weighted supervised contrastive learning is a strong approach for learning facial expression representation embeddings and dealing with the class imbalance bias. By feeding a classifier-head with the learned embeddings we reach competitive state-of-the-art results. Furthermore, we demonstrate the use case of measuring the distance between the expressions of a target face, a source face and the anonymized target face in the facial anonymization context. © 2021 IEEE.

In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). The AFFA module is embedded in the decoder and adaptively learns to fuse attribute features and features conditioned on identity information without requiring any additional facial segmentation process. In IFSR, we leverage the intermediate features in an identity encoder to preserve important attributes such as head pose, facial expression, lighting, and occlusion in the target face, while still transferring the identity of the source face with high fidelity. We conduct extensive quantitative and qualitative experiments on various datasets and show that the proposed FaceDancer outperforms other state-of-the-art networks in terms of identityn transfer, while having significantly better pose preservation than most of the previous methods. © 2023 IEEE.

In this paper, we present a new approach for facial anonymization in images and videos, abbreviated as FIVA. Our proposed method is able to maintain the same face anonymization consistently over frames with our suggested identity-tracking and guarantees a strong difference from the original face. FIVA allows for 0 true positives for a false acceptance rate of 0.001. Our work considers the important security issue of reconstruction attacks and investigates adversarial noise, uniform noise, and parameter noise to disrupt reconstruction attacks. In this regard, we apply different defense and protection methods against these privacy threats to demonstrate the scalability of FIVA. On top of this, we also show that reconstruction attack models can be used for detection of deep fakes. Last but not least, we provide experimental results showing how FIVA can even enable face swapping, which is purely trained on a single target image. © 2023 IEEE.

In this paper, we investigate the impact of adversar- ial attacks on identity encoders within a realistic de-identification framework. Our experiments show that the transferability of attacks transfers from an external surrogate model to the system model (e.g., CosFace to ArcFace), allows the adversary to cause identity information to leak in a sufficiently sensitive face recognition system. We present experimental evidence and propose strategies to mitigate this vulnerability. Specifically, we show how fine-tuning on adversarial examples helps to mitigate this effect for distortion-based attacks (i.e., snow, fog, etc.), while a simple low-pass filter can attenuate the effect of adversarial noise without affecting the de-identified images. Our mitigation results in a de-identification system that preserves its functionality while being significantly more robust to adversarial noise. 

Target-oriented face de-identification models aim to anonymize facial identities by transforming original faces to resemble specific targets. Such models commonly lever- age generative encoder-decoder architectures to manip- ulate facial appearances, enabling them to produce re- alistic high-fidelity de-identification results, while ensur- ing considerable attribute-retention capabilities. However, target-oriented models also carry the risk of inadvertently preserving subtle identity cues, making them (potentially) reversible and susceptible to reconstruction attacks. To address this problem, we introduce a novel robust face de-identification approach, called HYDRO, that combines target-oriented models with a dedicated diffusion process specifically designed to destroy any imperceptible informa- tion that may allow learning to reverse the de-identification procedure. HYDRO first de-identifies the given face im- age, injects noise into the de-identification result to im- pede reconstruction, and then applies a diffusion-based re- covery step to improve fidelity and minimize the impact of the noising process on the data characteristics. To further improve image fidelity and better retain gaze directions, a novel Eye Similarity Discriminator (ESD) is also intro- duced and incorporated it into the training of HYDRO. Ex- tensive quantitative and qualitative experiments on three diverse datasets demonstrate that HYDRO exhibits state- of-the-art (SOTA) fidelity and attribute-retention capabili- ties, while being the only target-oriented method resilient against reconstruction attacks. In comparison to multiple SOTA competitors, HYDRO reduces the success of recon- struction attacks by 85.7% on average. The code is avail- able at: https://github.com/anonymousiccv2025/HYDRO.