hh.sePublications
Planned maintenance
A system upgrade is planned for 17/11-2026, at 12:00-13:00. During this time DiVA will be unavailable.
Change search
ExportLink to record
Permanent link

Direct link
BETA

Project

Project type/Form of grant
Project grant
Title [sv]
Ansiktsanalys i eran av mobila enheter och ansiktsmasker
Title [en]
Facial Analysis in the Era of Mobile Devices and Face Masks
Abstract [en]
We address the challenge of reliable analysis of facial images when the ocular region is the only visible part. Occlusion may appear in unconstrained environments, but it is now an issue even in controlled setups due to mandatory masks. Solutions must be also capable to operate on devices with hardware restrictions, a necessity if they are to be employed on devices such as smartphones or assistive robots in home- or health-care environments.One goal is to provide reliable methods to detect the face. Impressive performance is shown by deep learning solutions, but they use heavy CNNs of hundreds of MB, infeasible in mobiles or robots. Also, most are trained to detect the entire face, and not specifically to cope with occlusion. We will use complex symmetry filters as attention mechanism to facilitate detection, coupled with CNNs with complex coefficients, given the success of such filters as a stand-alone method to detect face landmarks in controlled conditions.Another goal will be the estimation of soft-biometrics indicators (gender, age, ethnicity). These indicators are easier to extract in unconstrained scenarios and can complement a non-conclusive result of a hard modality (iris or face). They have other applications as well, such as customized advertising, enhanced HCI, age-dependent access, location of specific individuals in video streams, or child pornography detection. However, the use of ocular images for such task and with light CNNs are unexplored avenues.
Publications (8 of 8) Show all publications
Alonso-Fernandez, F., Hernandez-Diaz, K., Tiwari, P. & Bigun, J. (2024). Combined CNN and ViT features off-the-shelf: Another astounding baseline for recognition. In: Proceedings - 16th IEEE International Workshop on Information Forensics and Security, WIFS 2024: . Paper presented at 16th IEEE International Workshop on Information Forensics and Security, WIFS 2024, Rome, Italy, December 2-5, 2024 (pp. 1-6). IEEE
Open this publication in new window or tab >>Combined CNN and ViT features off-the-shelf: Another astounding baseline for recognition
2024 (English)In: Proceedings - 16th IEEE International Workshop on Information Forensics and Security, WIFS 2024, IEEE, 2024, p. 1-6Conference paper, Published paper (Refereed)
Abstract [en]

We apply pre-trained architectures, originally developed for the ImageNet Large Scale Visual Recognition Challenge, for periocular recognition. These architectures have demon-strated significant success in various computer vision tasks beyond the ones for which they were designed. This work builds on our previous study using off-the-shelf Convolutional Neural Network (CNN) and extends it to include the more recently proposed Vision Transformers (ViT). Despite being trained for generic object classification, middle-layer features from CNNs and ViTs are a suitable way to recognize individuals based on periocular images. We also demonstrate that CNNs and ViTs are highly complementary since their combination results in boosted accuracy. In addition, we show that a small portion of these pre-trained models can achieve good accuracy, resulting in thinner models with fewer parameters, suitable for resource-limited environments such as mobiles. This efficiency improves if traditional handcrafted features are added as well. ©2024 IEEE.

Place, publisher, year, edition, pages
IEEE, 2024
Series
IEEE International Workshop on Information Forensics and Security, ISSN 2157-4766, E-ISSN 2157-4774
Keywords
Periocular recognition, deep representation, biometrics, transfer learning, one-shot learning, Convolutional Neural Network, Vision Transformers
National Category
Signal Processing
Identifiers
urn:nbn:se:hh:diva-54713 (URN)10.1109/WIFS61860.2024.10810712 (DOI)001422478100039 ()2-s2.0-85215518296 (Scopus ID)979-8-3503-6442-2 (ISBN)979-8-3503-6443-9 (ISBN)
Conference
16th IEEE International Workshop on Information Forensics and Security, WIFS 2024, Rome, Italy, December 2-5, 2024
Funder
Swedish Research CouncilVinnova
Available from: 2024-10-06 Created: 2024-10-06 Last updated: 2025-10-01Bibliographically approved
Alonso-Fernandez, F., Hernandez-Diaz, K., Buades Rubio, J. M. & Bigun, J. (2024). SqueezerFaceNet: Reducing a Small Face Recognition CNN Even More Via Filter Pruning. In: Hernández Heredia, Y.; Milián Núñez, V.; Ruiz Shulcloper, J. (Ed.), Progress in Artificial Intelligence and Pattern Recognition. IWAIPR 2023.: . Paper presented at VIII International Workshop on Artificial Intelligence and Pattern Recognition, IWAIPR, Varadero, Cuba, September 27-29, 2023 (pp. 349-361). Cham: Springer, 14335
Open this publication in new window or tab >>SqueezerFaceNet: Reducing a Small Face Recognition CNN Even More Via Filter Pruning
2024 (English)In: Progress in Artificial Intelligence and Pattern Recognition. IWAIPR 2023. / [ed] Hernández Heredia, Y.; Milián Núñez, V.; Ruiz Shulcloper, J., Cham: Springer, 2024, Vol. 14335, p. 349-361Conference paper, Published paper (Refereed)
Abstract [en]

The widespread use of mobile devices for various digital services has created a need for reliable and real-time person authentication. In this context, facial recognition technologies have emerged as a dependable method for verifying users due to the prevalence of cameras in mobile devices and their integration into everyday applications. The rapid advancement of deep Convolutional Neural Networks (CNNs) has led to numerous face verification architectures. However, these models are often large and impractical for mobile applications, reaching sizes of hundreds of megabytes with millions of parameters. We address this issue by developing SqueezerFaceNet, a light face recognition network which less than 1M parameters. This is achieved by applying a network pruning method based on Taylor scores, where filters with small importance scores are removed iteratively. Starting from an already small network (of 1.24M) based on SqueezeNet, we show that it can be further reduced (up to 40%) without an appreciable loss in performance. To the best of our knowledge, we are the first to evaluate network pruning methods for the task of face recognition. © 2024, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Place, publisher, year, edition, pages
Cham: Springer, 2024
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 14335
Keywords
Face recognition, Mobile Biometrics, CNN pruning, Taylor scores
National Category
Signal Processing
Identifiers
urn:nbn:se:hh:diva-51299 (URN)10.1007/978-3-031-49552-6_30 (DOI)2-s2.0-85180788350 (Scopus ID)978-3-031-49551-9 (ISBN)978-3-031-49552-6 (ISBN)
Conference
VIII International Workshop on Artificial Intelligence and Pattern Recognition, IWAIPR, Varadero, Cuba, September 27-29, 2023
Funder
Swedish Research CouncilVinnova
Note

Funding: F. A.-F., K. H.-D., and J. B. thank the Swedish Research Council (VR) and the Swedish Innovation Agency (VINNOVA) for funding their research. Author J. M. B. thanks the project EX-PLAINING - "Project EXPLainable Artificial INtelligence systems for health and well-beING", under Spanish national projects funding (PID2019-104829RA-I00/AEI/10.13039/501100011033).

Available from: 2023-07-20 Created: 2023-07-20 Last updated: 2025-10-01Bibliographically approved
Busch, C., Deravi, F., Frings, D., Alonso-Fernandez, F. & Bigun, J. (2023). Facilitating free travel in the Schengen area—A position paper by the European Association for Biometrics. IET Biometrics, 12(2), 112-128
Open this publication in new window or tab >>Facilitating free travel in the Schengen area—A position paper by the European Association for Biometrics
Show others...
2023 (English)In: IET Biometrics, ISSN 2047-4938, E-ISSN 2047-4946, Vol. 12, no 2, p. 112-128Article in journal (Refereed) Published
Abstract [en]

Due to migration, terror-threats and the viral pandemic, various EU member states have re-established internal border control or even closed their borders. European Association for Biometrics (EAB), a non-profit organisation, solicited the views of its members on ways which biometric technologies and services may be used to help with re-establishing open borders within the Schengen area while at the same time mitigating any adverse effects. From the responses received, this position paper was composed to identify ideas to re-establish free travel between the member states in the Schengen area. The paper covers the contending needs for security, open borders and fundamental rights as well as legal constraints that any technological solution must consider. A range of specific technologies for direct biometric recognition alongside complementary measures are outlined. The interrelated issues of ethical and societal considerations are also highlighted. Provided a holistic approach is adopted, it may be possible to reach a more optimal trade-off with regards to open borders while maintaining a high-level of security and protection of fundamental rights. European Association for Biometrics and its members can play an important role in fostering a shared understanding of security and mobility challenges and their solutions. © 2023 The Authors. IET Biometrics published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology.

Place, publisher, year, edition, pages
Oxford: John Wiley & Sons, 2023
Keywords
biometric applications, biometric template protection, biometrics (access control), computer vision, data privacy, image analysis for biometrics, object tracking
National Category
Signal Processing
Identifiers
urn:nbn:se:hh:diva-50367 (URN)10.1049/bme2.12107 (DOI)000976420600001 ()2-s2.0-85153281620 (Scopus ID)
Available from: 2023-04-20 Created: 2023-04-20 Last updated: 2025-10-01Bibliographically approved
Zell, O., Påsson, J., Hernandez-Diaz, K., Alonso-Fernandez, F. & Nilsson, F. (2023). Image-Based Fire Detection in Industrial Environments with YOLOv4. In: Maria De Marsico; Gabriella Sanniti di Baja; Ana Fred (Ed.), Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods ICPRAM: . Paper presented at 12th International Conference on Pattern Recognition Applications and Methods, ICPRAM, Lisbon, Portugal, February 22-24, 2023 (pp. 379-386). Setúbal: SciTePress, 1
Open this publication in new window or tab >>Image-Based Fire Detection in Industrial Environments with YOLOv4
Show others...
2023 (English)In: Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods ICPRAM / [ed] Maria De Marsico; Gabriella Sanniti di Baja; Ana Fred, Setúbal: SciTePress, 2023, Vol. 1, p. 379-386Conference paper, Published paper (Refereed)
Abstract [en]

Fires have destructive power when they break out and affect their surroundings on a devastatingly large scale. The best way to minimize their damage is to detect the fire as quickly as possible before it has a chance to grow. Accordingly, this work looks into the potential of AI to detect and recognize fires and reduce detection time using object detection on an image stream. Object detection has made giant leaps in speed and accuracy over the last six years, making real-time detection feasible. To our end, we collected and labeled appropriate data from several public sources, which have been used to train and evaluate several models based on the popular YOLOv4 object detector. Our focus, driven by a collaborating industrial partner, is to implement our system in an industrial warehouse setting, which is characterized by high ceilings. A drawback of traditional smoke detectors in this setup is that the smoke has to rise to a sufficient height. The AI models brought forward in this research managed to outperform these detectors by a significant amount of time, providing precious anticipation that could help to minimize the effects of fires further.

Place, publisher, year, edition, pages
Setúbal: SciTePress, 2023
Series
International Conference on Pattern Recognition Applications and Methods (ICPRAM), ISSN 2184-4313
Keywords
Fire detection, Smoke Detection, Machine learning, Computer Vision, YOLOv4
National Category
Signal Processing
Identifiers
urn:nbn:se:hh:diva-48793 (URN)10.5220/0011689400003411 (DOI)978-989-758-626-2 (ISBN)
Conference
12th International Conference on Pattern Recognition Applications and Methods, ICPRAM, Lisbon, Portugal, February 22-24, 2023
Projects
2021-05038 Vinnova DIFFUSE Disentanglement of Features For Utilization in Systematic Evaluation
Funder
Swedish Research CouncilVinnova
Available from: 2022-12-09 Created: 2022-12-09 Last updated: 2025-10-01Bibliographically approved
Baaz, A., Yonan, Y., Hernandez-Diaz, K., Alonso-Fernandez, F. & Nilsson, F. (2023). Synthetic Data for Object Classification in Industrial Applications. In: Maria De Marsico; Gabriella Sanniti di Baja; Ana Fred (Ed.), Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods ICPRAM: . Paper presented at 12th International Conference on Pattern Recognition Applications and Methods, ICPRAM, Lisbon, Portugal, February 22-24, 2023 (pp. 387-394). SciTePress, 1
Open this publication in new window or tab >>Synthetic Data for Object Classification in Industrial Applications
Show others...
2023 (English)In: Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods ICPRAM / [ed] Maria De Marsico; Gabriella Sanniti di Baja; Ana Fred, SciTePress, 2023, Vol. 1, p. 387-394Conference paper, Published paper (Refereed)
Abstract [en]

One of the biggest challenges in machine learning is data collection. Training data is an important part since it determines how the model will behave. In object classification, capturing a large number of images per object and in different conditions is not always possible and can be very time-consuming and tedious. Accordingly, this work explores the creation of artificial images using a game engine to cope with limited data in the training dataset. We combine real and synthetic data to train the object classification engine, a strategy that has shown to be beneficial to increase confidence in the decisions made by the classifier, which is often critical in industrial setups. To combine real and synthetic data, we first train the classifier on a massive amount of synthetic data, and then we fine-tune it on real images. Another important result is that the amount of real images needed for fine-tuning is not very high, reaching top accuracy with just 12 or 24 images per class. This substantially reduces the requirements of capturing a great amount of real data. © 2023 by SCITEPRESS-Science and Technology Publications, Lda.

Place, publisher, year, edition, pages
SciTePress, 2023
Keywords
Synthetic Data, Object Classification, Machine Learning, Computer Vision, ResNet50
National Category
Signal Processing
Identifiers
urn:nbn:se:hh:diva-48794 (URN)10.5220/0011689900003411 (DOI)2-s2.0-85174507299 (Scopus ID)
Conference
12th International Conference on Pattern Recognition Applications and Methods, ICPRAM, Lisbon, Portugal, February 22-24, 2023
Projects
2021-05038 Vinnova DIFFUSE Disentanglement of Features For Utilization in Systematic Evaluation
Funder
Swedish Research CouncilVinnova
Available from: 2022-12-09 Created: 2022-12-09 Last updated: 2025-10-01Bibliographically approved
Karlsson, J., Strand, F., Bigun, J., Alonso-Fernandez, F., Hernandez-Diaz, K. & Nilsson, F. (2023). Visual Detection of Personal Protective Equipment and Safety Gear on Industry Workers. In: Maria De Marsico; Gabriella Sanniti di Baja; Ana Fred (Ed.), Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods: February 22-24, 2023, in Lisbon, Portugal. Paper presented at 12th International Conference on Pattern Recognition Applications and Methods, ICPRAM, Lisbon, Portugal, February 22-24, 2023 (pp. 395-402). SciTePress, 1
Open this publication in new window or tab >>Visual Detection of Personal Protective Equipment and Safety Gear on Industry Workers
Show others...
2023 (English)In: Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods: February 22-24, 2023, in Lisbon, Portugal / [ed] Maria De Marsico; Gabriella Sanniti di Baja; Ana Fred, SciTePress, 2023, Vol. 1, p. 395-402Conference paper, Published paper (Refereed)
Abstract [en]

Workplace injuries are common in today’s society due to a lack of adequately worn safety equipment. A system that only admits appropriately equipped personnel can be created to improve working conditions. The goal is thus to develop a system that will improve workers’ safety using a camera that will detect the usage of Personal Protective Equipment (PPE). To this end, we collected and labeled appropriate data from several public sources, which have been used to train and evaluate several models based on the popular YOLOv4 object detector. Our focus, driven by a collaborating industrial partner, is to implement our system into an entry control point where workers must present themselves to obtain access to a restricted area. Combined with facial identity recognition, the system would ensure that only authorized people wearing appropriate equipment are granted access. A novelty of this work is that we increase the number of classes to five objects (hardhat, safety vest, safety gloves, safety glasses, and hearing protection), whereas most existing works only focus on one or two classes, usually hardhats or vests. The AI model developed provides good detection accuracy at a distance of 3 and 5 meters in the collaborative environment where we aim at operating (mAP of 99/89%, respectively). The small size of some objects or the potential occlusion by body parts have been identified as potential factors that are detrimental to accuracy, which we have counteracted via data augmentation and cropping of the body before applying PPE detection. © 2023 by SCITEPRESS-Science and Technology Publications, Lda.

Place, publisher, year, edition, pages
SciTePress, 2023
Series
ICPRAM, E-ISSN 2184-4313
Keywords
PPE, PPE Detection, Personal Protective Equipment, Machine Learning, Computer Vision, YOLO
National Category
Signal Processing
Identifiers
urn:nbn:se:hh:diva-48795 (URN)10.5220/0011693500003411 (DOI)2-s2.0-85174511525 (Scopus ID)978-989-758-626-2 (ISBN)
Conference
12th International Conference on Pattern Recognition Applications and Methods, ICPRAM, Lisbon, Portugal, February 22-24, 2023
Projects
2021-05038 Vinnova DIFFUSE Disentanglement of Features For Utilization in Systematic Evaluation
Available from: 2022-12-09 Created: 2022-12-09 Last updated: 2025-10-01Bibliographically approved
Hedman, P., Skepetzis, V., Hernandez-Diaz, K., Bigun, J. & Alonso-Fernandez, F. (2022). On the effect of selfie beautification filters on face detection and recognition. Pattern Recognition Letters, 163, 104-111
Open this publication in new window or tab >>On the effect of selfie beautification filters on face detection and recognition
Show others...
2022 (English)In: Pattern Recognition Letters, ISSN 0167-8655, E-ISSN 1872-7344, Vol. 163, p. 104-111Article in journal (Refereed) Published
Abstract [en]

Beautification and augmented reality filters are very popular in applications that use selfie images. However, they can distort or modify biometric features, severely affecting the ability to recognise the individuals’ identity or even detect the face. Accordingly, we address the effect of such filters on the accuracy of automated face detection and recognition. The social media image filters studied modify the image contrast, illumination, or occlude parts of the face. We observe that the effect of some of these filters is harmful to face detection and identity recognition, especially if they obfuscate the eye or (to a lesser extent) the nose. To counteract such effect, we develop a method to reverse the applied manipulation with a modified version of the U-NET segmentation network. This method is observed to contribute to better face detection and recognition accuracy. From a recognition perspective, we employ distance measures and trained machine learning algorithms applied to features extracted using several CNN backbones. We also evaluate if incorporating filtered images into the training set of machine learning approaches is beneficial. Our results show good recognition when filters do not occlude important landmarks, especially the eyes. The combined effect of the proposed approaches also allows mitigating the impact produced by filters that occlude parts of the face. © 2022 The Authors. Published by Elsevier B.V.

Place, publisher, year, edition, pages
Amsterdam: Elsevier, 2022
Keywords
Face detection, Face recognition, Social media filters, Beautification, U-NET, Convolutional neural network
National Category
Signal Processing
Identifiers
urn:nbn:se:hh:diva-48481 (URN)10.1016/j.patrec.2022.09.018 (DOI)000877215000005 ()2-s2.0-85139592291 (Scopus ID)
Funder
Swedish Research CouncilVinnova
Available from: 2022-10-14 Created: 2022-10-14 Last updated: 2025-10-01Bibliographically approved
Alonso-Fernandez, F., Hernandez-Diaz, K. & Bigun, J.Estimating Soft Biometrics Demographic Attributes from Ocular Images: A Survey.
Open this publication in new window or tab >>Estimating Soft Biometrics Demographic Attributes from Ocular Images: A Survey
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Soft-biometric attributes such as gender, age, and ethnicity provide valuable ancillary information that can complement primary biometric recognition systems or support applications where full identity recognition is not feasible. Among the different biometric modalities, the ocular region has emerged as a particularly robust source of soft-biometric cues due to its visibility under partial occlusion and its availability across a wide range of acquisition conditions. This paper presents an ongoing survey, currently under completion, on demographic attribute estimation from ocular imagery. We first review publicly available datasets used in this field, including both dedicated ocular databases and facial datasets from which periocular crops are extracted. Then, we provide a comprehensive, chronological description of existing work on gender, age, and ethnicity estimation using ocular images. The survey analyzes methodological trends ranging from early handcrafted feature descriptors and classical machine learning techniques to recent deep learning and transformer-based approaches. Finally, we discuss open challenges and outline promising research directions for advancing demographic inference from ocular imagery in unconstrained environments. This work is expected to provide an insight of the most relevant issues in periocular demographics estimation, giving a comprehensive coverage of the existing literature and current state of the art.

Keywords
Periocular biometrics, demographics estimation, gender estimation, age estimation, ethnicity estimation
National Category
Signal Processing
Research subject
Smart Cities and Communities, Future industry
Identifiers
urn:nbn:se:hh:diva-58564 (URN)
Projects
Horizon 101168317
Funder
Swedish Research Council, 2021-05110EU, Horizon Europe, 101168317
Available from: 2026-03-16 Created: 2026-03-16 Last updated: 2026-04-09Bibliographically approved
Principal InvestigatorAlonso-Fernandez, Fernando
Co-InvestigatorBigun, Josef
Coordinating organisation
Halmstad University
Funder
Period
2022-01-01 - 2025-12-31
National Category
Computer ScienceComputer Vision and Robotics (Autonomous Systems)Signal Processing
Identifiers
DiVA, id: project:2688Project, id: 2021-05110_VR