The maritime industry is under pressure to increase energy efficiency for climate change mitigation. Navigational data, combining vessel operational and environmental measurements from onboard instruments and external sources, are critical for achieving this goal. Short-sea shipping presents a unique challenge due to the significant influence of surrounding landscape characteristics. With high-resolution onboard data increasingly accessible through IoT devices, appropriate data representations and AI/ML analytical tools are needed for effective decision support. The aim of this study is to investigate the fuel consumption estimation model’s role in developing an energy efficiency decision support tool. ML models that lacking explainability may neglect important factors and essential constraints, such as the need to meet arrival time requirements. Onboard weather measurements are compared to external forecasts, and our findings demonstrate the necessity of eXplainable Artificial Intelligence (XAI) techniques for effective decision support. Real-world data from a short-sea passenger vessel in southern Sweden, consisting of 1754 voyages over 15 months (More of data description and code sources of this study can be found in the GitHub repository at https://github.com/MohamedAbuella/ST4EESSS), are used to support our conclusions.  © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

This paper addresses the challenge of identifying the paths for vessels with operating routes of repetitive paths, partially repetitive paths, and new paths. We propose a spatial clustering approach for labeling the vessel paths by using only position information. We develop a path clustering framework employing two methods: a distance-based path modeling and a likelihood estimation method. The former enhances the accuracy of path clustering through the integration of unsupervised machine learning techniques, while the latter focuses on likelihood-based path modeling and introduces segmentation for a more detailed analysis. The result findings highlight the superior performance and efficiency of the developed approach, as both methods for clustering vessel paths into five clusters achieve a perfect F1-score. The approach aims to offer valuable insights for route planning, ultimately contributing to improving safety and efficiency in maritime transportation. © 2013 IEEE.

Deep learning and Computer vision are becoming a part of everyday objects and machines. Involvement of artificial intelligence in human’s daily life open doors to new opportunities and research. This involvement provides the idea of improving upon the in-hand research of spatial relations and coming up with a more generic and robust algorithm that provides us with 2-D and 3-D spatial relations and uses RGB and RGB-D images which can help us with few complex relations such as ‘on’ or ‘in’ as well. Suggested methods are tested on the dataset with animated and real objects, where the number of objects varies in every image from at least 4 to at most 10 objects. The size and orientation of objects are also different in every image.  

Survival Analysis models usually output functions (survival or hazard functions) rather than point predictions like regression and classification models. This makes the explanations of such models a challenging task, especially using the Shapley values. We propose SurvSHAP, a new model-agnostic algorithm to explain survival models that predict survival curves. The algorithm is based on discovering patterns in the predicted survival curves, the output of the survival model, that would identify significantly different survival behaviors, and utilizing a proxy model and SHAP method to explain these distinct survival behaviors. Experiments on synthetic and real datasets demonstrate that the SurvSHAP is able to capture the underlying factors of the survival patterns. Moreover, SurvSHAP results on the Cox Proportional Hazard model are compared with the weights of the model to show that we provide faithful overall explanations, with more fine-grained explanations of the sub-populations. We also illustrate the wrong model and explanations learned by a Cox model when applied to heterogeneous sub-populations. We show that a non-linear machine learning survival model with SurvSHAP can better model the data and provide better explanations than linear models.

Time-To-Event (TTE) modeling using survival analysis in industrial settings faces the challenge of premature replacements of machine components, which leads to bias and errors in survival prediction. Typically, TTE survival data contains information about components and if they had failed or not up to a certain time. For failed components, the time is noted, and a failure is referred to as an event. A component that has not failed is denoted as censored. In industrial settings, in contrast to medical settings, there can be considerable uncertainty in an event; a component can be replaced before it fails to prevent operation stops or because maintenance staff believe that the component is faulty. This shows up as “no fault found” in warranty studies, where a significant proportion of replaced components may appear fault-free when tested or inspected after replacement.

In this work, we propose an expectation-maximization-like method for discovering such premature replacements in survival data. The method is a two-phase iterative algorithm employing a genetic algorithm in the maximization phase to learn better event assignments on a validation set. The learned labels through iterations are accumulated and averaged to be used to initialize the following expectation phase. The assumption is that the more often the event is selected, the more likely it is to be an actual failure and not a “no fault found”.

Experiments on synthesized and simulated data show that the proposed method can correctly detect a significant percentage of premature replacement cases.

Cardiovascular diseases (CVD) rank among the leading global causes of mortality. Early detection and diagnosis are paramount in minimizing their impact. The application of ML and DL in classifying the occurrence of cardiovascular diseases holds significant potential for reducing diagnostic errors. This research endeavors to construct a model capable of accurately predicting cardiovascular diseases, thereby mitigating the fatality associated with CVD. In this paper, the authors introduce a novel approach that combines an artificial intelligence network (AIN)-based feature selection (FS) technique with cutting-edge DL and ML classifiers for the early detection of heart diseases based on patient medical histories. The proposed model is rigorously evaluated using two real-world datasets sourced from the University of California. The authors conduct extensive data preprocessing and analysis, and the findings from this study demonstrate that the proposed methodology surpasses the performance of existing state-of-the-art methods, achieving an exceptional accuracy rate of 99.99%. © 2023 IGI Global. All rights reserved.

This paper describes an adaptation of the Local Interpretable Model-Agnostic Explanations (LIME) AI method to operate under a biometric verification setting. LIME was initially proposed for networks with the same output classes used for training, and it employs the softmax probability to determine which regions of the image contribute the most to classification. However, in a verification setting, the classes to be recognized have not been seen during training. In addition, instead of using the softmax output, face descriptors are usually obtained from a layer before the classification layer. The model is adapted to achieve explainability via cosine similarity between feature vectors of perturbated versions of the input image. The method is showcased for face biometrics with two CNN models based on MobileNetv2 and ResNet50. © 2023 IEEE.

The widespread use of mobile devices for various digital services has created a need for reliable and real-time person authentication. In this context, facial recognition technologies have emerged as a dependable method for verifying users due to the prevalence of cameras in mobile devices and their integration into everyday applications. The rapid advancement of deep Convolutional Neural Networks (CNNs) has led to numerous face verification architectures. However, these models are often large and impractical for mobile applications, reaching sizes of hundreds of megabytes with millions of parameters. We address this issue by developing SqueezerFaceNet, a light face recognition network which less than 1M parameters. This is achieved by applying a network pruning method based on Taylor scores, where filters with small importance scores are removed iteratively. Starting from an already small network (of 1.24M) based on SqueezeNet, we show that it can be further reduced (up to 40%) without an appreciable loss in performance. To the best of our knowledge, we are the first to evaluate network pruning methods for the task of face recognition. © 2024, The Author(s), under exclusive license to Springer Nature Switzerland AG.

There is a major ongoing transition from utilizing fossil fuel to electricity in buses for enabling a more sustainable, environmentally friendly, and connected transportation ecosystem. Batteries are expensive, up to 30% of the total cost for the vehicle (A. Fotouhi 2016), and considered safety-critical components for electric vehicles (EV). As they deteriorate over time, monitoring the health status and performing the maintenance accordingly in a proactive manner is crucial to achieving not only a safe and sustainable transportation system but also a cost-effective operation and thus a greater market satisfaction. As a widely used indicator, the State of Health (SOH) is a measurement that reflects the current capability of the battery in comparison to an ideal condition. Accurate estimation of SOH is important to evaluate the validity of the batteries for the intended application and can be utilized as a proxy to estimate the remaining useful life (RUL) and predict the end-of-life (EOL) of batteries for maintenance planning. The SOH is computed via an on-board computing device, i.e. battery management unit (BMU), which is commonly developed based on controlled experiments and many of them are physical-model based approaches that only depend on the internal parameters of the battery (B. Pattipati 2008; M. H. Lipu 2018). However, the deterioration processes of batteries in hybrid and full-electric buses depend not only on the designing parameters but also on the operating environment and usage patterns of the vehicle. Therefore, utilizing multiple data sources to estimate the health status and EOL of the batteries is of potential internet. In this study, a data-driven prognostic method is developed to estimate SOH and predict EOL for batteries in heterogeneous fleets of hybrid buses, using various types of data sources, e.g. physical configuration of the vehicle, deployment information, on-board sensor readings, and diagnostic fault codes. A set of new features was generated from the existing sensor readings by inducing artificial resets on each battery replacement. A neural network-based regression model achieved accurate estimates of battery SOH status. Another network was used to indicate the EOL of batteries and the result was evaluated using battery replacement based on the current maintenance strategy. © ESREL2020-PSAM15 Organizers. Published by Research Publishing, Singapore.

This paper investigates the effects of various randomization techniques on Deep Neural Networks (DNNs) learning performance. We categorize the existing randomness techniques into four key types: injection of noise/randomness at the data, model structure, optimization or learning stage. We use this classification to identify gaps in the current coverage of potential mechanisms for the introduction of randomness, leading to proposing two new techniques: adding noise to the loss function and random masking of the gradient updates. We use a Particle Swarm Optimizer (PSO) for hyperparameter optimization and evaluate over 30,000 configurations across standard computer vision benchmarks. Our study reveals that data augmentation and weight initialization randomness significantly improve performance, and different optimizers prefer distinct randomization types. The complete implementation and dataset are available on GitHub1. This paper for the Hot-off-the-Press track at GECCO 2024 summarizes the original work published at [2]. © 2024 Copyright held by the owner/author(s).

[2] Mohammed Ghaith Altarabichi, Sławomir Nowaczyk, Sepideh Pashami, Peyman Sheikholharam Mashhadi, and Julia Handl. 2024. Rolling the dice for better deep learning performance: A study of randomness techniques in deep neural networks. Information Sciences 667 (2024), 120500.

Recently and due to the advances in sensor technology and Internet-of-Things, the operation of machinery can be monitored, using a higher number of sources and modalities. In this study, we demonstrate that Multi-Modal Translation is capable of transferring knowledge from a modality with higher level of applicability (more usefulness to solve an specific task) but lower level of accessibility (how easy and affordable it is to collect information from this modality) to another one with higher level of accessibility but lower level of applicability. Unlike the fusion of multiple modalities which requires all of the modalities to be available during the deployment stage, our proposed method depends only on the more accessible one; which results in the reduction of the costs regarding instrumentation equipment. The presented case study demonstrates that by the employment of the proposed method we are capable of replacing five acceleration sensors with three current sensors, while the classification accuracy is also increased by more than 1%.

Millions of vehicles are transported every year, tightly parked in vessels or boats. To reduce the risks of associated safety issues like fires, knowing the location of vehicles is essential, since different vehicles may need different mitigation measures, e.g. electric cars. This work is aimed at creating a solution based on a nano-drone that navigates across rows of parked vehicles and detects their license plates. We do so via a wall-following algorithm, and a CNN trained to detect license plates. All computations are done in real-time on the drone, which just sends position and detected images that allow the creation of a 2D map with the position of the plates. Our solution is capable of reading all plates across eight test cases (with several rows of plates, different drone speeds, or low light) by aggregation of measurements across several drone journeys. © 2024, The Author(s), under exclusive license to Springer Nature Switzerland AG.

One of the biggest challenges in machine learning is data collection. Training data is an important part since it determines how the model will behave. In object classification, capturing a large number of images per object and in different conditions is not always possible and can be very time-consuming and tedious. Accordingly, this work explores the creation of artificial images using a game engine to cope with limited data in the training dataset. We combine real and synthetic data to train the object classification engine, a strategy that has shown to be beneficial to increase confidence in the decisions made by the classifier, which is often critical in industrial setups. To combine real and synthetic data, we first train the classifier on a massive amount of synthetic data, and then we fine-tune it on real images. Another important result is that the amount of real images needed for fine-tuning is not very high, reaching top accuracy with just 12 or 24 images per class. This substantially reduces the requirements of capturing a great amount of real data. © 2023 by SCITEPRESS-Science and Technology Publications, Lda.

The performance versus interpretability trade-off has been well-established in the literature for many years in the context of machine learning models. This paper demonstrates its twin, namely the data-centric performance versus interpretability trade-off. In a case study of bearing fault diagnosis, we found that substituting the original acceleration signal with a demodulated version offers a higher level of interpretability, but it comes at the cost of significantly lower classification performance. We demonstrate these results on two different datasets and across four different machine learning algorithms. Our results suggest that “there is no free lunch,” i.e., the contradictory relationship between interpretability and performance should be considered earlier in the analysis process than it is typically done in the literature today; in other words, already in the preprocessing and feature extraction step. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

In many practical applications data used for training a machine learning model and the deployment data does not always preserve the same distribution. Transfer learning and, in particular, domain adaptation allows to overcome this issue, by adapting the source model to a new target data distribution and therefore generalizing the knowledge from source to target domain. In this work, we present a method that makes the adaptation process more transparent by providing two complementary explanation mechanisms. The first mechanism explains how the source and target distributions are aligned in the latent space of the domain adaptation model. The second mechanism provides descriptive explanations on how the decision boundary changes in the adapted model with respect to the source model. Along with a description of a method, we also provide initial results obtained on publicly available, real-life dataset. © The Author(s) 2024.

This thesis presents a comparison of a GPU implementation of the Conjugate Residual method as a sequence of generic library kernels against implementations ofthe method with custom kernels to expose the performance gains of a keyoptimization strategy, kernel fusion, for memory-bound operations which is to makeefficient reuse of the processed data.

For massive MIMO the iterative solver is to be employed at the linear detection stageto overcome the computational bottleneck of the matrix inversion required in theequalization process, which is 𝒪(𝑛3) for direct solvers. A detailed analysis of howone more of the Krylov subspace methods that is feasible for massive MIMO can beimplemented on a GPU as a unified kernel is given.

Further, to show that kernel fusion can improve the execution performance not onlywhen the input data is large matrices-vectors as in scientific computing but also inthe case of massive MIMO and possibly similar cases where the input data is a largenumber of small matrices-vectors that must be processed in parallel.In more details, focusing on the small number of iterations required for the solver toachieve a close enough approximation of the exact solution in the case of massiveMIMO, and the case where the number of users matches the size of a warp, twodifferent approaches that allow to fully unroll the algorithm and gradually fuse allthe separate kernels into a single, until reaching a top-down hardcodedimplementation are proposed and tested.

Targeting to overcome the algorithms computational burden which is the matrixvector product, further optimization techniques such as two ways to utilize the faston-chip memories, preloading the matrix in shared memory and preloading thevector in shared memory, are tested and proposed to achieve high efficiency andhigh parallelism.

Synthetic data generation has been proposed as a potential solution to accessing Electronic Health Records (EHRs) while minimizing the privacy risks associated with real EHRs. Nevertheless, the practical use of synthetic EHRs rests on their ability to resemble the quality of real EHRs. Existing evaluations of synthetic EHRs often focus on assessing them as static snapshots frozen in time, neglecting temporal dependencies and varying temporal patterns. Moreover, some of these methods rely on subjective judgments, are limited to segmentable time-series, and employ methods that adopt a one-to-one approach. This study employs a comprehensive approach to evaluating fidelity in synthetic time-series EHRs to address these challenges. We extend the functionality of time-series analysis methods such as temporal clustering, time-series similarity measures, Sample Entropy, and trend analysis, to evaluate varying temporal patterns in synthetic time-series EHRs. Our findings provide valuable insights into how synthetic EHRs align with real EHRs in the temporal context, considering aspects such as patient groupings, temporal dynamics, predictability, and directional change. We empirically demonstrate the feasibility of assessing temporal fidelity with these methods, offering an understanding of the quality of synthetic EHRs in capturing the varying temporal patterns inherent in EHRs. © 2024 IEEE.

Due to migration, terror-threats and the viral pandemic, various EU member states have re-established internal border control or even closed their borders. European Association for Biometrics (EAB), a non-profit organisation, solicited the views of its members on ways which biometric technologies and services may be used to help with re-establishing open borders within the Schengen area while at the same time mitigating any adverse effects. From the responses received, this position paper was composed to identify ideas to re-establish free travel between the member states in the Schengen area. The paper covers the contending needs for security, open borders and fundamental rights as well as legal constraints that any technological solution must consider. A range of specific technologies for direct biometric recognition alongside complementary measures are outlined. The interrelated issues of ethical and societal considerations are also highlighted. Provided a holistic approach is adopted, it may be possible to reach a more optimal trade-off with regards to open borders while maintaining a high-level of security and protection of fundamental rights. European Association for Biometrics and its members can play an important role in fostering a shared understanding of security and mobility challenges and their solutions. © 2023 The Authors. IET Biometrics published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology.

Artificial Intelligence (AI) provides the opportunity to analyze complex transportation domains from various perspectives. Sustainability is one of the important transportation factors vital for a robust, fair, and efficient living environment and the livability of a city. This article leverages different feature engineering techniques on the New York City mobility dataset to identify the significant sustainability factors and employ the k-means clustering technique to cluster the commuters based on their transportation modes and demographics. Cluster analysis is performed based on the specified features and sustainable mode of transportation. Our cluster analysis of commuters on the New York City dataset shows that demographic information such as gender or race does not influence the sustainable mode of transportation, while the "start location"of travellers and their car access are influencing factors on sustainability. © 2022 IEEE.

Machine Activity Recognition (MAR) is a research topic that focuses on the development of data-driven methods to improve productivity monitoring. The application and the perspective of MAR research jointly influence the diffi- culty of a MAR problem. Unlike previous MAR works, which have studied construction machinery from the viewpoint of the user, this project focuses on logistics equipment from the viewpoint of the original equipment manufac- turer. In terms of the application, forklift trucks have flexible functions and complex usage. The former is an intrinsic characteristic, as forklifts are me- chanically agile, and the latter is an extrinsic factor, as forklift usage can vary greatly with different drivers, loads, work shifts, and warehouse environments. As for the standpoint, manufacturers have customers who use their products all over the world. Studying a single machine or machines in a single site, i.e. the conventional MAR setting, cannot provide a general understanding of the equipment usage. Therefore, existing MAR methods with external sensory data and only supervised learning techniques are impractical in this case.

This thesis investigates learning representation-based methods for recog- nizing forklift routine activities using on-board sensory signals. Three methods are developed to capture important data features to overcome the challenges of forklift MAR. First, by pre-training autoencoders with unlabeled data and then fine-tuning them with pseudo-labeled data, discriminative features can be ex- tracted. Classifiers built on these features can outperform conventional MAR solutions that use only the labeled data. Second, training gated recurrent unit networks to recognize activities in different contexts can help to learn a repre- sentation that captures activities and their transitions, which further improves the MAR result. Third, implementing domain adversarial-training neural net- works with pseudo-labeled data can essentially compensate for the limited la- beled data from source domains, leading to representations that are informative for more than one domain. In addition, testing the full method on a real truck has demonstrated the applicability of the proposed method and the feasibility of an online MAR solution.

As the rise of the Internet of Things has made a vast amount of sensory data readily available, research that develops data-driven methods for industrial applications has become increasingly popular. Yet, there are not many reports presenting the deployment of these methods. One can always expect “there is a gap between theory and reality,” but then, what is the gap? How big is it, and how to handle it? This paper demonstrates the deployment of machine learning (ML) models on a real forklift truck and the utilization of an interactive method that essentially bridges the gap between laboratory and realistic settings of the forklift application. The interactive method suggests a gradual adaptation to various user cases in practice: to test the offline method in an environment slightly different from what the training data presents and adjust the method according to these new usages. Additionally, the interactive model deployment allows modification of the offline method in the telematics unit of the forklift truck, which enables an immediate validation of the method adjustment. The result shows that the proposed method can effectively revise erroneous predictions from the ML method and provide quick adaptation to different forklift operations. It also gives a positive signal for further large-scale deployment of offline ML methods and shows their potential to create value and provide optimization in the industry. © 2024 IEEE.

We introduce an LSTM-based method for predicting compressor failures using aggregated sensory data, and evaluate it using historical information from over 1000 heavy duty vehicles during 2015 and 2016. The goal is to proactively identify trucks that will require maintenance in the near future, so that component replacement can be scheduled before the failure happens, translating into improved uptime. The problem is formulated as a classification task of whether a compressor failure will happen within the specified prediction horizon. A recurrent neural network using Long Short-Term Memory (LSTM) architecture is employed as the prediction model, and compared against Random Forest (RF), the solution used in industrial deployment at the moment. Experimental results show that while Random Forest slightly outperforms LSTM in terms of AUC score, the predictions of LSTM stay significantly more stable over time, showing a consistent trend from healthy to faulty class. Additionally, LSTM is also better at detecting the switch from faulty class to the healthy one after a repair. We demonstrate that this stability is important for making repair decisions, especially in questionable cases, and therefore LSTM model is likely to lead to better results in practice. © Springer Nature Switzerland AG 2019

Machine activity recognition is important for accurately esti- mating machine productivity and machine maintenance needs. In this paper, we present ongoing work on how to recognize activities of forklift trucks from on-board data streaming on the controller area network. We show that such recognition works across different sites. We first demon- strate the baseline classification performance of a Random Forest that uses 14 signals over 20 time steps, for a 280-dimensional input. Next, we show how a deep neural network can learn low-dimensional representa- tions that, with fine-tuning, achieve comparable accuracy. The proposed representation achieves machine activity recognition. Also, it visualizes the forklift operation over time and illustrates the relationships across different activities. © Springer Nature Switzerland AG 2020

The success of domain adaptation relies on high-quality labeled data from the source domain, which is a luxury setup for applied machine learning problems. This article investigates a particular challenge: the source labeled data are neither plentiful nor sufficiently representative. We studied the challenge of limited data with an industrial application, i.e., forklift truck activity recognition. The task is to develop data-driven methods to recognize forklift usage performed in different warehouses with a large scale of signals collected from the onboard sensors. The preliminary results show that using pseudo-labeled data from the source domain can significantly improve classification performance on the target domain in some tasks. As the real-world problems are much more complex than typical research settings, it is not clearly understood in what circumstance the improvement may occur. Therefore, we provided discussions regarding this phenomenon and shared several inspirations on the difficulty of understanding and debugging domain adaptation problems in practice. © 2023 IEEE.

We propose a new sequential decision-making setting, combining key aspects of two established online learning problems with bandit feedback. The optimal action to play at any given moment is contingent on an underlying changing state which is not directly observable by the agent. Each state is associated with a context distribution, possibly corrupted, allowing the agent to identify the state. Furthermore, states evolve in a Markovian fashion, providing useful information to estimate the current state via state history. In the proposed problem setting, we tackle the challenge of deciding on which of the two sources of information the agent should base its arm selection. We present an algorithm that uses a referee to dynamically combine the policies of a contextual bandit and a multi-armed bandit. We capture the time-correlation of states through iteratively learning the action-reward transition model, allowing for efficient exploration of actions. Our setting is motivated by adaptive mobile health (mHealth) interventions. Users transition through different, time-correlated, but only partially observable internal states, determining their current needs. The side information associated with each internal state might not always be reliable, and standard approaches solely rely on the context risk of incurring high regret. Similarly, some users might exhibit weaker correlations between subsequent states, leading to approaches that solely rely on state transitions risking the same. We analyze our setting and algorithm in terms of regret lower bound and upper bounds and evaluate our method on simulated medication adherence intervention data and several real-world data sets, showing improved empirical performance compared to several popular algorithms. 

An autonomous service robot should be able to interact with its environment safely and robustly without requiring human assistance. Unstructured environments are challenging for robots since the exact prediction of outcomes is not always possible. Even when the robot behaviors are well-designed, the unpredictable nature of the physical robot-object interaction may lead to failures in object manipulation. In this letter, we focus on detecting and classifying both manipulation and post-manipulation phase failures using the same exteroception setup. We cover a diverse set of failure types for primary tabletop manipulation actions. In order to detect these failures, we propose FINO-Net (Inceoglu et al., 2021), a deep multimodal sensor fusion-based classifier network architecture. FINO-Net accurately detects and classifies failures from raw sensory data without any additional information on task description and scene state. In this work, we use our extended FAILURE dataset (Inceoglu et al., 2021) with 99 new multimodal manipulation recordings and annotate them with their corresponding failure types. FINO-Net achieves 0.87 failure detection and 0.80 failure classification F1 scores. Experimental results show that FINO-Net is also appropriate for real-time use. © 2016 IEEE.

In the rapidly evolving field of AI, Explainable Artificial Intelligence (XAI) has become paramount, particularly in Intelligent Environments applications. It offers clarity and understanding in complex decision-making processes, fostering trust and enabling rigorous scrutiny. The Shapley value, renowned for its accurate quantification of feature importance, has emerged as a prevalent standard in both academic research and practical application. Nevertheless, the Shapley value's reliance on the calculation of all possible coalitions poses a significant computational challenge, as it falls within the class of NP-hard problems. Consequently, approximation techniques are employed in most practical scenarios as a substitute for precise computations. The most common of those is the SHAP (SHapley Additive exPlanations) technique, which quantifies the influence exerted by a specific feature on decision outcomes of a specific Machine Learning model. However, the Shapley value's theoretical underpinnings focus on assessing and understanding feature impact on model evaluation metrics, rather than just alterations in the responses. This paper conducts a comparative analysis using controlled synthetic data with established ground truths. It juxtaposes the practical implementation of the SHAP approach with the theoretical model in two distinct scenarios: one using the F1-score and the other, the accuracy metric. These are two representative characteristic functions, capturing different aspects and whose appropriateness depends on the specific requirements and context of the task to be solved. We analyze how the three alternatives exhibit similarity and disparity in their manifestation of feature effects. We explore the parallels and differences between these approaches in reflecting feature effects. Ultimately, our research seeks to determine the conditions under which SHAP outcomes are more aligned with either the F1-score or the accuracy metric, thereby providing valuable insights for their application in various Intelligent Environment contexts. © 2024 IEEE.

One of the most widely adopted approaches for eXplainable Artificial Intelligence (XAI) involves employing of Shapley values (SVs) to determine the relative importance of input features. While based on a solid mathematical foundation derived from cooperative game theory, SVs have a significant drawback: high computational cost. Calculating the exact SV is an NP-hard problem, necessitating the use of approximations, particularly when dealing with more than twenty features. On the other hand, determining SVs for all features is seldom necessary in practice; users are primarily interested in the most important ones only. This paper introduces the Economic Hierarchical Shapley values (ecoShap) method for calculating SVs for the most crucial features only, with reduced computational cost. EcoShap iteratively expands disjoint groups of features in a tree-like manner, avoiding the expensive computations for the majority of less important features. Our experimental results across eight datasets demonstrate that the proposed technique efficiently identifies top features; at a 50% reduction in computational costs, it can determine between three and seven of the most important features. © The Author(s) 2024.

The purpose of the study is to demonstrate the feasibility of combining traffic simulator technology with machine learning (ML) methods to create realistic and comprehensive synthetic traffic data. Synthetic data alleviates many ethical and privacy concerns, significantly reduces the costs associated with data collection, and enables researchers to study scenarios and conditions that are difficult or impossible to replicate in real-world environments. Access to large amounts of diverse and controlled data is essential for developing and testing artificial intelligence (AI) models and leads to more reliable and robust results. Traffic simulators like SUMO have been successfully used for that purpose in the past, creating realistic vehicular traces. One drawback is that, without coupling them with complex physics emulators, they are not capable of generating internal vehicle parameters. Such parameters, on the other hand, are crucial for many purposes, from understanding energy efficiency and optimizing driver behavior to predictive maintenance and monitoring the degradation of key components, such as driveline batteries. In this paper, we propose Synthetic Traffic Data Generator (STDG) and demonstrate that an ML model that is trained on the internal parameters of a vehicle in one set of conditions (Sweden) can be used to generate synthetic data corresponding to another setting (Monaco). The proposed method promises to eliminate the need for an expensive collection of the original vehicle parameters across many different settings. Moreover, sharing the synthetic data with additional stakeholders is easier due to the reduced security and integrity risk of exposing the vehicle’s privacy-sensitive original parameters. This study compares several ML techniques, including deep learning (DL) based, for generating internal parameters of vehicles, such as fuel rate, engine speed, and wet tank air pressure. Using the actual bus data from a small city to train our ML models, we attempt to forecast the internal parameters of the buses in various scenarios. The proposed method first utilizes SUMO to generate synthetic waypoints for the bus and then predicts the other parameters using the trained model, thereby producing synthetic data with internal parameters for buses operating in a new urban environment. Our preliminary results indicated that our model is performing well within a 90% confidence interval. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.

Workplace injuries are common in today’s society due to a lack of adequately worn safety equipment. A system that only admits appropriately equipped personnel can be created to improve working conditions. The goal is thus to develop a system that will improve workers’ safety using a camera that will detect the usage of Personal Protective Equipment (PPE). To this end, we collected and labeled appropriate data from several public sources, which have been used to train and evaluate several models based on the popular YOLOv4 object detector. Our focus, driven by a collaborating industrial partner, is to implement our system into an entry control point where workers must present themselves to obtain access to a restricted area. Combined with facial identity recognition, the system would ensure that only authorized people wearing appropriate equipment are granted access. A novelty of this work is that we increase the number of classes to five objects (hardhat, safety vest, safety gloves, safety glasses, and hearing protection), whereas most existing works only focus on one or two classes, usually hardhats or vests. The AI model developed provides good detection accuracy at a distance of 3 and 5 meters in the collaborative environment where we aim at operating (mAP of 99/89%, respectively). The small size of some objects or the potential occlusion by body parts have been identified as potential factors that are detrimental to accuracy, which we have counteracted via data augmentation and cropping of the body before applying PPE detection. © 2023 by SCITEPRESS-Science and Technology Publications, Lda.

The new advances in Vehicular Systems and Technologies have resulted in a sheer increase in the number of connected vehicles. These connected vehicles use IoT technologies to communicate operational signals with the OEMs, such as the vehicle’s speed, torque, temperature, load, RPM, etc. These signals have provided an unprecedented opportunity to adaptively monitor the status of each piece of the vehicle’s equipment and discover any possible risk of failure before it happens. This emerging field of study is called predictive maintenance (also known as condition-based maintenance) and has recently received much attention. In this paper, we apply Integrated Gradients (IG), an XAI method until now primarily used on image data, on datasets containing tabular and time-series data in the domain of predictive maintenance of trucks’ turbochargers. We evaluate how the results of IG differ, in these new settings, for various types of models. In particular, we investigate how the change of baseline can affect the outcome. Experimental results verify that IG can be applied successfully to both sequenced and non-sequenced data. Contrary to the opinion common in the literature, the gradient baseline does not affect the results of IG significantly, especially on models such as RNN, LSTM, and GRU, where the data contains time series; the effect is more visible for models like MLP with non-sequenced data. To confirm these findings, and to understand them deeper, we have also applied IG to SVM models, which gave the results that the choice of gradient baseline has a significant impact on the performance of SVM. (c) IARIA, 2023

This research is an interdisciplinary work between data scientists, innovation management researchers, and experts from a Swedish hygiene and health company. Based on this collaboration, we have developed a novel package for automatic idea detection to control and prevent healthcare-associated infections (HAI). The principal idea of this study is to use machine learning methods to extract informative ideas from social media to assist healthcare professionals in reducing the rate of HAI. Therefore, the proposed package offers a corpus of data collected from Twitter, associated expert-created labels, and software implementation of an annotation framework based on the Active Learning paradigm. We employed Transfer Learning and built a two-step deep neural network model that incrementally extracts the semantic representation of the collected text data using the BERTweet language model in the first step and classifies these representations as informative or non-informative using a multi-layer perception (MLP) in the second step. The package is AID4HAI (Automatic Idea Detection for controlling and preventing Healthcare-Associated Infections) and is made fully available (software code and the collected data) through a public GitHub repository (https://github.com/XaraKar/AID4HAI). We believe that sharing our ideas and releasing these ready-to-use tools contributes to the development of the field and inspires future research.

At present, decision-making solutions developed based on deep learning (DL) models have received extensive attention in predictive maintenance (PM) applications along with the rapid improvement of computing power. Relying on the superior properties of shared weights and spatial pooling, convolutional neural networks (CNNs) can learn effective representations of health states from industrial data. Many developed CNN-based schemes, such as advanced CNNs that introduce residual learning and multi-scale learning, have shown good performance in health states recognition tasks under the assumption that all the classes are known. However, these schemes have no ability to deal with new abnormal samples that belong to state classes not part of the training set. In this paper, a collective decision framework for different CNNs is proposed. It is based on a one-vs.-rest network (OVRN) to simultaneously achieve classification of known and unknown health states. OVRNs learn class-specific discriminative features and enhance the ability to reject new abnormal samples incorporated to different CNNs. According to the validation results on the public dataset of the Tennessee Eastman process (TEP), the proposed CNN-based decision schemes incorporating an OVRN have outstanding recognition ability for samples of unknown heath states while maintaining satisfactory accuracy on known states. The results show that the new DL framework outperforms state-of-the-art CNNs, and the one based on residual and multi-scale learning has the best overall performance. © 2023 by the authors.

As an important branch of machine learning, deep learning (DL) models with multiple hidden layer structures have the ability to extract highly representative features from the input. At present, fault detection and diagnosis (FDD) and health monitoring solutions developed based on DL models have received extensive attention in academia and industry along with the rapid improvement of computing power. Therefore, this paper focuses on a comprehensive review of DL model–based FDD and health monitoring schemes in view of common problems of industrial systems. First, brief theoretical backgrounds of basic DL models are introduced. Then, related publications are discussed about the development of DL and graphical models in the industrial context. Afterwards, public data sets are summarized, which are associated with several research papers. More importantly, suggestions on DL model–based diagnosis and health monitoring solutions and future developments are given. Our work will have a positive impact on the selection and design of FDD solutions based on DL and graphical models in the future. © The Author(s) 2023.

The district heating (DH) industry is facing an important transformation towards more efficient networks that utilise significantly lower water temperatures to distribute the heat. This change requires taking advantage of new technologies, and Machine Learning (ML) is a popular direction. In the last decade, we have witnessed an extreme growth in the number of published research papers that focus on applying ML techniques to the DH domain. However, based on our experience in the field, and an extensive review of the state-of-the-art, we perceive a mismatch between the most popular research directions, such as forecasting, and the challenges faced by the DH industry. In this work, we present our findings, explain and demonstrate the key gaps between the two communities and suggest a road-map ahead towards increasing the impact of ML research in the DH industry. © 2021 by the authors. Licensee MDPI, Basel, Switzerland.

In this survey, we provide a comprehensive review of more than 200 articles, technical reports, and GitHub repositories published over the last 10 years on the recent developments of deep learning techniques for iris recognition, covering broad topics on algorithm designs, open-source tools, open challenges, and emerging research. First, we conduct a comprehensive analysis of deep learning techniques developed for two main sub-tasks in iris biometrics: segmentation and recognition. Second, we focus on deep learning techniques for the robustness of iris recognition systems against presentation attacks and via human-machine pairing. Third, we delve deep into deep learning techniques for forensic application, especially in post-mortem iris recognition. Fourth, we review open-source resources and tools in deep learning techniques for iris recognition. Finally, we highlight the technical challenges, emerging research trends, and outlook for the future of deep learning in iris recognition. © 2024 Copyright held by the owner/author(s).

The discrepancy in the distribution of source and target domains is usually referred to as a domain shift. It is one of the reasons for the inferior performance of machine learning solutions at deployment. We illustrate that the domain shift issue is pertinent to the readings of the vehicles’ operational sensors. This is due to the fact that these measurements are collected over a period of time and are susceptible to various changes that happen in the meantime. Examples of these changes are usage pattern variations, aging of the vehicles, seasonal shifts, and driver changes. However, domain adversarial neural networks (DANN) have shown promising results to reduce the negative impact of the domain shift. The present study investigates domain adaptation (DA) in the predictive maintenance field by estimating the remaining useful life (RUL) of turbochargers. The devices are operating on a fleet of VOLVO trucks, and the information about their services is collected over four years between 2016 and 2019. The input features to the model are a set of bi-weekly collected measurements called logged vehicle data (LVD). The contributions of this paper are two-fold. First, we propose a new approach for detecting domain (covariate) shift using an autoencoder. Second, we adapt domain adversarial neural networks to the specific application of predicting turbocharger failures. Finally, we deploy a recurrent feature extraction layer in the DANN architecture to incorporate temporal aspect of the data. The experimental results demonstrate the superiority of the proposed method over the traditional approach.

Adverse weather conditions such as snow, rain, and fog are natural phenomena that can impair the performance of the perception algorithms in autonomous vehicles. Although LiDARs provide accurate and reliable scans of the surroundings, its output can be substantially degraded by precipitation (e.g., snow particles) leading to an undesired effect on the downstream perception tasks. Several studies have been performed to battle this undesired effect by filtering out precipitation outliers, however, these works have large memory consumption and long execution times which are not desired for onboard applications. To that end, we introduce a novel outlier detector for 3D LiDAR point clouds captured under adverse weather conditions. Our proposed detector 3D-OutDet is based on a novel convolution operation that processes nearest neighbors only, allowing the model to capture the most relevant points. This reduces the number of layers, resulting in a model with a low memory footprint and fast execution time, while producing a competitive performance compared to state-of-the-art models. We conduct extensive experiments on three different datasets (WADS, SnowyKITTI, and SemanticSpray) and show that with a sacrifice of 0.16% mIOU performance, our model reduces the memory consumption by 99.92%, number of operations by 96.87%, and execution time by 82.84% per point cloud on the real-scanned WADS dataset. Our experimental evaluations also showed that the mIOU performance of the downstream semantic segmentation task on WADS can be improved up to 5.08% after applying our proposed outlier detector. We release our source code, supplementary material and videos in https://sporsho.github.io/3DOutDet. Upon clicking the link you will have to option to go to source code, see supplementary information and view videos generated with our 3D-OutDet. © 2024 IEEE.

Knowledge graphs have proven themselves as a robust tool in clinical applications to aid patient care and help identify treatments for new diseases. They have impacted many information retrieval systems in healthcare. In this study, we construct a disease knowledge graph using Neo4j (a knowledge graph tool) for a disease database to answer complex questions that are time-consuming and labour-intensive to be answered in the previous system. We demonstrate that new information can be inferred in a knowledge graph based on existing semantic relationships between the medical concepts and the ability to perform reasoning in the knowledge graph.

Mobility as a Service (MaaS) is a relatively new domain where new types of knowledge systems have recently emerged. It combines various modes of transportation and different kinds of data to present personalized services to travellers based on transport needs. A knowledge-based framework based on Artificial Intelligence (AI) is proposed in this paper to integrate, analyze, and process different types of mobility data. The framework includes a knowledge acquisition process to extract and structure data from various sources, including mobility experts and add new information to a knowledge base. The role of AI in this framework is to aid in automatically discovering knowledge from various data sets and recommend efficient and personalized mobility services with explanations. A scenario is also presented to demonstrate the interaction of the proposed framework’s modules.

Mobility as a Service (MaaS) combines various modes of transportation to present mobility services to travellers based on their transport needs. This paper proposes a knowledge-based framework based on Artificial Intelligence (AI) to integrate various mobility data types and provide travellers with customized services. The proposed framework includes a knowledge acquisition process to extract and structure data from multiple sources of information (such as mobility experts and weather data). It also adds new information to a knowledge base and improves the quality of previously acquired knowledge. We discuss how AI can help discover knowledge from various data sources and recommend sustainable and personalized mobility services with explanations. The proposed knowledge-based AI framework is implemented using a synthetic dataset as a proof of concept. Combining different information sources to generate valuable knowledge is identified as one of the challenges in this study. Finally, explanations of the proposed decisions provide a criterion for evaluating and understanding the proposed knowledge-based AI framework. © 2023 by the authors.

In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). The AFFA module is embedded in the decoder and adaptively learns to fuse attribute features and features conditioned on identity information without requiring any additional facial segmentation process. In IFSR, we leverage the intermediate features in an identity encoder to preserve important attributes such as head pose, facial expression, lighting, and occlusion in the target face, while still transferring the identity of the source face with high fidelity. We conduct extensive quantitative and qualitative experiments on various datasets and show that the proposed FaceDancer outperforms other state-of-the-art networks in terms of identityn transfer, while having significantly better pose preservation than most of the previous methods. © 2023 IEEE.

Deep convolutional neural networks (CNNs) have been shown to predict poverty and development indicators from satellite images with surprising accuracy. This paper presents a first attempt at analyzing the CNNs responses in detail and explaining the basis for the predictions. The CNN model, while trained on relatively low resolution day- and night-time satellite images, is able to outperform human subjects who look at high-resolution images in ranking the Wealth Index categories. Multiple explainability experiments performed on the model indicate the importance of the sizes of the objects, pixel colors in the image, and provide a visualization of the importance of different structures in input images. A visualization is also provided of type images that maximize the network prediction of Wealth Index, which provides clues on what the CNN prediction is based on.

In urban settings, the prevalence of traffic lights often leads to fluctuations in traffic patterns and increased energy utilization among vehicles. Recognizing this challenge, this research addresses the adverse effects of traffic lights on the energy efficiency of electric vehicles (EVs) through the introduction of a Multi-Intersections-Based Eco-Approach and Departure strategy (M-EAD). This innovative strategy is designed to enhance various aspects of urban mobility, including vehicle energy efficiency, traffic flow optimization, and battery longevity, all while ensuring a satisfactory driving experience. The M-EAD strategy unfolds in two distinct stages: First, it optimizes eco-friendly green signal windows at traffic lights, with a primary focus on minimizing travel delays by solving the shortest path problem. Subsequently, it employs a receding horizon framework and leverages an iterative dynamic programming algorithm to refine speed trajectories. The overarching objective is to curtail energy consumption and reduce battery wear by identifying the optimal speed trajectory for EVs in urban environments. Furthermore, the research substantiates the real-world efficacy of this approach through on-road vehicle tests, attesting to its viability and practicality in actual road scenarios. In the proposed case, the simulation results showcase notable achievements, with energy consumption reduced by 0.92% and battery wear minimized to a mere 0.0017%. This research, driven by the pressing issue of urban traffic energy efficiency, not only presents a solution in the form of the M-EAD strategy but also contributes to the fields of sustainable urban mobility and EV performance optimization. By tackling the challenges posed by traffic lights, this work offers valuable insights and practical implications for improving the sustainability and efficiency of urban transportation systems. © 2023 by the authors.

Federated Learning (FL) allows several clients to construct a common global machine-learning model without having to share their data. FL, however, faces the challenge of statistical heterogeneity between the client's data, which degrades performance and slows down the convergence toward the global model. In this paper, we provide theoretical proof that minimizing heterogeneity between clients facilitates the convergence of a global model for every single client. This becomes particularly important under empirical concept shifts among clients, rather than merely considering imbalanced classes, which have been studied until now. Therefore, we propose a method for knowledge transfer between clients where the server trains client-specific generators. Each generator generates samples for the corresponding client to remove the conflict with other clients' models. Experiments conducted on synthetic and real data, along with a theoretical study, support the effectiveness of our method in constructing a well-generalizable global model by reducing the conflict between local models.

Federated Learning (FL) is a setting where different clients collaboratively train a Machine Learning model in a privacy-preserving manner, i.e., without the requirement to share data. Given the importance of security and privacy in real-world applications, FL is gaining popularity in many areas, including predictive maintenance. For example, it allows independent companies to construct a model collaboratively. However, since different companies operate in different environments, their working conditions may differ, resulting in heterogeneity among their data distributions. This paper considers the fault identification problem and simulates different scenarios of data heterogeneity. Such a setting remains challenging for popular FL algorithms, and thus we demonstrate the considerations to be taken into account when designing federated predictive maintenance solutions.  

Deformable object manipulations, such as those involving textiles, present a significant challenge due to their high dimensionality and complexity. In this paper, we propose a solution for estimating semantic states in cloth manipulation tasks. To this end, we introduce a new, large-scale, fully-annotated RGB image dataset of semantic states featuring a diverse range of human demonstrations of various complex cloth manipulations. This effectively transforms the problem of action recognition into a classification task. We then evaluate the generalizability of our approach by employing domain adaptation techniques to transfer knowledge from human demonstrations to two distinct robotic platforms: Kinova and UR robots. Additionally, we further improve performance by utilizing a semantic state graph learned from human manipulation data. © 2024 by SCITEPRESS – Science and Technology Publications, Lda.

Few-shot meta-learning involves training a model on multiple tasks to enable it to efficiently adapt to new, previously unseen tasks with only a limited number of samples. However, current meta-learning methods assume that all tasks are closely related and belong to a common domain, whereas in practice, tasks can be highly diverse and originate from multiple domains, resulting in a multimodal task distribution. This poses a challenge for existing methods as they struggle to learn a shared representation that can be easily adapted to all tasks within the distribution. To address this challenge, we propose a meta-learning framework that can handle multimodal task distributions by conditioning the model on the current task, resulting in a faster adaptation. Our proposed method learns to encode each task and generate task embeddings that modulate the model’s activations. The resulting modulated model becomes specialized for the current task and leads to more effective adaptation. Our framework is designed to work in a realistic setting where the mode from which a task is sampled is unknown. Nonetheless, we also explore the possibility of incorporating auxiliary information, such as the task-mode-label, to further enhance the performance of our method if such information is available. We evaluate our proposed framework on various few-shot regression and image classification tasks, demonstrating its superiority over other state-of-the-art meta-learning methods. The results highlight the benefits of learning to embed task-specific information in the model to guide the adaptation when tasks are sampled from a multimodal distribution. © The Author(s) 2024.

Federated learning has emerged as a promising approach for training machine learning models on decentralized data sources while preserving data privacy. However, challenges such as communication bottlenecks, heterogeneity of client devices, and non-i.i.d. data distribution pose significant obstacles to achieving optimal model performance. We propose a novel framework that combines federated learning with meta-learning techniques to enhance both efficiency and generalization capabilities. Our approach introduces a federated modulator that learns contextual information from data batches and uses this knowledge to generate modulation parameters. These parameters dynamically adjust the activations of a base model, which operates using a MAML-based approach for model personalization. Experimental results across diverse datasets highlight the improvements in convergence speed and model performance compared to existing federated learning approaches. These findings highlight the potential of incorporating contextual information and meta-learning techniques into federated learning, paving the way for advancements in distributed machine learning paradigms. Copyright © 2024 by SIAM.