hh.sePublications
Change search
Link to record
Permanent link

Direct link
Publications (10 of 53) Show all publications
Altarabichi, M. G., Alabdallah, A., Pashami, S., Ohlsson, M., Rögnvaldsson, T. & Nowaczyk, S. (2024). Improving Concordance Index in Regression-based Survival Analysis: Discovery of Loss Function for Neural Networks.
Open this publication in new window or tab >>Improving Concordance Index in Regression-based Survival Analysis: Discovery of Loss Function for Neural Networks
Show others...
2024 (English)Manuscript (preprint) (Other academic)
Abstract [en]

In this work, we use an Evolutionary Algorithm (EA) to discover a novel Neural Network (NN) regression-based survival loss function with the aim of improving the C-index performance. Our contribution is threefold; firstly, we propose an evolutionary meta-learning algorithm SAGA$_{loss}$ for optimizing a neural-network regression-based loss function that maximizes the C-index; our algorithm consistently discovers specialized loss functions that outperform MSCE. Secondly, based on our analysis of the evolutionary search results, we highlight a non-intuitive insight that signifies the importance of the non-zero gradient for the censored cases part of the loss function, a property that is shown to be useful in improving concordance. Finally, based on this insight, we propose MSCE$_{Sp}$, a novel survival regression loss function that can be used off-the-shelf and generally performs better than the Mean Squared Error for censored cases. We performed extensive experiments on 19 benchmark datasets to validate our findings.

Keywords
evolutionary meta-learning, loss function, neural networks, survival analysis, regression
National Category
Computer Systems
Identifiers
urn:nbn:se:hh:diva-52468 (URN)
Note

As manuscript in thesis.

Available from: 2024-01-24 Created: 2024-01-24 Last updated: 2024-02-05Bibliographically approved
Altarabichi, M. G., Nowaczyk, S., Pashami, S., Sheikholharam Mashhadi, P. & Handl, J. (2024). Rolling the Dice for Better Deep Learning Performance: A Study of Randomness Techniques in Deep Neural Networks.
Open this publication in new window or tab >>Rolling the Dice for Better Deep Learning Performance: A Study of Randomness Techniques in Deep Neural Networks
Show others...
2024 (English)Manuscript (preprint) (Other (popular science, discussion, etc.))
Abstract [en]

This paper presents a comprehensive empirical investigation into the interactions between various randomness techniques in Deep Neural Networks (DNNs) and how they contribute to network performance. It is well-established that injecting randomness into the training process of DNNs, through various approaches at different stages, is often beneficial for reducing overfitting and improving generalization. However, the interactions between randomness techniques such as weight noise, dropout, and many others remain poorly understood. Consequently, it is challenging to determine which methods can be effectively combined to optimize DNN performance. To address this issue, we categorize the existing randomness techniques into four key types: data, model, optimization, and learning. We use this classification to identify gaps in the current coverage of potential mechanisms for the introduction of noise, leading to proposing two new techniques: adding noise to the loss function and random masking of the gradient updates.

In our empirical study, we employ a Particle Swarm Optimizer (PSO) to explore the space of possible configurations to answer where and how much randomness should be injected to maximize DNN performance. We assess the impact of various types and levels of randomness for DNN architectures applied to standard computer vision benchmarks: MNIST, FASHION-MNIST, CIFAR10, and CIFAR100. Across more than 30\,000 evaluated configurations, we perform a detailed examination of the interactions between randomness techniques and their combined impact on DNN performance. Our findings reveal that randomness in data augmentation and in weight initialization are the main contributors to performance improvement. Additionally, correlation analysis demonstrates that different optimizers, such as Adam and Gradient Descent with Momentum, prefer distinct types of randomization during the training process. A GitHub repository with the complete implementation and generated dataset is available\footnote[1]{https://github.com/Ghaith81/Radnomness\_in\_Neural\_Network}.

Keywords
Neural Networks, Randomized Neural Networks, Convolutional Neural Network, hyperparameter optimization, Particle swarm optimization
National Category
Computer Systems
Identifiers
urn:nbn:se:hh:diva-52467 (URN)
Note

As manuscript in thesis.

Available from: 2024-01-24 Created: 2024-01-24 Last updated: 2024-02-05Bibliographically approved
Rajabi, E., Nowaczyk, S., Pashami, S., Bergquist, M., Ebby, G. S. & Wajid, S. (2023). A Knowledge-Based AI Framework for Mobility as a Service. Sustainability, 15(3), Article ID 2717.
Open this publication in new window or tab >>A Knowledge-Based AI Framework for Mobility as a Service
Show others...
2023 (English)In: Sustainability, E-ISSN 2071-1050, Vol. 15, no 3, article id 2717Article in journal (Refereed) Published
Abstract [en]

Mobility as a Service (MaaS) combines various modes of transportation to present mobility services to travellers based on their transport needs. This paper proposes a knowledge-based framework based on Artificial Intelligence (AI) to integrate various mobility data types and provide travellers with customized services. The proposed framework includes a knowledge acquisition process to extract and structure data from multiple sources of information (such as mobility experts and weather data). It also adds new information to a knowledge base and improves the quality of previously acquired knowledge. We discuss how AI can help discover knowledge from various data sources and recommend sustainable and personalized mobility services with explanations. The proposed knowledge-based AI framework is implemented using a synthetic dataset as a proof of concept. Combining different information sources to generate valuable knowledge is identified as one of the challenges in this study. Finally, explanations of the proposed decisions provide a criterion for evaluating and understanding the proposed knowledge-based AI framework. © 2023 by the authors.

Place, publisher, year, edition, pages
Basel: MDPI, 2023
Keywords
mobility as a service, knowledge-based, explainability
National Category
Computer Sciences
Research subject
Smart Cities and Communities
Identifiers
urn:nbn:se:hh:diva-49970 (URN)10.3390/su15032717 (DOI)000929663500001 ()2-s2.0-85148043364 (Scopus ID)
Funder
Knowledge Foundation, 20180181
Available from: 2023-02-14 Created: 2023-02-14 Last updated: 2023-08-21Bibliographically approved
Taghiyarrenani, Z., Nowaczyk, S. & Pashami, S. (2023). Analysis of Statistical Data Heterogeneity in Federated Fault Identification. In: Takehisa Yairi; Samir Khan; Seiji Tsutsumi (Ed.), Proceedings of the Asia Pacific Conference of the PHM Society 2023: . Paper presented at 4th Asia Pacific Conference of the Prognostics and Health Management, Tokyo, Japan, September 11-14, 2023. New York: The Prognostics and Health Management Society, 4
Open this publication in new window or tab >>Analysis of Statistical Data Heterogeneity in Federated Fault Identification
2023 (English)In: Proceedings of the Asia Pacific Conference of the PHM Society 2023 / [ed] Takehisa Yairi; Samir Khan; Seiji Tsutsumi, New York: The Prognostics and Health Management Society , 2023, Vol. 4Conference paper, Published paper (Refereed)
Abstract [en]

Federated Learning (FL) is a setting where different clients collaboratively train a Machine Learning model in a privacy-preserving manner, i.e., without the requirement to share data. Given the importance of security and privacy in real-world applications, FL is gaining popularity in many areas, including predictive maintenance. For example, it allows independent companies to construct a model collaboratively. However, since different companies operate in different environments, their working conditions may differ, resulting in heterogeneity among their data distributions. This paper considers the fault identification problem and simulates different scenarios of data heterogeneity. Such a setting remains challenging for popular FL algorithms, and thus we demonstrate the considerations to be taken into account when designing federated predictive maintenance solutions.  

Place, publisher, year, edition, pages
New York: The Prognostics and Health Management Society, 2023
Series
Proceedings of the Asia Pacific Conference of the PHM Society, E-ISSN 2994-7219
Keywords
Predictive Maintenance, Federated Learning, Predictive Maintenance Federated Learning Statistical Heterogeneity
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-52478 (URN)10.36001/phmap.2023.v4i1.3708 (DOI)
Conference
4th Asia Pacific Conference of the Prognostics and Health Management, Tokyo, Japan, September 11-14, 2023
Funder
Vinnova
Available from: 2024-01-25 Created: 2024-01-25 Last updated: 2024-01-31Bibliographically approved
Alabdallah, A., Rögnvaldsson, T., Fan, Y., Pashami, S. & Ohlsson, M. (2023). Discovering Premature Replacements in Predictive Maintenance Time-to-Event Data. In: Takehisa Yairi; Samir Khan; Seiji Tsutsumi (Ed.), Proceedings of the Asia Pacific Conference of the PHM Society 2023: . Paper presented at 4th Asia Pacific Conference of the Prognostics and Health Management, Tokyo, Japan, September 11-14, 2023. New York: The Prognostics and Health Management Society, 4
Open this publication in new window or tab >>Discovering Premature Replacements in Predictive Maintenance Time-to-Event Data
Show others...
2023 (English)In: Proceedings of the Asia Pacific Conference of the PHM Society 2023 / [ed] Takehisa Yairi; Samir Khan; Seiji Tsutsumi, New York: The Prognostics and Health Management Society , 2023, Vol. 4Conference paper, Published paper (Refereed)
Abstract [en]

Time-To-Event (TTE) modeling using survival analysis in industrial settings faces the challenge of premature replacements of machine components, which leads to bias and errors in survival prediction. Typically, TTE survival data contains information about components and if they had failed or not up to a certain time. For failed components, the time is noted, and a failure is referred to as an event. A component that has not failed is denoted as censored. In industrial settings, in contrast to medical settings, there can be considerable uncertainty in an event; a component can be replaced before it fails to prevent operation stops or because maintenance staff believe that the component is faulty. This shows up as “no fault found” in warranty studies, where a significant proportion of replaced components may appear fault-free when tested or inspected after replacement.

In this work, we propose an expectation-maximization-like method for discovering such premature replacements in survival data. The method is a two-phase iterative algorithm employing a genetic algorithm in the maximization phase to learn better event assignments on a validation set. The learned labels through iterations are accumulated and averaged to be used to initialize the following expectation phase. The assumption is that the more often the event is selected, the more likely it is to be an actual failure and not a “no fault found”.

Experiments on synthesized and simulated data show that the proposed method can correctly detect a significant percentage of premature replacement cases.

Place, publisher, year, edition, pages
New York: The Prognostics and Health Management Society, 2023
Series
Proceedings of the Asia Pacific Conference of the PHM Society, E-ISSN 2994-7219
Keywords
Survival Analysis, Predictive Maintenance, Early Replacements, Genetic Algorithms
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-52105 (URN)10.36001/phmap.2023.v4i1.3609 (DOI)
Conference
4th Asia Pacific Conference of the Prognostics and Health Management, Tokyo, Japan, September 11-14, 2023
Funder
Knowledge Foundation, 20200001
Note

Som manuscript i avhandling/As manuscript in thesis.

Available from: 2023-11-23 Created: 2023-11-23 Last updated: 2023-12-19Bibliographically approved
Altarabichi, M. G., Pashami, S., Nowaczyk, S. & Sheikholharam Mashhadi, P. (2023). Fast Genetic Algorithm For Feature Selection — A Qualitative Approximation Approach. In: Evolutionary Computation Conference Companion (GECCO ’23 Companion), July 15–19, 2023, Lisbon, Portugal: . Paper presented at 2023 Genetic and Evolutionary Computation Conference Companion, GECCO 2023 Companion, 15-19 July, 2023 (pp. 11-12). New York, NY: Association for Computing Machinery (ACM)
Open this publication in new window or tab >>Fast Genetic Algorithm For Feature Selection — A Qualitative Approximation Approach
2023 (English)In: Evolutionary Computation Conference Companion (GECCO ’23 Companion), July 15–19, 2023, Lisbon, Portugal, New York, NY: Association for Computing Machinery (ACM), 2023, p. 11-12Conference paper, Published paper (Refereed)
Abstract [en]

We propose a two-stage surrogate-assisted evolutionary approach to address the computational issues arising from using Genetic Algorithm (GA) for feature selection in a wrapper setting for large datasets. The proposed approach involves constructing a lightweight qualitative meta-model by sub-sampling data instances and then using this meta-model to carry out the feature selection task. We define "Approximation Usefulness" to capture the necessary conditions that allow the meta-model to lead the evolutionary computations to the correct maximum of the fitness function. Based on our procedure we create CHCQX a Qualitative approXimations variant of the GA-based algorithm CHC (Cross generational elitist selection, Heterogeneous recombination and Cataclysmic mutation). We show that CHCQX converges faster to feature subset solutions of significantly higher accuracy, particularly for large datasets with over 100K instances. We also demonstrate the applicability of our approach to Swarm Intelligence (SI), with results of PSOQX, a qualitative approximation adaptation of the Particle Swarm Optimization (PSO) method. A GitHub repository with the complete implementation is available2. This paper for the Hot-off-the-Press track at GECCO 2023 summarizes the original work published at [3].

References

[1] Mohammed Ghaith Altarabichi, Yuantao Fan, Sepideh Pashami, Peyman Sheikholharam Mashhadi, and Sławomir Nowaczyk. 2021. Extracting invariant features for predicting state of health of batteries in hybrid energy buses. In 2021 ieee 8th international conference on data science and advanced analytics (dsaa). IEEE, 1–6.

[2] Mohammed Ghaith Altarabichi, Sławomir Nowaczyk, Sepideh Pashami, and Peyman Sheikholharam Mashhadi. 2021. Surrogate-assisted genetic algorithm for wrapper feature selection. In 2021 IEEE Congress on Evolutionary Computation (CEC). IEEE, 776–785.

[3] Mohammed Ghaith Altarabichi, Sławomir Nowaczyk, Sepideh Pashami, and Peyman Sheikholharam Mashhadi. 2023. Fast Genetic Algorithm for feature selection—A qualitative approximation approach. Expert systems with applications 211 (2023), 118528.

© 2023 Copyright held by the owner/author(s).

Place, publisher, year, edition, pages
New York, NY: Association for Computing Machinery (ACM), 2023
Keywords
Evolutionary computation, Feature selection, Fitness approximation, Genetic Algorithm, Particle Swarm Intelligence
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-51635 (URN)10.1145/3583133.3595823 (DOI)2-s2.0-85168991798 (Scopus ID)9798400701207 (ISBN)
Conference
2023 Genetic and Evolutionary Computation Conference Companion, GECCO 2023 Companion, 15-19 July, 2023
Available from: 2023-09-19 Created: 2023-09-19 Last updated: 2023-09-19Bibliographically approved
Altarabichi, M. G., Nowaczyk, S., Pashami, S. & Sheikholharam Mashhadi, P. (2023). Fast Genetic Algorithm for feature selection — A qualitative approximation approach. Expert systems with applications, 211, Article ID 118528.
Open this publication in new window or tab >>Fast Genetic Algorithm for feature selection — A qualitative approximation approach
2023 (English)In: Expert systems with applications, ISSN 0957-4174, E-ISSN 1873-6793, Vol. 211, article id 118528Article in journal (Refereed) Published
Abstract [en]

Evolutionary Algorithms (EAs) are often challenging to apply in real-world settings since evolutionary computations involve a large number of evaluations of a typically expensive fitness function. For example, an evaluation could involve training a new machine learning model. An approximation (also known as meta-model or a surrogate) of the true function can be used in such applications to alleviate the computation cost. In this paper, we propose a two-stage surrogate-assisted evolutionary approach to address the computational issues arising from using Genetic Algorithm (GA) for feature selection in a wrapper setting for large datasets. We define “Approximation Usefulness” to capture the necessary conditions to ensure correctness of the EA computations when an approximation is used. Based on this definition, we propose a procedure to construct a lightweight qualitative meta-model by the active selection of data instances. We then use a meta-model to carry out the feature selection task. We apply this procedure to the GA-based algorithm CHC (Cross generational elitist selection, Heterogeneous recombination and Cataclysmic mutation) to create a Qualitative approXimations variant, CHCQX. We show that CHCQX converges faster to feature subset solutions of significantly higher accuracy (as compared to CHC), particularly for large datasets with over 100K instances. We also demonstrate the applicability of the thinking behind our approach more broadly to Swarm Intelligence (SI), another branch of the Evolutionary Computation (EC) paradigm with results of PSOQX, a qualitative approximation adaptation of the Particle Swarm Optimization (PSO) method. A GitHub repository with the complete implementation is available. © 2022 The Author(s)

Place, publisher, year, edition, pages
Oxford: Elsevier, 2023
Keywords
Evolutionary computation, Feature selection, Fitness approximation, Genetic Algorithm, Meta-model, Optimization, Particle Swarm Intelligence
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-48909 (URN)10.1016/j.eswa.2022.118528 (DOI)000992359900001 ()2-s2.0-85137157028 (Scopus ID)
Note

Som manuscript i avhandling/As manuscript in thesis

Available from: 2022-12-16 Created: 2022-12-16 Last updated: 2024-01-24Bibliographically approved
Chen, K., Rögnvaldsson, T., Nowaczyk, S., Pashami, S., Klang, J. & Sternelöv, G. (2023). Material handling machine activity recognition by context ensemble with gated recurrent units. Engineering applications of artificial intelligence, 126(Part C), Article ID 106992.
Open this publication in new window or tab >>Material handling machine activity recognition by context ensemble with gated recurrent units
Show others...
2023 (English)In: Engineering applications of artificial intelligence, ISSN 0952-1976, E-ISSN 1873-6769, Vol. 126, no Part C, article id 106992Article in journal (Refereed) Published
Abstract [en]

Research on machine activity recognition (MAR) is drawing more attention because MAR can provide productivity monitoring for efficiency optimization, better maintenance scheduling, product design improvement, and potential material savings. A particular challenge of MAR for human-operated machines is the overlap when transiting from one activity to another: during transitions, operators often perform two activities simultaneously, e.g., lifting the fork already while approaching a rack, so the exact time when one activity ends and another begins is uncertain. Machine learning models are often uncertain during such activity transitions, and we propose a novel ensemble-based method adapted to fuzzy transitions in a forklift MAR problem. Unlike traditional ensembles, where models in the ensemble are trained on different subsets of data, or with costs that force them to be diverse in their responses, our approach is to train a single model that predicts several activity labels, each under a different context. These individual predictions are not made by independent networks but are made using a structure that allows for sharing important features, i.e., a context ensemble. The results show that the gated recurrent unit network can provide medium or strong confident context ensembles for 95% of the cases in the test set, and the final forklift MAR result achieves accuracies of 97% for driving and 90% for load-handling activities. This study is the first to highlight the overlapping activity issue in MAR problems and to demonstrate that the recognition results can be significantly improved by designing a machine learning framework that addresses this issue. © 2023 The Author(s)

Place, publisher, year, edition, pages
Oxford: Elsevier, 2023
Keywords
Context ensemble, Gated recurrent unit, Machine activity recognition, Material handling, Productivity monitoring
National Category
Computer Sciences Production Engineering, Human Work Science and Ergonomics
Research subject
Smart Cities and Communities
Identifiers
urn:nbn:se:hh:diva-48552 (URN)10.1016/j.engappai.2023.106992 (DOI)001070748600001 ()2-s2.0-85169031390 (Scopus ID)
Funder
Knowledge Foundation, 20200001
Note

As manuscript in thesis. 

Funding agency: Toyota Material Handling Manufacturing Sweden AB

Available from: 2022-10-31 Created: 2022-10-31 Last updated: 2023-11-28Bibliographically approved
Taghiyarrenani, Z., Nowaczyk, S., Pashami, S. & Bouguelia, M.-R. (2023). Multi-Domain Adaptation for Regression under Conditional Distribution Shift. Expert systems with applications, 224, Article ID 119907.
Open this publication in new window or tab >>Multi-Domain Adaptation for Regression under Conditional Distribution Shift
2023 (English)In: Expert systems with applications, ISSN 0957-4174, E-ISSN 1873-6793, Vol. 224, article id 119907Article in journal (Refereed) Published
Abstract [en]

Domain adaptation (DA) methods facilitate cross-domain learning by minimizing the marginal or conditional distribution shift between domains. However, the conditional distribution shift is not well addressed by existing DA techniques for the cross-domain regression learning task. In this paper, we propose Multi-Domain Adaptation for Regression under Conditional shift (DARC) method. DARC constructs a shared feature space such that linear regression on top of that space generalizes to all domains. In other words, DARC aligns different domains and makes explicit the task-related information encoded in the values of the dependent variable. It is achieved using a novel Pairwise Similarity Preserver (PSP) loss function. PSP incentivizes the differences between the outcomes of any two samples, regardless of their domain(s), to match the distance between these samples in the constructed space.

We perform experiments in both two-domain and multi-domain settings. The two-domain setting is helpful, especially when one domain contains few available labeled samples and can benefit from adaptation to a domain with many labeled samples. The multi-domain setting allows several domains, each with limited data, to be adapted collectively; thus, multiple domains compensate for each other’s lack of data. The results from all the experiments conducted both on synthetic and real-world datasets confirm the effectiveness of DARC. © 2023 The Authors

Place, publisher, year, edition, pages
Oxford: Elsevier, 2023
Keywords
Regression, Multi-Domain Adaptation, Conditional Shift, Concept Shift, Neural Networks, Siamese neural networks
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-47894 (URN)10.1016/j.eswa.2023.119907 (DOI)000966508000001 ()2-s2.0-85151474329 (Scopus ID)
Funder
VinnovaKnowledge Foundation
Available from: 2022-08-17 Created: 2022-08-17 Last updated: 2024-01-31Bibliographically approved
Bobek, S., Nowaczyk, S., Pashami, S., Taghiyarrenani, Z. & Nalepa, G. J. (2023). Towards Explainable Deep Domain Adaptation. In: Sławomir Nowaczyk, Przemysław Biecek, Neo Christopher Chung, Mauro Vallati, Paweł Skruch, Joanna Jaworek-Korjakowska... (Ed.), Artificial Intelligence. ECAI 2023 International Workshops: Part 1. Paper presented at European Conference on Artificial Intelligence (ECAI 2023), Krakow, Poland, September 30 - October 4, 2023 (pp. 101-113). Cham: Springer
Open this publication in new window or tab >>Towards Explainable Deep Domain Adaptation
Show others...
2023 (English)In: Artificial Intelligence. ECAI 2023 International Workshops: Part 1 / [ed] Sławomir Nowaczyk, Przemysław Biecek, Neo Christopher Chung, Mauro Vallati, Paweł Skruch, Joanna Jaworek-Korjakowska..., Cham: Springer, 2023, p. 101-113Conference paper, Published paper (Refereed)
Abstract [en]

In many practical applications data used for training a machine learning model and the deployment data does not always preserve the same distribution. Transfer learning and, in particular, domain adaptation allows to overcome this issue, by adapting the source model to a new target data distribution and therefore generalizing the knowledge from source to target domain. In this work, we present a method that makes the adaptation process more transparent by providing two complementary explanation mechanisms. The first mechanism explains how the source and target distributions are aligned in the latent space of the domain adaptation model. The second mechanism provides descriptive explanations on how the decision boundary changes in the adapted model with respect to the source model. Along with a description of a method, we also provide initial results obtained on publicly available, real-life dataset.

Place, publisher, year, edition, pages
Cham: Springer, 2023
Series
Communications in Computer and Information Science (CCIS), ISSN 1865-0929, E-ISSN 1865-0937 ; 1947
Keywords
Explainable AI (XAI, Domain adaptation, artificial intelligence
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:hh:diva-52508 (URN)10.1007/978-3-031-50396-2_6 (DOI)978-3-031-50395-5 (ISBN)978-3-031-50396-2 (ISBN)
Conference
European Conference on Artificial Intelligence (ECAI 2023), Krakow, Poland, September 30 - October 4, 2023
Note

CC BY 4.0

Available from: 2024-01-31 Created: 2024-01-31 Last updated: 2024-02-01Bibliographically approved
Projects
Data-Driven Predictive Maintenance for Trucks [2016-03451_Vinnova]; Halmstad University
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0003-3272-4145

Search in DiVA

Show all publications