hh.sePublications
Change search
Link to record
Permanent link

Direct link
Bouguelia, Mohamed-RafikORCID iD iconorcid.org/0000-0002-2859-6155
Publications (10 of 27) Show all publications
Vettoruzzo, A., Bouguelia, M.-R., Vanschoren, J., Rögnvaldsson, T. & Santosh, K. (2024). Advances and Challenges in Meta-Learning: A Technical Review. IEEE Transactions on Pattern Analysis and Machine Intelligence
Open this publication in new window or tab >>Advances and Challenges in Meta-Learning: A Technical Review
Show others...
2024 (English)In: IEEE Transactions on Pattern Analysis and Machine Intelligence, ISSN 0162-8828, E-ISSN 1939-3539Article, review/survey (Refereed) Epub ahead of print
Abstract [en]

Meta-learning empowers learning systems with the ability to acquire knowledge from multiple tasks, enabling faster adaptation and generalization to new tasks. This review provides a comprehensive technical overview of meta-learning, emphasizing its importance in real-world applications where data may be scarce or expensive to obtain. The paper covers the state-of-the-art meta-learning approaches and explores the relationship between meta-learning and multi-task learning, transfer learning, domain adaptation and generalization, selfsupervised learning, personalized federated learning, and continual learning. By highlighting the synergies between these topics and the field of meta-learning, the paper demonstrates how advancements in one area can benefit the field as a whole, while avoiding unnecessary duplication of efforts. Additionally, the paper delves into advanced meta-learning topics such as learning from complex multi-modal task distributions, unsupervised metalearning, learning to efficiently adapt to data distribution shifts, and continual meta-learning. Lastly, the paper highlights open problems and challenges for future research in the field. By synthesizing the latest research developments, this paper provides a thorough understanding of meta-learning and its potential impact on various machine learning applications. We believe that this technical overview will contribute to the advancement of meta-learning and its practical implications in addressing realworld problems.

Place, publisher, year, edition, pages
Piscataway, NJ: IEEE, 2024
Keywords
Adaptation models, Data models, deep neural networks, few-shot learning, Meta-learning, Metalearning, representation learning, Surveys, Task analysis, Training, transfer learning, Transfer learning
National Category
Robotics
Identifiers
urn:nbn:se:hh:diva-52730 (URN)10.1109/TPAMI.2024.3357847 (DOI)2-s2.0-85183973598 (Scopus ID)
Funder
Knowledge Foundation
Available from: 2024-02-23 Created: 2024-02-23 Last updated: 2024-02-23Bibliographically approved
Vettoruzzo, A., Bouguelia, M.-R. & Rögnvaldsson, T. (2024). Meta-learning for efficient unsupervised domain adaptation. Neurocomputing, 574, Article ID 127264.
Open this publication in new window or tab >>Meta-learning for efficient unsupervised domain adaptation
2024 (English)In: Neurocomputing, ISSN 0925-2312, E-ISSN 1872-8286, Vol. 574, article id 127264Article in journal (Refereed) In press
Abstract [en]

The standard machine learning assumption that training and test data are drawn from the same probability distribution does not hold in many real-world applications due to the inability to reproduce testing conditions at training time. Existing unsupervised domain adaption (UDA) methods address this problem by learning a domain-invariant feature space that performs well on available source domain(s) (labeled training data) and the specific target domain (unlabeled test data). In contrast, instead of simply adapting to domains, this paper aims for an approach that learns to adapt effectively to new unlabeled domains. To do so, we leverage meta-learning to optimize a neural network such that an unlabeled adaptation of its parameters to any domain would yield a good generalization on this latter. The experimental evaluation shows that the proposed approach outperforms standard approaches even when a small amount of unlabeled test data is used for adaptation, demonstrating the benefit of meta-learning prior knowledge from various domains to solve UDA problems.

Place, publisher, year, edition, pages
Amsterdam: Elsevier, 2024
Keywords
Domain adaptation, Meta-learning, Unsupervised learning, Distribution shift
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-52450 (URN)10.1016/j.neucom.2024.127264 (DOI)
Funder
Knowledge Foundation
Available from: 2024-01-22 Created: 2024-01-22 Last updated: 2024-01-22Bibliographically approved
Vettoruzzo, A., Bouguelia, M.-R. & Rögnvaldsson, T. (2023). Meta-Learning from Multimodal Task Distributions Using Multiple Sets of Meta-Parameters. In: 2023 International Joint Conference on Neural Networks (IJCNN): . Paper presented at International Joint Conference on Neural Networks (IJCNN 2023), Gold Coast, Australia, 18-23 June, 2023 (pp. 1-8). Piscataway, NJ: IEEE
Open this publication in new window or tab >>Meta-Learning from Multimodal Task Distributions Using Multiple Sets of Meta-Parameters
2023 (English)In: 2023 International Joint Conference on Neural Networks (IJCNN), Piscataway, NJ: IEEE, 2023, p. 1-8Conference paper, Published paper (Refereed)
Abstract [en]

Meta-learning or learning to learn involves training a model on various learning tasks in a way that allows it to quickly learn new tasks from the same distribution using only a small amount of training data (i.e., few-shot learning). Current meta-learning methods implicitly assume that the distribution over tasks is unimodal and consists of tasks belonging to a common domain, which significantly reduces the variety of task distributions they can handle. However, in real-world applications, tasks are often very diverse and come from multiple different domains, making it challenging to meta-learn common knowledge shared across the entire task distribution. In this paper, we propose a method for meta-learning from a multimodal task distribution. The proposed method learns multiple sets of meta-parameters (acting as different initializations of a neural network model) and uses a task encoder to select the best initialization to fine-tune for a new task. More specifically, with a few training examples from a task sampled from an unknown mode, the proposed method predicts which set of meta-parameters (i.e., model’s initialization) would lead to a fast adaptation and a good post-adaptation performance on that task. We evaluate the proposed method on a diverse set of few-shot regression and image classification tasks. The results demonstrate the superiority of the proposed method compared to other state of-the-art meta-learning methods and the benefit of learning multiple model initializations when tasks are sampled from a multimodal task distribution. © 2023 IEEE.

Place, publisher, year, edition, pages
Piscataway, NJ: IEEE, 2023
Keywords
Meta-Learning, Few-Shot Learning, Transfer Learning, Task Representation, Multimodal Distribution
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-51352 (URN)10.1109/IJCNN54540.2023.10191944 (DOI)001046198707013 ()2-s2.0-85169561819 (Scopus ID)978-1-6654-8867-9 (ISBN)
Conference
International Joint Conference on Neural Networks (IJCNN 2023), Gold Coast, Australia, 18-23 June, 2023
Available from: 2023-08-10 Created: 2023-08-10 Last updated: 2023-12-05Bibliographically approved
Taghiyarrenani, Z., Nowaczyk, S., Pashami, S. & Bouguelia, M.-R. (2023). Multi-Domain Adaptation for Regression under Conditional Distribution Shift. Expert systems with applications, 224, Article ID 119907.
Open this publication in new window or tab >>Multi-Domain Adaptation for Regression under Conditional Distribution Shift
2023 (English)In: Expert systems with applications, ISSN 0957-4174, E-ISSN 1873-6793, Vol. 224, article id 119907Article in journal (Refereed) Published
Abstract [en]

Domain adaptation (DA) methods facilitate cross-domain learning by minimizing the marginal or conditional distribution shift between domains. However, the conditional distribution shift is not well addressed by existing DA techniques for the cross-domain regression learning task. In this paper, we propose Multi-Domain Adaptation for Regression under Conditional shift (DARC) method. DARC constructs a shared feature space such that linear regression on top of that space generalizes to all domains. In other words, DARC aligns different domains and makes explicit the task-related information encoded in the values of the dependent variable. It is achieved using a novel Pairwise Similarity Preserver (PSP) loss function. PSP incentivizes the differences between the outcomes of any two samples, regardless of their domain(s), to match the distance between these samples in the constructed space.

We perform experiments in both two-domain and multi-domain settings. The two-domain setting is helpful, especially when one domain contains few available labeled samples and can benefit from adaptation to a domain with many labeled samples. The multi-domain setting allows several domains, each with limited data, to be adapted collectively; thus, multiple domains compensate for each other’s lack of data. The results from all the experiments conducted both on synthetic and real-world datasets confirm the effectiveness of DARC. © 2023 The Authors

Place, publisher, year, edition, pages
Oxford: Elsevier, 2023
Keywords
Regression, Multi-Domain Adaptation, Conditional Shift, Concept Shift, Neural Networks, Siamese neural networks
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-47894 (URN)10.1016/j.eswa.2023.119907 (DOI)000966508000001 ()2-s2.0-85151474329 (Scopus ID)
Funder
VinnovaKnowledge Foundation
Available from: 2022-08-17 Created: 2022-08-17 Last updated: 2024-01-31Bibliographically approved
Nilsson, F., Bouguelia, M.-R. & Rögnvaldsson, T. (2023). Practical Joint Human-Machine Exploration of Industrial Time Series Using the Matrix Profile. Data mining and knowledge discovery, 37, 1-38
Open this publication in new window or tab >>Practical Joint Human-Machine Exploration of Industrial Time Series Using the Matrix Profile
2023 (English)In: Data mining and knowledge discovery, ISSN 1384-5810, E-ISSN 1573-756X, Vol. 37, p. 1-38Article in journal (Refereed) Published
Abstract [en]

Technological advancements and widespread adaptation of new technology in industry have made industrial time series data more available than ever before. With this development grows the need for versatile methods for mining industrial time series data. This paper introduces a practical approach for joint human-machine exploration of industrial time series data using the Matrix Profile (MP), and presents some challenges involved. The approach is demonstrated on three real-life industrial data sets to show how it enables the user to quickly extract semantic information, detect cycles, find deviating patterns, and gain a deeper understanding of the time series. A benchmark test is also presented on ECG (electrocardiogram) data, showing that the approach works well in comparison to previously suggested methods for extracting relevant time series motifs. © 2022, The Author(s).

Place, publisher, year, edition, pages
New York, NY: Springer, 2023
Keywords
Time Series, Matrix Profile, Motif Discovery, Industry 4.0
National Category
Computer Sciences
Research subject
Smart Cities and Communities
Identifiers
urn:nbn:se:hh:diva-48164 (URN)10.1007/s10618-022-00871-y (DOI)000864041100001 ()2-s2.0-85139448908 (Scopus ID)
Note

Funding: Open access funding provided by Halmstad University.

Available from: 2022-09-28 Created: 2022-09-28 Last updated: 2023-01-12Bibliographically approved
Farouq, S., Byttner, S., Bouguelia, M.-R. & Gadd, H. (2022). A conformal anomaly detection based industrial fleet monitoring framework: A case study in district heating. Expert systems with applications, 201, Article ID 116864.
Open this publication in new window or tab >>A conformal anomaly detection based industrial fleet monitoring framework: A case study in district heating
2022 (English)In: Expert systems with applications, ISSN 0957-4174, E-ISSN 1873-6793, Vol. 201, article id 116864Article in journal (Refereed) Published
Abstract [en]

The monitoring infrastructure of an industrial fleet can rely on the so-called unit-level and subfleet-level models to observe the behavior of a target unit. However, such infrastructure has to confront several challenges. First, from an anomaly detection perspective of monitoring a target unit, unit-level and subfleet-level models can give different information about the nature of an anomaly, and which approach or level model is appropriate is not always clear. Second, in the absence of well-understood prior models of unit and subfleet behavior, the choice of a base model at their respective levels, especially in an online/streaming setting, may not be clear. Third, managing false alarms is a major problem. To deal with these challenges, we proposed to rely on the conformal anomaly detection framework. In addition, an ensemble approach was deployed to mitigate the knowledge gap in understanding the underlying data-generating process at the unit and subfleet levels. Therefore, to monitor the behavior of a target unit, a unit-level ensemble model (ULEM) and a subfleet-level ensemble model (SLEM) were constructed, where each member of the respective ensemble is based on a conformal anomaly detector (CAD). However, since the information obtained by these two ensemble models through their p-values may not always agree, a combined ensemble model (CEM) was proposed. The results are based on real-world operational data obtained from district heating (DH) substations. Here, it was observed that CEM reduces the overall false alarms compared to ULEM or SLEM, albeit at the cost of some detection delay. The analysis demonstrated the advantages and limitations of ULEM, SLEM, and CEM. Furthermore, discords obtained from the state-of-the-art matrix-profile (MP) method and the combined calibration scores obtained from ULEM and SLEM were compared in an offline setting. Here, it was observed that SLEM achieved a better overall precision and detection delay. Finally, the different components related to ULEM, SLEM, and CEM were put together into what we refer to as TRANTOR: a conformal anomaly detection based industrial fleet monitoring framework. The proposed framework is expected to enable fleet operators in various domains to improve their monitoring infrastructure by efficiently detecting anomalous behavior and controlling false alarms at the target units. © 2022

Place, publisher, year, edition, pages
Oxford: Elsevier, 2022
Keywords
Conformal anomaly detection, Fleet monitoring, Unit-level model, Subfleet-level model, Ensemble model, District heating substations
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-46273 (URN)10.1016/j.eswa.2022.116864 (DOI)000798741200007 ()2-s2.0-85129522080 (Scopus ID)
Funder
Knowledge Foundation, 20160103
Note

Som manuskript i avhandling / As manuscript in thesis

Available from: 2022-02-01 Created: 2022-02-01 Last updated: 2022-09-01Bibliographically approved
Taghiyarrenani, Z., Nowaczyk, S., Pashami, S. & Bouguelia, M.-R. (2022). Adversarial Contrastive Semi-Supervised Domain Adaptation. IEEE Transactions on Knowledge and Data Engineering
Open this publication in new window or tab >>Adversarial Contrastive Semi-Supervised Domain Adaptation
2022 (English)In: IEEE Transactions on Knowledge and Data Engineering, ISSN 1041-4347, E-ISSN 1558-2191Article in journal (Refereed) Submitted
Abstract [en]

Domain Adaptation (DA) aims to transfer knowledge from a source to a target domain by aligning their respective data distributions. In the unsupervised setting, however, this may cause the source and target samples of different classes to align to each other, consequently leading to negative transfer. Semi-Supervised Domain Adaptation (SSDA) tries to solve such class misalignment problem by exploiting a few sample labels in the target domain. This paper proposes a new SSDA method called Adversarial Contrastive Semi-Supervised Domain Adaptation (ACSSDA) which combines two objectives, optimized for the case where very few target sample labels are available, to learn a shared feature representation for both source and target domains. ACSSDA uses a domain classifier to ensure that the resulting feature space is domain agnostic. Simultaneously, Contrastive loss aims to pull together samples of the same class and push apart samples of different classes. This is shown to reduce class misalignment and negative transfer even with as little as a single labeled sample per class. We demonstrate the effectiveness of ACSSDA with experiments on several benchmark data sets. The results show the superiority of our method over state-of-the-art approaches.

Place, publisher, year, edition, pages
Piscataway, NJ: IEEE, 2022
Keywords
Semi-Supervised Learning, Domain Adaptation, Contrastive Loss, Neural Networks
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-47895 (URN)
Available from: 2022-08-17 Created: 2022-08-17 Last updated: 2022-08-23
Taghiyarrenani, Z., Nowaczyk, S., Pashami, S. & Bouguelia, M.-R. (2022). Towards Geometry-Preserving Domain Adaptation for Fault Identification. In: Irena Koprinska, Paolo Mignone, Riccardo Guidotti, Szymon Jaroszewicz, Holger Fröning (Ed.), Machine Learning and Principles and Practice of Knowledge Discovery in Databases: International Workshops of ECML PKDD 2022, Grenoble, France, September 19-23, 2022, Proceedings, Part II. Paper presented at ECML/PKDD 2022 Workshop on IoT Streams for Predictive Maintenance, Grenoble, France, 19-23 September, 2022. Springer Nature
Open this publication in new window or tab >>Towards Geometry-Preserving Domain Adaptation for Fault Identification
2022 (English)In: Machine Learning and Principles and Practice of Knowledge Discovery in Databases: International Workshops of ECML PKDD 2022, Grenoble, France, September 19-23, 2022, Proceedings, Part II / [ed] Irena Koprinska, Paolo Mignone, Riccardo Guidotti, Szymon Jaroszewicz, Holger Fröning, Springer Nature, 2022Conference paper, Published paper (Refereed)
Abstract [en]

In most industries, the working conditions of equipment vary significantly from one site to another, from one time of a year to another, and so on. This variation poses a severe challenge for data-driven fault identification methods: it introduces a change in the data distribution. This contradicts the underlying assumption of most machine learning methods, namely that training and test samples follow the same distribution. Domain Adaptation (DA) methods aim to address this problem by minimizing the distribution distance between training (source) and test (target) samples.

However, in the area of predictive maintenance, this idea is complicated by the fact that different classes – fault categories – also vary across domains. Most of the state-of-the-art DA methods assume that the data in the target domain is complete, i.e., that we have access to examples from all the possible classes or faulty categories during adaptation. In reality, this is often very difficult to guarantee.

Therefore, there is a need for a domain adaptation method that is able to align the source and target domains even in cases of having access to an incomplete set of test data. This paper presents our work in progress as we propose an approach for such a setting based on maintaining the geometry information of source samples during the adaptation. This way, the model can capture the relationships between different fault categories and preserve them in the constructed domain-invariant feature space, even in situations where some classes are entirely missing. This paper examines this idea using artificial data sets to demonstrate the effectiveness of geometry-preserving transformation. We have also started investigations on real-world predictive maintenance datasets, such as CWRU.

Place, publisher, year, edition, pages
Springer Nature, 2022
Series
Communications in Computer and Information Science ; 1753
Keywords
Predictive Maintenance, Fault identification, Domain Adaptation, Geometry
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-47896 (URN)10.1007/978-3-031-23633-4_30 (DOI)000967761200030 ()2-s2.0-85149926744 (Scopus ID)978-3-031-23632-7 (ISBN)978-3-031-23633-4 (ISBN)
Conference
ECML/PKDD 2022 Workshop on IoT Streams for Predictive Maintenance, Grenoble, France, 19-23 September, 2022
Funder
Knowledge FoundationVinnova
Available from: 2022-08-17 Created: 2022-08-17 Last updated: 2024-01-31Bibliographically approved
Calikus, E., Nowaczyk, S., Bouguelia, M.-R. & Dikmen, O. (2022). Wisdom of the contexts: active ensemble learning for contextual anomaly detection. Data mining and knowledge discovery, 36, 2410-2458
Open this publication in new window or tab >>Wisdom of the contexts: active ensemble learning for contextual anomaly detection
2022 (English)In: Data mining and knowledge discovery, ISSN 1384-5810, E-ISSN 1573-756X, Vol. 36, p. 2410-2458Article in journal (Refereed) Published
Abstract [en]

In contextual anomaly detection, an object is only considered anomalous within a specific context. Most existing methods use a single context based on a set of user-specified contextual features. However, identifying the right context can be very challenging in practice, especially in datasets with a large number of attributes. Furthermore, in real-world systems, there might be multiple anomalies that occur in different contexts and, therefore, require a combination of several "useful" contexts to unveil them. In this work, we propose a novel approach, called WisCon (Wisdom of the Contexts), to effectively detect complex contextual anomalies in situations where the true contextual and behavioral attributes are unknown. Our method constructs an ensemble of multiple contexts, with varying importance scores, based on the assumption that not all useful contexts are equally so. We estimate the importance of each context using an active learning approach with a novel query strategy. Experiments show that WisCon significantly outperforms existing baselines in different categories (i.e., active classifiers, unsupervised contextual, and non-contextual anomaly detectors) on 18 datasets. Furthermore, the results support our initial hypothesis that there is no single perfect context that successfully uncovers all kinds of contextual anomalies, and leveraging the "wisdom" of multiple contexts is necessary. © 2022, The Author(s).

Place, publisher, year, edition, pages
New York: Springer-Verlag New York, 2022
Keywords
Anomaly detection, Active learning, Contextual anomaly detection, Ensemble learning, Active learning
National Category
Computer Sciences
Research subject
Smart Cities and Communities
Identifiers
urn:nbn:se:hh:diva-46401 (URN)10.1007/s10618-022-00868-7 (DOI)000864233400001 ()2-s2.0-85139454448 (Scopus ID)
Funder
Knowledge Foundation, 20160103
Note

Som manuskript i avhandling / As manuscript in thesis

Available from: 2022-02-22 Created: 2022-02-22 Last updated: 2023-01-12Bibliographically approved
Farouq, S., Byttner, S., Bouguelia, M.-R. & Gadd, H. (2021). Mondrian conformal anomaly detection for fault sequence identification in heterogeneous fleets. Neurocomputing, 462, 591-606
Open this publication in new window or tab >>Mondrian conformal anomaly detection for fault sequence identification in heterogeneous fleets
2021 (English)In: Neurocomputing, ISSN 0925-2312, E-ISSN 1872-8286, Vol. 462, p. 591-606Article in journal (Refereed) Published
Abstract [en]

We considered the case of monitoring a large fleet where heterogeneity in the operational behavior among its constituent units (i.e., systems or machines) is non-negligible, and no labeled data is available. Each unit in the fleet, referred to as a target, is tracked by its sub-fleet. A conformal sub-fleet (CSF) is a set of units that act as a proxy for the normal operational behavior of a target unit by relying on the Mondrian conformal anomaly detection framework. Two approaches, the k-nearest neighbors and conformal clustering, were investigated for constructing such a sub-fleet by formulating a stability criterion. Moreover, it is important to discover the sub-sequence of events that describes an anomalous behavior in a target unit. Hence, we proposed to extract such sub-sequences for further investigation without pre-specifying their length. We refer to it as a conformal anomaly sequence (CAS). Furthermore, different nonconformity measures were evaluated for their efficiency, i.e., their ability to detect anomalous behavior in a target unit, based on the length of the observed CAS and the S-criterion value. The CSF approach was evaluated in the context of monitoring district heating substations. Anomalous behavior sub-sequences were corroborated with the domain expert leading to the conclusion that the proposed approach has the potential to be useful for both diagnostic and knowledge extraction purposes, especially in domains where labeled data is not available or hard to obtain. © 2021

Place, publisher, year, edition, pages
Amsterdam: Elsevier, 2021
Keywords
Conformal anomaly detection, Conformal anomaly sequence (CAS), District heating, Sub-fleet based monitoring, Substation monitoring
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-45735 (URN)10.1016/j.neucom.2021.08.016 (DOI)000696933600013 ()2-s2.0-85115660223 (Scopus ID)
Available from: 2021-10-15 Created: 2021-10-15 Last updated: 2022-02-01Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-2859-6155

Search in DiVA

Show all publications