hh.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Learning from Multiple Domains
Halmstad University, School of Information Technology.ORCID iD: 0000-0002-1759-8593
2022 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Domain adaptation (DA) transfers knowledge between domains by adapting them. The most well-known DA scenario in the literature is adapting two domains of source and target using the available labeled source samples to construct a model generalizable to the target domain. Although the primary purpose of DA is to compensate for the target domain’s labeled data shortage, the concept of adaptation can be utilized to solve other problems.

One issue that may occur during adaptation is the problem of class misalignment, which would result in a negative transfer. Therefore, preventing negative transfer should be considered while designing DA methods. In addition, the sample availability in domains is another matter that should also be taken into account.

Considering the two mentioned matters, this thesis aims to develop DA techniques to solve primary predictive maintenance problems.

This thesis considers a spectrum of cases with different amounts of available target data. One endpoint is the case in which we have access to enough labeled target samples for all classes. In this case, we use the concept of DA for 1) Analyzing two different physical properties, i.e., vibration and current, to measure their robustness for fault identification and 2) Developing a denoising method to construct a robust model for a noisy test environment.

Next, we consider the case where we have access to unlabeled and a few labeled target samples. Using the few labeled samples available, we aim to prevent negative transfer while adapting source and target domains. To achieve this, we construct a unified features representation using a few-shot and an adaptation learning technique.

In the subsequent considered setting, we assume we only have access to very few labeled target samples, which are insufficient to train a domain-specific model. Furthermore, for the first time in the literature, we solve the DA for regression in a setting in which it adapts multiple domains with any arbitrary shift.

Sometimes, due to the dynamic nature of the environment, we need to update a model to reflect the changes continuously. An example is in the field of computer network security. There is always the possibility of intrusion into a computer network, which makes each Intrusion Detection System (IDS) subject to concept shifts. In addition, different types of intrusions may occur in different networks. This thesis presents a framework for handling concept shift in one single network through incremental learning and simultaneously adapting samples from different networks to transfer knowledge about various intrusions. In addition, we employ active learning to use expert knowledge to label the samples for the adaptation purpose.

During adaptation, all cases mentioned so far have the same label space for the source and target domains. Occasionally, this is not the case, and we do not have access to samples for specific classes, either in the source or target; This is the final scenario addressed in this thesis.

One case is when we do not have access to some classes in the source domain. This setting is called Partial Domain Adaptation (PDA). This setting is beneficial to network traffic classification systems because, in general, every network has different types of applications and, therefore, different types of traffic. We develop a method for transferring knowledge from a source network to a target network even if the source network does not contain all types of traffic.

Another case is when we have access to unlabeled target samples but not for all classes. We call this Limited Domain Adaptation (LDA) setting and propose a DA method for fault identification. The motivation behind this setting is that for developing a fault identification model for a system, we don’t want to wait until the occurrence of all faults for collecting even unlabeled samples; instead, we aim to use the knowledge about those faults from other domains.

We provide results on synthetic and real-world datasets for the scenarios mentioned above. Results indicate that the proposed methods outperform the state-of-art and are effective and practical in solving real-world problems.

For future works, we plan to extend the proposed methods to adapt domains with different input features, especially for solving predictive maintenance problems. Furthermore, we intend to extend our work to out-of-distribution learning methods, such as domain generalization.

Place, publisher, year, edition, pages
Halmstad: Halmstad University Press, 2022. , p. 26
Series
Halmstad University Dissertations ; 92
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:hh:diva-47890ISBN: 978-91-88749-96-3 (print)ISBN: 978-91-88749-95-6 (electronic)OAI: oai:DiVA.org:hh-47890DiVA, id: diva2:1687945
Presentation
2022-09-14, Wigforssalen, Hus J (Visionen), Kristian IV:s väg 3, Halmstad, 13:00 (English)
Opponent
Supervisors
Funder
VinnovaAvailable from: 2022-08-18 Created: 2022-08-17 Last updated: 2022-08-18Bibliographically approved
List of papers
1. An Analysis of Vibrations and Currents for Broken Rotor Bar Detection in Three-phase Induction Motors
Open this publication in new window or tab >>An Analysis of Vibrations and Currents for Broken Rotor Bar Detection in Three-phase Induction Motors
2022 (English)In: Proceedings of the European Conference of the Prognostics and Health Management Society 2022 / [ed] Phuc Do; Gabriel Michau; Cordelia Ezhilarasu, State College, PA: PHM Society , 2022, Vol. 7 (1), p. 43-48Conference paper, Published paper (Refereed)
Abstract [en]

Selecting the physical property capable of representing the health state of a machine is an important step in designing fault detection systems. In addition, variation of the loading condition is a challenge in deploying an industrial predictive maintenance solution. The robustness of the physical properties to variations in loading conditions is, therefore, an important consideration. In this paper, we focus specifically on squirrel cage induction motors and analyze the capabilities of three-phase current and five vibration signals acquired from different locations of the motor for the detection of Broken Rotor Bar generated in different loads. In particular, we examine the mentioned signals in relation to the performance of classifiers trained with them. Regarding the classifiers, we employ deep conventional classifiers and also propose a hybrid classifier that utilizes contrastive loss in order to mitigate the effect of different variations. The analysis shows that vibration signals are more robust under varying load conditions. Furthermore, the proposed hybrid classifier outperforms conventional classifiers and is able to achieve an accuracy of 90.96% when using current signals and 97.69% when using vibration signals.

Place, publisher, year, edition, pages
State College, PA: PHM Society, 2022
Series
Proceedings of the European Conference of the Prognostics and Health Management Society (PHME), E-ISSN 2325-016X
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-47892 (URN)10.36001/phme.2022.v7i1.3332 (DOI)978-1-936263-36-3 (ISBN)
Conference
The 7th European Conference of the Prognostics and Health Management Society, Turin, Italy, July 6-8, 2022
Funder
Vinnova
Available from: 2022-08-17 Created: 2022-08-17 Last updated: 2023-03-21Bibliographically approved
2. Noise-robust representation for fault identification with limited data via data augmentation
Open this publication in new window or tab >>Noise-robust representation for fault identification with limited data via data augmentation
2022 (English)In: Proceedings of the European Conference of the Prognostics and Health Management Society 2022 / [ed] Phuc Do; Gabriel Michau; Cordelia Ezhilarasu, State College, PA: Prognostics and Health Management Society , 2022, Vol. 7 (1), p. 473-479Conference paper, Published paper (Refereed)
Abstract [en]

Noise will be unavoidably present in the data collected from physical environments, regardless of how sophisticated the measurement equipment is. Furthermore, collecting enough faulty data is a challenge since operating industrial machines in faulty modes not only has severe consequences to the machine health, but also may affect collateral machinery critically, from health state point of view. In this paper, we propose a method of denoising with limited data for the purpose of fault identification. In addition, our method is capable of removing multiple levels of noise simultaneously. For this purpose, inspired by unsupervised contrastive learning, we first augment the data with multiple levels of noise. Later, we construct a new feature representation using Contrastive Loss. The last step is building a classifier on top of the learned representation; this classifier can detect various faults in noisy environments. The experiments on the SOUTHEAST UNIVERSITY (SEU) dataset of bearings confirm that our method can simultaneously remove multiple noise levels.

Place, publisher, year, edition, pages
State College, PA: Prognostics and Health Management Society, 2022
Series
Proceedings of the European Conference of the Prognostics and Health Management Society (PHME), E-ISSN 2325-016X
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-47893 (URN)10.36001/phme.2022.v7i1.3334 (DOI)978-1-936263-36-3 (ISBN)
Conference
The 7th European Conference of the Prognostics and Health Management Society, Turin, Italy, July 6-8, 2022
Funder
Vinnova
Available from: 2022-08-17 Created: 2022-08-17 Last updated: 2023-03-21Bibliographically approved
3. Adversarial Contrastive Semi-Supervised Domain Adaptation
Open this publication in new window or tab >>Adversarial Contrastive Semi-Supervised Domain Adaptation
2022 (English)In: IEEE Transactions on Knowledge and Data Engineering, ISSN 1041-4347, E-ISSN 1558-2191Article in journal (Refereed) Submitted
Abstract [en]

Domain Adaptation (DA) aims to transfer knowledge from a source to a target domain by aligning their respective data distributions. In the unsupervised setting, however, this may cause the source and target samples of different classes to align to each other, consequently leading to negative transfer. Semi-Supervised Domain Adaptation (SSDA) tries to solve such class misalignment problem by exploiting a few sample labels in the target domain. This paper proposes a new SSDA method called Adversarial Contrastive Semi-Supervised Domain Adaptation (ACSSDA) which combines two objectives, optimized for the case where very few target sample labels are available, to learn a shared feature representation for both source and target domains. ACSSDA uses a domain classifier to ensure that the resulting feature space is domain agnostic. Simultaneously, Contrastive loss aims to pull together samples of the same class and push apart samples of different classes. This is shown to reduce class misalignment and negative transfer even with as little as a single labeled sample per class. We demonstrate the effectiveness of ACSSDA with experiments on several benchmark data sets. The results show the superiority of our method over state-of-the-art approaches.

Place, publisher, year, edition, pages
Piscataway, NJ: IEEE, 2022
Keywords
Semi-Supervised Learning, Domain Adaptation, Contrastive Loss, Neural Networks
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-47895 (URN)
Available from: 2022-08-17 Created: 2022-08-17 Last updated: 2022-08-23
4. Multi-Domain Adaptation for Regression under Conditional Distribution Shift
Open this publication in new window or tab >>Multi-Domain Adaptation for Regression under Conditional Distribution Shift
2023 (English)In: Expert systems with applications, ISSN 0957-4174, E-ISSN 1873-6793, Vol. 224, article id 119907Article in journal (Refereed) Published
Abstract [en]

Domain adaptation (DA) methods facilitate cross-domain learning by minimizing the marginal or conditional distribution shift between domains. However, the conditional distribution shift is not well addressed by existing DA techniques for the cross-domain regression learning task. In this paper, we propose Multi-Domain Adaptation for Regression under Conditional shift (DARC) method. DARC constructs a shared feature space such that linear regression on top of that space generalizes to all domains. In other words, DARC aligns different domains and makes explicit the task-related information encoded in the values of the dependent variable. It is achieved using a novel Pairwise Similarity Preserver (PSP) loss function. PSP incentivizes the differences between the outcomes of any two samples, regardless of their domain(s), to match the distance between these samples in the constructed space.

We perform experiments in both two-domain and multi-domain settings. The two-domain setting is helpful, especially when one domain contains few available labeled samples and can benefit from adaptation to a domain with many labeled samples. The multi-domain setting allows several domains, each with limited data, to be adapted collectively; thus, multiple domains compensate for each other’s lack of data. The results from all the experiments conducted both on synthetic and real-world datasets confirm the effectiveness of DARC. © 2023 The Authors

Place, publisher, year, edition, pages
Oxford: Elsevier, 2023
Keywords
Regression, Multi-Domain Adaptation, Conditional Shift, Concept Shift, Neural Networks, Siamese neural networks
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-47894 (URN)10.1016/j.eswa.2023.119907 (DOI)000966508000001 ()2-s2.0-85151474329 (Scopus ID)
Funder
VinnovaKnowledge Foundation
Available from: 2022-08-17 Created: 2022-08-17 Last updated: 2023-06-21Bibliographically approved
5. ITL-IDS: Incremental Transfer Learning for Intrusion Detection Systems
Open this publication in new window or tab >>ITL-IDS: Incremental Transfer Learning for Intrusion Detection Systems
2022 (English)In: Knowledge-Based Systems, ISSN 0950-7051, E-ISSN 1872-7409, Vol. 253, article id 109542Article in journal (Refereed) Published
Abstract [en]

Utilizing machine learning methods to detect intrusion into computer networks is a trending topic in information security research. The limitation of labeled samples is one of the challenges in this area. This challenge makes it difficult to build accurate learning models for intrusion detection. Transfer learning is one of the methods to counter such a challenge in machine learning topics. On the other hand, the emergence of new technologies and applications might bring new vulnerabilities to computer networks. Therefore, the learning process cannot occur all at once. Incremental learning is a practical standpoint to confront this challenge. This research presents a new framework for intrusion detection systems called ITL-IDS that can potentially start learning in a network without prior knowledge. It begins with an incremental clustering algorithm to detect clusters’ numbers and shape without prior assumptions about the attacks. The outcomes are candidates to transfer knowledge between other instances of ITL-IDS. In each iteration, transfer learning provides target environments with incremental knowledge. Our evaluation shows that this method can combine incremental and transfer learning to identify new attacks. © 2022 Published by Elsevier B.V.

Place, publisher, year, edition, pages
Amsterdam: Elsevier, 2022
Keywords
Network security, Intrusion detection system, NIDS, Transfer learning, Incremental learning
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-47891 (URN)10.1016/j.knosys.2022.109542 (DOI)000861208200008 ()2-s2.0-85135717752 (Scopus ID)
Available from: 2022-08-17 Created: 2022-08-17 Last updated: 2022-10-24Bibliographically approved
6. Domain Adaptation with Maximum Margin Criterion with application to network traffic classification
Open this publication in new window or tab >>Domain Adaptation with Maximum Margin Criterion with application to network traffic classification
2023 (English)In: Machine Learning and Principles and Practice of Knowledge Discovery in Databases: International Workshops of ECML PKDD 2022, Grenoble, France, September 19-23, 2022, Proceedings, Part II / [ed] Irena Koprinska et al., Cham: Springer, 2023, p. 159-169Conference paper, Published paper (Refereed)
Abstract [en]

A fundamental assumption in machine learning is that training and test samples follow the same distribution. Therefore, for training a machine learning-based network traffic classifier, it is necessary to use samples obtained from the desired network. Collecting enough training data, however, can be challenging in many cases. Domain adaptation allows samples from other networks to be utilized. In order to satisfy the aforementioned assumption, domain adaptation reduces the distance between the distribution of the samples in the desired network and that of the available samples in other networks. However, it is important to note that the applications in two different networks can differ considerably. Taking this into account, in this paper, we present a new domain adaptation method for classifying network traffic. Thus, we use the labeled samples from a network and adapt them to the few labeled samples from the desired network; In other words, we adapt shared applications while preserving the information about non-shared applications. In order to demonstrate the efficacy of our method, we construct five different cross-network datasets using the Brazil dataset. These results indicate the effectiveness of adapting samples between different domains using the proposed method.

Place, publisher, year, edition, pages
Cham: Springer, 2023
Series
Communications in Computer and Information Science, ISSN 1865-0929, E-ISSN 1865-0937 ; 1753
Keywords
Traffic classification, Domain Adaptation, Transfer Learning
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-47897 (URN)10.1007/978-3-031-23633-4_12 (DOI)000967761200012 ()2-s2.0-85149944135 (Scopus ID)978-3-031-23632-7 (ISBN)978-3-031-23633-4 (ISBN)
Conference
ECML/PKDD 2022 Workshop on Machine Learning for Cyber Security, Grenoble, France, 19-23 September, 2022
Available from: 2022-08-17 Created: 2022-08-17 Last updated: 2023-08-11Bibliographically approved
7. Towards Geometry-Preserving Domain Adaptation for Fault Identification
Open this publication in new window or tab >>Towards Geometry-Preserving Domain Adaptation for Fault Identification
2022 (English)In: Machine Learning and Principles and Practice of Knowledge Discovery in Databases: International Workshops of ECML PKDD 2022, Grenoble, France, September 19-23, 2022, Proceedings, Part II / [ed] Irena Koprinska, Paolo Mignone, Riccardo Guidotti, Szymon Jaroszewicz, Holger Fröning, Springer Nature, 2022Conference paper, Published paper (Refereed)
Abstract [en]

In most industries, the working conditions of equipment vary significantly from one site to another, from one time of a year to another, and so on. This variation poses a severe challenge for data-driven fault identification methods: it introduces a change in the data distribution. This contradicts the underlying assumption of most machine learning methods, namely that training and test samples follow the same distribution. Domain Adaptation (DA) methods aim to address this problem by minimizing the distribution distance between training (source) and test (target) samples.

However, in the area of predictive maintenance, this idea is complicated by the fact that different classes – fault categories – also vary across domains. Most of the state-of-the-art DA methods assume that the data in the target domain is complete, i.e., that we have access to examples from all the possible classes or faulty categories during adaptation. In reality, this is often very difficult to guarantee.

Therefore, there is a need for a domain adaptation method that is able to align the source and target domains even in cases of having access to an incomplete set of test data. This paper presents our work in progress as we propose an approach for such a setting based on maintaining the geometry information of source samples during the adaptation. This way, the model can capture the relationships between different fault categories and preserve them in the constructed domain-invariant feature space, even in situations where some classes are entirely missing. This paper examines this idea using artificial data sets to demonstrate the effectiveness of geometry-preserving transformation. We have also started investigations on real-world predictive maintenance datasets, such as CWRU.

Place, publisher, year, edition, pages
Springer Nature, 2022
Series
Communications in Computer and Information Science ; 1753
Keywords
Predictive Maintenance, Fault identification, Domain Adaptation, Geometry
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-47896 (URN)10.1007/978-3-031-23633-4_30 (DOI)000967761200030 ()2-s2.0-85149926744 (Scopus ID)978-3-031-23632-7 (ISBN)978-3-031-23633-4 (ISBN)
Conference
ECML/PKDD 2022 Workshop on IoT Streams for Predictive Maintenance, Grenoble, France, 19-23 September, 2022
Funder
Knowledge FoundationVinnova
Available from: 2022-08-17 Created: 2022-08-17 Last updated: 2023-08-11Bibliographically approved

Open Access in DiVA

fulltext(2709 kB)238 downloads
File information
File name FULLTEXT01.pdfFile size 2709 kBChecksum SHA-512
b0e9339692570f6be904b2ccaab9b747a4be310e1662d9a0e3bc121921854a6e4815bd5fe95d31f1323021b75cc0181876a0393e1a8e78fce212d940759c1e4c
Type fulltextMimetype application/pdf

Authority records

Taghiyarrenani, Zahra

Search in DiVA

By author/editor
Taghiyarrenani, Zahra
By organisation
School of Information Technology
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 238 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 622 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf