hh.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Adversarial Contrastive Semi-Supervised Domain Adaptation
Halmstad University, School of Information Technology.ORCID iD: 0000-0002-1759-8593
Halmstad University, School of Information Technology.ORCID iD: 0000-0002-7796-5201
Halmstad University, School of Information Technology.ORCID iD: 0000-0003-3272-4145
Halmstad University, School of Information Technology.ORCID iD: 0000-0002-2859-6155
2022 (English)In: IEEE Transactions on Knowledge and Data Engineering, ISSN 1041-4347, E-ISSN 1558-2191Article in journal (Refereed) Submitted
Abstract [en]

Domain Adaptation (DA) aims to transfer knowledge from a source to a target domain by aligning their respective data distributions. In the unsupervised setting, however, this may cause the source and target samples of different classes to align to each other, consequently leading to negative transfer. Semi-Supervised Domain Adaptation (SSDA) tries to solve such class misalignment problem by exploiting a few sample labels in the target domain. This paper proposes a new SSDA method called Adversarial Contrastive Semi-Supervised Domain Adaptation (ACSSDA) which combines two objectives, optimized for the case where very few target sample labels are available, to learn a shared feature representation for both source and target domains. ACSSDA uses a domain classifier to ensure that the resulting feature space is domain agnostic. Simultaneously, Contrastive loss aims to pull together samples of the same class and push apart samples of different classes. This is shown to reduce class misalignment and negative transfer even with as little as a single labeled sample per class. We demonstrate the effectiveness of ACSSDA with experiments on several benchmark data sets. The results show the superiority of our method over state-of-the-art approaches.

Place, publisher, year, edition, pages
Piscataway, NJ: IEEE, 2022.
Keywords [en]
Semi-Supervised Learning, Domain Adaptation, Contrastive Loss, Neural Networks
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:hh:diva-47895OAI: oai:DiVA.org:hh-47895DiVA, id: diva2:1687988
Available from: 2022-08-17 Created: 2022-08-17 Last updated: 2022-08-23
In thesis
1. Learning from Multiple Domains
Open this publication in new window or tab >>Learning from Multiple Domains
2022 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Domain adaptation (DA) transfers knowledge between domains by adapting them. The most well-known DA scenario in the literature is adapting two domains of source and target using the available labeled source samples to construct a model generalizable to the target domain. Although the primary purpose of DA is to compensate for the target domain’s labeled data shortage, the concept of adaptation can be utilized to solve other problems.

One issue that may occur during adaptation is the problem of class misalignment, which would result in a negative transfer. Therefore, preventing negative transfer should be considered while designing DA methods. In addition, the sample availability in domains is another matter that should also be taken into account.

Considering the two mentioned matters, this thesis aims to develop DA techniques to solve primary predictive maintenance problems.

This thesis considers a spectrum of cases with different amounts of available target data. One endpoint is the case in which we have access to enough labeled target samples for all classes. In this case, we use the concept of DA for 1) Analyzing two different physical properties, i.e., vibration and current, to measure their robustness for fault identification and 2) Developing a denoising method to construct a robust model for a noisy test environment.

Next, we consider the case where we have access to unlabeled and a few labeled target samples. Using the few labeled samples available, we aim to prevent negative transfer while adapting source and target domains. To achieve this, we construct a unified features representation using a few-shot and an adaptation learning technique.

In the subsequent considered setting, we assume we only have access to very few labeled target samples, which are insufficient to train a domain-specific model. Furthermore, for the first time in the literature, we solve the DA for regression in a setting in which it adapts multiple domains with any arbitrary shift.

Sometimes, due to the dynamic nature of the environment, we need to update a model to reflect the changes continuously. An example is in the field of computer network security. There is always the possibility of intrusion into a computer network, which makes each Intrusion Detection System (IDS) subject to concept shifts. In addition, different types of intrusions may occur in different networks. This thesis presents a framework for handling concept shift in one single network through incremental learning and simultaneously adapting samples from different networks to transfer knowledge about various intrusions. In addition, we employ active learning to use expert knowledge to label the samples for the adaptation purpose.

During adaptation, all cases mentioned so far have the same label space for the source and target domains. Occasionally, this is not the case, and we do not have access to samples for specific classes, either in the source or target; This is the final scenario addressed in this thesis.

One case is when we do not have access to some classes in the source domain. This setting is called Partial Domain Adaptation (PDA). This setting is beneficial to network traffic classification systems because, in general, every network has different types of applications and, therefore, different types of traffic. We develop a method for transferring knowledge from a source network to a target network even if the source network does not contain all types of traffic.

Another case is when we have access to unlabeled target samples but not for all classes. We call this Limited Domain Adaptation (LDA) setting and propose a DA method for fault identification. The motivation behind this setting is that for developing a fault identification model for a system, we don’t want to wait until the occurrence of all faults for collecting even unlabeled samples; instead, we aim to use the knowledge about those faults from other domains.

We provide results on synthetic and real-world datasets for the scenarios mentioned above. Results indicate that the proposed methods outperform the state-of-art and are effective and practical in solving real-world problems.

For future works, we plan to extend the proposed methods to adapt domains with different input features, especially for solving predictive maintenance problems. Furthermore, we intend to extend our work to out-of-distribution learning methods, such as domain generalization.

Place, publisher, year, edition, pages
Halmstad: Halmstad University Press, 2022. p. 26
Series
Halmstad University Dissertations ; 92
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-47890 (URN)978-91-88749-96-3 (ISBN)978-91-88749-95-6 (ISBN)
Presentation
2022-09-14, Wigforssalen, Hus J (Visionen), Kristian IV:s väg 3, Halmstad, 13:00 (English)
Opponent
Supervisors
Funder
Vinnova
Available from: 2022-08-18 Created: 2022-08-17 Last updated: 2022-08-18Bibliographically approved

Open Access in DiVA

No full text in DiVA

Authority records

Taghiyarrenani, ZahraNowaczyk, SławomirPashami, SepidehBouguelia, Mohamed-Rafik

Search in DiVA

By author/editor
Taghiyarrenani, ZahraNowaczyk, SławomirPashami, SepidehBouguelia, Mohamed-Rafik
By organisation
School of Information Technology
In the same journal
IEEE Transactions on Knowledge and Data Engineering
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 724 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf