hh.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Multi-Domain Adaptation for Regression under Conditional Distribution Shift
Halmstad University, School of Information Technology.ORCID iD: 0000-0002-1759-8593
Halmstad University, School of Information Technology.ORCID iD: 0000-0002-7796-5201
Halmstad University, School of Information Technology.ORCID iD: 0000-0003-3272-4145
Halmstad University, School of Information Technology.ORCID iD: 0000-0002-2859-6155
2023 (English)In: Expert systems with applications, ISSN 0957-4174, E-ISSN 1873-6793, Vol. 224, article id 119907Article in journal (Refereed) Published
Abstract [en]

Domain adaptation (DA) methods facilitate cross-domain learning by minimizing the marginal or conditional distribution shift between domains. However, the conditional distribution shift is not well addressed by existing DA techniques for the cross-domain regression learning task. In this paper, we propose Multi-Domain Adaptation for Regression under Conditional shift (DARC) method. DARC constructs a shared feature space such that linear regression on top of that space generalizes to all domains. In other words, DARC aligns different domains and makes explicit the task-related information encoded in the values of the dependent variable. It is achieved using a novel Pairwise Similarity Preserver (PSP) loss function. PSP incentivizes the differences between the outcomes of any two samples, regardless of their domain(s), to match the distance between these samples in the constructed space.

We perform experiments in both two-domain and multi-domain settings. The two-domain setting is helpful, especially when one domain contains few available labeled samples and can benefit from adaptation to a domain with many labeled samples. The multi-domain setting allows several domains, each with limited data, to be adapted collectively; thus, multiple domains compensate for each other’s lack of data. The results from all the experiments conducted both on synthetic and real-world datasets confirm the effectiveness of DARC. © 2023 The Authors

Place, publisher, year, edition, pages
Oxford: Elsevier, 2023. Vol. 224, article id 119907
Keywords [en]
Regression, Multi-Domain Adaptation, Conditional Shift, Concept Shift, Neural Networks, Siamese neural networks
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:hh:diva-47894DOI: 10.1016/j.eswa.2023.119907ISI: 000966508000001Scopus ID: 2-s2.0-85151474329OAI: oai:DiVA.org:hh-47894DiVA, id: diva2:1687984
Funder
VinnovaKnowledge FoundationAvailable from: 2022-08-17 Created: 2022-08-17 Last updated: 2023-06-21Bibliographically approved
In thesis
1. Learning from Multiple Domains
Open this publication in new window or tab >>Learning from Multiple Domains
2022 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Domain adaptation (DA) transfers knowledge between domains by adapting them. The most well-known DA scenario in the literature is adapting two domains of source and target using the available labeled source samples to construct a model generalizable to the target domain. Although the primary purpose of DA is to compensate for the target domain’s labeled data shortage, the concept of adaptation can be utilized to solve other problems.

One issue that may occur during adaptation is the problem of class misalignment, which would result in a negative transfer. Therefore, preventing negative transfer should be considered while designing DA methods. In addition, the sample availability in domains is another matter that should also be taken into account.

Considering the two mentioned matters, this thesis aims to develop DA techniques to solve primary predictive maintenance problems.

This thesis considers a spectrum of cases with different amounts of available target data. One endpoint is the case in which we have access to enough labeled target samples for all classes. In this case, we use the concept of DA for 1) Analyzing two different physical properties, i.e., vibration and current, to measure their robustness for fault identification and 2) Developing a denoising method to construct a robust model for a noisy test environment.

Next, we consider the case where we have access to unlabeled and a few labeled target samples. Using the few labeled samples available, we aim to prevent negative transfer while adapting source and target domains. To achieve this, we construct a unified features representation using a few-shot and an adaptation learning technique.

In the subsequent considered setting, we assume we only have access to very few labeled target samples, which are insufficient to train a domain-specific model. Furthermore, for the first time in the literature, we solve the DA for regression in a setting in which it adapts multiple domains with any arbitrary shift.

Sometimes, due to the dynamic nature of the environment, we need to update a model to reflect the changes continuously. An example is in the field of computer network security. There is always the possibility of intrusion into a computer network, which makes each Intrusion Detection System (IDS) subject to concept shifts. In addition, different types of intrusions may occur in different networks. This thesis presents a framework for handling concept shift in one single network through incremental learning and simultaneously adapting samples from different networks to transfer knowledge about various intrusions. In addition, we employ active learning to use expert knowledge to label the samples for the adaptation purpose.

During adaptation, all cases mentioned so far have the same label space for the source and target domains. Occasionally, this is not the case, and we do not have access to samples for specific classes, either in the source or target; This is the final scenario addressed in this thesis.

One case is when we do not have access to some classes in the source domain. This setting is called Partial Domain Adaptation (PDA). This setting is beneficial to network traffic classification systems because, in general, every network has different types of applications and, therefore, different types of traffic. We develop a method for transferring knowledge from a source network to a target network even if the source network does not contain all types of traffic.

Another case is when we have access to unlabeled target samples but not for all classes. We call this Limited Domain Adaptation (LDA) setting and propose a DA method for fault identification. The motivation behind this setting is that for developing a fault identification model for a system, we don’t want to wait until the occurrence of all faults for collecting even unlabeled samples; instead, we aim to use the knowledge about those faults from other domains.

We provide results on synthetic and real-world datasets for the scenarios mentioned above. Results indicate that the proposed methods outperform the state-of-art and are effective and practical in solving real-world problems.

For future works, we plan to extend the proposed methods to adapt domains with different input features, especially for solving predictive maintenance problems. Furthermore, we intend to extend our work to out-of-distribution learning methods, such as domain generalization.

Place, publisher, year, edition, pages
Halmstad: Halmstad University Press, 2022. p. 26
Series
Halmstad University Dissertations ; 92
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-47890 (URN)978-91-88749-96-3 (ISBN)978-91-88749-95-6 (ISBN)
Presentation
2022-09-14, Wigforssalen, Hus J (Visionen), Kristian IV:s väg 3, Halmstad, 13:00 (English)
Opponent
Supervisors
Funder
Vinnova
Available from: 2022-08-18 Created: 2022-08-17 Last updated: 2022-08-18Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Taghiyarrenani, ZahraNowaczyk, SławomirPashami, SepidehBouguelia, Mohamed-Rafik

Search in DiVA

By author/editor
Taghiyarrenani, ZahraNowaczyk, SławomirPashami, SepidehBouguelia, Mohamed-Rafik
By organisation
School of Information Technology
In the same journal
Expert systems with applications
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 514 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf