hh.sePublications
Change search
Link to record
Permanent link

Direct link
Rögnvaldsson, ThorsteinnORCID iD iconorcid.org/0000-0001-5163-2997
Publications (10 of 88) Show all publications
Vettoruzzo, A., Bouguelia, M.-R., Vanschoren, J., Rögnvaldsson, T. & Santosh, K. (2024). Advances and Challenges in Meta-Learning: A Technical Review. IEEE Transactions on Pattern Analysis and Machine Intelligence
Open this publication in new window or tab >>Advances and Challenges in Meta-Learning: A Technical Review
Show others...
2024 (English)In: IEEE Transactions on Pattern Analysis and Machine Intelligence, ISSN 0162-8828, E-ISSN 1939-3539Article, review/survey (Refereed) Epub ahead of print
Abstract [en]

Meta-learning empowers learning systems with the ability to acquire knowledge from multiple tasks, enabling faster adaptation and generalization to new tasks. This review provides a comprehensive technical overview of meta-learning, emphasizing its importance in real-world applications where data may be scarce or expensive to obtain. The paper covers the state-of-the-art meta-learning approaches and explores the relationship between meta-learning and multi-task learning, transfer learning, domain adaptation and generalization, selfsupervised learning, personalized federated learning, and continual learning. By highlighting the synergies between these topics and the field of meta-learning, the paper demonstrates how advancements in one area can benefit the field as a whole, while avoiding unnecessary duplication of efforts. Additionally, the paper delves into advanced meta-learning topics such as learning from complex multi-modal task distributions, unsupervised metalearning, learning to efficiently adapt to data distribution shifts, and continual meta-learning. Lastly, the paper highlights open problems and challenges for future research in the field. By synthesizing the latest research developments, this paper provides a thorough understanding of meta-learning and its potential impact on various machine learning applications. We believe that this technical overview will contribute to the advancement of meta-learning and its practical implications in addressing realworld problems.

Place, publisher, year, edition, pages
Piscataway, NJ: IEEE, 2024
Keywords
Adaptation models, Data models, deep neural networks, few-shot learning, Meta-learning, Metalearning, representation learning, Surveys, Task analysis, Training, transfer learning, Transfer learning
National Category
Robotics
Identifiers
urn:nbn:se:hh:diva-52730 (URN)10.1109/TPAMI.2024.3357847 (DOI)2-s2.0-85183973598 (Scopus ID)
Funder
Knowledge Foundation
Available from: 2024-02-23 Created: 2024-02-23 Last updated: 2024-02-23Bibliographically approved
Altarabichi, M. G., Alabdallah, A., Pashami, S., Ohlsson, M., Rögnvaldsson, T. & Nowaczyk, S. (2024). Improving Concordance Index in Regression-based Survival Analysis: Discovery of Loss Function for Neural Networks.
Open this publication in new window or tab >>Improving Concordance Index in Regression-based Survival Analysis: Discovery of Loss Function for Neural Networks
Show others...
2024 (English)Manuscript (preprint) (Other academic)
Abstract [en]

In this work, we use an Evolutionary Algorithm (EA) to discover a novel Neural Network (NN) regression-based survival loss function with the aim of improving the C-index performance. Our contribution is threefold; firstly, we propose an evolutionary meta-learning algorithm SAGA$_{loss}$ for optimizing a neural-network regression-based loss function that maximizes the C-index; our algorithm consistently discovers specialized loss functions that outperform MSCE. Secondly, based on our analysis of the evolutionary search results, we highlight a non-intuitive insight that signifies the importance of the non-zero gradient for the censored cases part of the loss function, a property that is shown to be useful in improving concordance. Finally, based on this insight, we propose MSCE$_{Sp}$, a novel survival regression loss function that can be used off-the-shelf and generally performs better than the Mean Squared Error for censored cases. We performed extensive experiments on 19 benchmark datasets to validate our findings.

Keywords
evolutionary meta-learning, loss function, neural networks, survival analysis, regression
National Category
Computer Systems
Identifiers
urn:nbn:se:hh:diva-52468 (URN)
Note

As manuscript in thesis.

Available from: 2024-01-24 Created: 2024-01-24 Last updated: 2024-02-05Bibliographically approved
Vettoruzzo, A., Bouguelia, M.-R. & Rögnvaldsson, T. (2024). Meta-learning for efficient unsupervised domain adaptation. Neurocomputing, 574, Article ID 127264.
Open this publication in new window or tab >>Meta-learning for efficient unsupervised domain adaptation
2024 (English)In: Neurocomputing, ISSN 0925-2312, E-ISSN 1872-8286, Vol. 574, article id 127264Article in journal (Refereed) In press
Abstract [en]

The standard machine learning assumption that training and test data are drawn from the same probability distribution does not hold in many real-world applications due to the inability to reproduce testing conditions at training time. Existing unsupervised domain adaption (UDA) methods address this problem by learning a domain-invariant feature space that performs well on available source domain(s) (labeled training data) and the specific target domain (unlabeled test data). In contrast, instead of simply adapting to domains, this paper aims for an approach that learns to adapt effectively to new unlabeled domains. To do so, we leverage meta-learning to optimize a neural network such that an unlabeled adaptation of its parameters to any domain would yield a good generalization on this latter. The experimental evaluation shows that the proposed approach outperforms standard approaches even when a small amount of unlabeled test data is used for adaptation, demonstrating the benefit of meta-learning prior knowledge from various domains to solve UDA problems.

Place, publisher, year, edition, pages
Amsterdam: Elsevier, 2024
Keywords
Domain adaptation, Meta-learning, Unsupervised learning, Distribution shift
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-52450 (URN)10.1016/j.neucom.2024.127264 (DOI)
Funder
Knowledge Foundation
Available from: 2024-01-22 Created: 2024-01-22 Last updated: 2024-01-22Bibliographically approved
Khoshkangini, R., Tajgardan, M., Sheikholharam Mashhadi, P., Rögnvaldsson, T. & Tegnered, D. (2024). Optimal Task Grouping Approach in Multitask Learning. In: Biao Luo; Long Cheng; Zheng-Guang Wu, Hongyi Li; Chaojie Li (Ed.), Neural Information Processing. ICONIP 2023: . Paper presented at 30th International Conference on Neural Information Processing, ICONIP 2023, Changsha, China, November 20–23, 2023 (pp. 206-225). Heidelberg: Springer Nature
Open this publication in new window or tab >>Optimal Task Grouping Approach in Multitask Learning
Show others...
2024 (English)In: Neural Information Processing. ICONIP 2023 / [ed] Biao Luo; Long Cheng; Zheng-Guang Wu, Hongyi Li; Chaojie Li, Heidelberg: Springer Nature, 2024, p. 206-225Conference paper, Published paper (Refereed)
Abstract [en]

Multi-task learning has become a powerful solution in which multiple tasks are trained together to leverage the knowledge learned from one task to improve the performance of the other tasks. However, the tasks are not always constructive on each other in the multi-task formulation and might play negatively during the training process leading to poor results. Thus, this study focuses on finding the optimal group of tasks that should be trained together for multi-task learning in an automotive context. We proposed a multi-task learning approach to model multiple vehicle long-term behaviors using low-resolution data and utilized gradient descent to efficiently discover the optimal group of tasks/vehicle behaviors that can increase the performance of the predictive models in a single training process. In this study, we also quantified the contribution of individual tasks in their groups and to the other groups’ performance. The experimental evaluation of the data collected from thousands of heavy-duty trucks shows that the proposed approach is promising. © 2024 Springer Nature

Place, publisher, year, edition, pages
Heidelberg: Springer Nature, 2024
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 14452
Keywords
Machine Learning, Vehicle Usage Behavior, Multitask Learning
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:hh:diva-52349 (URN)10.1007/978-981-99-8076-5_15 (DOI)978-981-99-8075-8 (ISBN)978-981-99-8076-5 (ISBN)
Conference
30th International Conference on Neural Information Processing, ICONIP 2023, Changsha, China, November 20–23, 2023
Funder
Knowledge Foundation
Available from: 2024-01-03 Created: 2024-01-03 Last updated: 2024-01-04Bibliographically approved
Budu, E., Soliman, A., Etminani, K. & Rögnvaldsson, T. (2023). A Framework for Evaluating Synthetic Electronic Health Records. In: Hägglund, Maria et al. (Ed.), Caring is Sharing – Exploiting the Value in Data for Health and Innovation: . Paper presented at 33rd Medical Informatics Europe Conference: Caring is Sharing - Exploiting the Value in Data for Health and Innovation (MIE2023), Gothenburg, Sweden, 22-25 May, 2023 (pp. 378-379). Amsterdam: IOS Press, 302
Open this publication in new window or tab >>A Framework for Evaluating Synthetic Electronic Health Records
2023 (English)In: Caring is Sharing – Exploiting the Value in Data for Health and Innovation / [ed] Hägglund, Maria et al., Amsterdam: IOS Press, 2023, Vol. 302, p. 378-379Conference paper, Published paper (Refereed)
Abstract [en]

Synthetic data generation can be applied to Electronic Health Records (EHRs) to obtain synthetic versions that do not compromise patients' privacy. However, the proliferation of synthetic data generation techniques has led to the introduction of a wide variety of methods for evaluating the quality of generated data. This makes the task of evaluating generated data from different models challenging as there is no consensus on the methods used. Hence the need for standard ways of evaluating the generated data. In addition, the available methods do not assess whether dependencies between different variables are maintained in the synthetic data. Furthermore, synthetic time series EHRs (patient encounters) are not well investigated, as the available methods do not consider the temporality of patient encounters. In this work, we present an overview of evaluation methods and propose an evaluation framework to guide the evaluation of synthetic EHRs. © 2023 European Federation for Medical Informatics (EFMI) and IOS Press.

Place, publisher, year, edition, pages
Amsterdam: IOS Press, 2023
Series
Studies in Health Technology and Informatics, ISSN 0926-9630, E-ISSN 1879-8365 ; 302
Keywords
Electronic Health Records, evaluation, Synthetic data
National Category
Computer and Information Sciences Medical and Health Sciences
Research subject
Health Innovation, IDC
Identifiers
urn:nbn:se:hh:diva-52041 (URN)10.3233/SHTI230149 (DOI)001071432900094 ()37203694 (PubMedID)2-s2.0-85159759461 (Scopus ID)978-1-64368-388-1 (ISBN)978-1-64368-389-8 (ISBN)
Conference
33rd Medical Informatics Europe Conference: Caring is Sharing - Exploiting the Value in Data for Health and Innovation (MIE2023), Gothenburg, Sweden, 22-25 May, 2023
Available from: 2023-11-15 Created: 2023-11-15 Last updated: 2023-11-16Bibliographically approved
Rahat, M., Kharazian, Z., Sheikholharam Mashhadi, P., Rögnvaldsson, T. & Choudhury, S. (2023). Bridging the Gap: A Comparative Analysis of Regressive Remaining Useful Life Prediction and Survival Analysis Methods for Predictive Maintenance. In: Takehisa Yairi; Samir Khan; Seiji Tsutsumi (Ed.), Proceedings of the Asia Pacific Conference of the PHM Society 2023: . Paper presented at 4th Asia Pacific Conference of the Prognostics and Health Management, Tokyo, Japan, September 11-14, 2023. New York: The Prognostics and Health Management Society, 4
Open this publication in new window or tab >>Bridging the Gap: A Comparative Analysis of Regressive Remaining Useful Life Prediction and Survival Analysis Methods for Predictive Maintenance
Show others...
2023 (English)In: Proceedings of the Asia Pacific Conference of the PHM Society 2023 / [ed] Takehisa Yairi; Samir Khan; Seiji Tsutsumi, New York: The Prognostics and Health Management Society , 2023, Vol. 4Conference paper, Published paper (Refereed)
Abstract [en]

Regressive Remaining Useful Life Prediction and Survival Analysis are two lines of research with similar goals but different origins; one from engineering and the other from survival study in clinical research. Although the two research paths share a common objective of predicting the time to an event, researchers from each path typically do not compare their methods with methods from the other direction. Given the mentioned gap, we propose a framework to compare methods from the two lines of research using run-to-failure datasets. Then by utilizing the proposed framework, we compare six models incorporating three widely recognized degradation models along with two learning algorithms. The first dataset used in this study is C-MAPSS which includes simulation data from aircraft turbofan engines. The second dataset is real-world data from streamed condition monitoring of turbocharger devices installed on a fleet of Volvo trucks.

Place, publisher, year, edition, pages
New York: The Prognostics and Health Management Society, 2023
Series
Proceedings of the Asia Pacific Conference of the PHM Society, E-ISSN 2994-7219
National Category
Electrical Engineering, Electronic Engineering, Information Engineering Computer Systems
Identifiers
urn:nbn:se:hh:diva-52221 (URN)10.36001/phmap.2023.v4i1.3646 (DOI)
Conference
4th Asia Pacific Conference of the Prognostics and Health Management, Tokyo, Japan, September 11-14, 2023
Funder
Knowledge Foundation
Available from: 2023-12-13 Created: 2023-12-13 Last updated: 2023-12-13Bibliographically approved
Alabdallah, A., Rögnvaldsson, T., Fan, Y., Pashami, S. & Ohlsson, M. (2023). Discovering Premature Replacements in Predictive Maintenance Time-to-Event Data. In: Takehisa Yairi; Samir Khan; Seiji Tsutsumi (Ed.), Proceedings of the Asia Pacific Conference of the PHM Society 2023: . Paper presented at 4th Asia Pacific Conference of the Prognostics and Health Management, Tokyo, Japan, September 11-14, 2023. New York: The Prognostics and Health Management Society, 4
Open this publication in new window or tab >>Discovering Premature Replacements in Predictive Maintenance Time-to-Event Data
Show others...
2023 (English)In: Proceedings of the Asia Pacific Conference of the PHM Society 2023 / [ed] Takehisa Yairi; Samir Khan; Seiji Tsutsumi, New York: The Prognostics and Health Management Society , 2023, Vol. 4Conference paper, Published paper (Refereed)
Abstract [en]

Time-To-Event (TTE) modeling using survival analysis in industrial settings faces the challenge of premature replacements of machine components, which leads to bias and errors in survival prediction. Typically, TTE survival data contains information about components and if they had failed or not up to a certain time. For failed components, the time is noted, and a failure is referred to as an event. A component that has not failed is denoted as censored. In industrial settings, in contrast to medical settings, there can be considerable uncertainty in an event; a component can be replaced before it fails to prevent operation stops or because maintenance staff believe that the component is faulty. This shows up as “no fault found” in warranty studies, where a significant proportion of replaced components may appear fault-free when tested or inspected after replacement.

In this work, we propose an expectation-maximization-like method for discovering such premature replacements in survival data. The method is a two-phase iterative algorithm employing a genetic algorithm in the maximization phase to learn better event assignments on a validation set. The learned labels through iterations are accumulated and averaged to be used to initialize the following expectation phase. The assumption is that the more often the event is selected, the more likely it is to be an actual failure and not a “no fault found”.

Experiments on synthesized and simulated data show that the proposed method can correctly detect a significant percentage of premature replacement cases.

Place, publisher, year, edition, pages
New York: The Prognostics and Health Management Society, 2023
Series
Proceedings of the Asia Pacific Conference of the PHM Society, E-ISSN 2994-7219
Keywords
Survival Analysis, Predictive Maintenance, Early Replacements, Genetic Algorithms
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-52105 (URN)10.36001/phmap.2023.v4i1.3609 (DOI)
Conference
4th Asia Pacific Conference of the Prognostics and Health Management, Tokyo, Japan, September 11-14, 2023
Funder
Knowledge Foundation, 20200001
Note

Som manuscript i avhandling/As manuscript in thesis.

Available from: 2023-11-23 Created: 2023-11-23 Last updated: 2023-12-19Bibliographically approved
Chen, K., Rögnvaldsson, T., Nowaczyk, S., Pashami, S., Klang, J. & Sternelöv, G. (2023). Material handling machine activity recognition by context ensemble with gated recurrent units. Engineering applications of artificial intelligence, 126(Part C), Article ID 106992.
Open this publication in new window or tab >>Material handling machine activity recognition by context ensemble with gated recurrent units
Show others...
2023 (English)In: Engineering applications of artificial intelligence, ISSN 0952-1976, E-ISSN 1873-6769, Vol. 126, no Part C, article id 106992Article in journal (Refereed) Published
Abstract [en]

Research on machine activity recognition (MAR) is drawing more attention because MAR can provide productivity monitoring for efficiency optimization, better maintenance scheduling, product design improvement, and potential material savings. A particular challenge of MAR for human-operated machines is the overlap when transiting from one activity to another: during transitions, operators often perform two activities simultaneously, e.g., lifting the fork already while approaching a rack, so the exact time when one activity ends and another begins is uncertain. Machine learning models are often uncertain during such activity transitions, and we propose a novel ensemble-based method adapted to fuzzy transitions in a forklift MAR problem. Unlike traditional ensembles, where models in the ensemble are trained on different subsets of data, or with costs that force them to be diverse in their responses, our approach is to train a single model that predicts several activity labels, each under a different context. These individual predictions are not made by independent networks but are made using a structure that allows for sharing important features, i.e., a context ensemble. The results show that the gated recurrent unit network can provide medium or strong confident context ensembles for 95% of the cases in the test set, and the final forklift MAR result achieves accuracies of 97% for driving and 90% for load-handling activities. This study is the first to highlight the overlapping activity issue in MAR problems and to demonstrate that the recognition results can be significantly improved by designing a machine learning framework that addresses this issue. © 2023 The Author(s)

Place, publisher, year, edition, pages
Oxford: Elsevier, 2023
Keywords
Context ensemble, Gated recurrent unit, Machine activity recognition, Material handling, Productivity monitoring
National Category
Computer Sciences Production Engineering, Human Work Science and Ergonomics
Research subject
Smart Cities and Communities
Identifiers
urn:nbn:se:hh:diva-48552 (URN)10.1016/j.engappai.2023.106992 (DOI)001070748600001 ()2-s2.0-85169031390 (Scopus ID)
Funder
Knowledge Foundation, 20200001
Note

As manuscript in thesis. 

Funding agency: Toyota Material Handling Manufacturing Sweden AB

Available from: 2022-10-31 Created: 2022-10-31 Last updated: 2023-11-28Bibliographically approved
Vettoruzzo, A., Bouguelia, M.-R. & Rögnvaldsson, T. (2023). Meta-Learning from Multimodal Task Distributions Using Multiple Sets of Meta-Parameters. In: 2023 International Joint Conference on Neural Networks (IJCNN): . Paper presented at International Joint Conference on Neural Networks (IJCNN 2023), Gold Coast, Australia, 18-23 June, 2023 (pp. 1-8). Piscataway, NJ: IEEE
Open this publication in new window or tab >>Meta-Learning from Multimodal Task Distributions Using Multiple Sets of Meta-Parameters
2023 (English)In: 2023 International Joint Conference on Neural Networks (IJCNN), Piscataway, NJ: IEEE, 2023, p. 1-8Conference paper, Published paper (Refereed)
Abstract [en]

Meta-learning or learning to learn involves training a model on various learning tasks in a way that allows it to quickly learn new tasks from the same distribution using only a small amount of training data (i.e., few-shot learning). Current meta-learning methods implicitly assume that the distribution over tasks is unimodal and consists of tasks belonging to a common domain, which significantly reduces the variety of task distributions they can handle. However, in real-world applications, tasks are often very diverse and come from multiple different domains, making it challenging to meta-learn common knowledge shared across the entire task distribution. In this paper, we propose a method for meta-learning from a multimodal task distribution. The proposed method learns multiple sets of meta-parameters (acting as different initializations of a neural network model) and uses a task encoder to select the best initialization to fine-tune for a new task. More specifically, with a few training examples from a task sampled from an unknown mode, the proposed method predicts which set of meta-parameters (i.e., model’s initialization) would lead to a fast adaptation and a good post-adaptation performance on that task. We evaluate the proposed method on a diverse set of few-shot regression and image classification tasks. The results demonstrate the superiority of the proposed method compared to other state of-the-art meta-learning methods and the benefit of learning multiple model initializations when tasks are sampled from a multimodal task distribution. © 2023 IEEE.

Place, publisher, year, edition, pages
Piscataway, NJ: IEEE, 2023
Keywords
Meta-Learning, Few-Shot Learning, Transfer Learning, Task Representation, Multimodal Distribution
National Category
Computer Sciences
Identifiers
urn:nbn:se:hh:diva-51352 (URN)10.1109/IJCNN54540.2023.10191944 (DOI)001046198707013 ()2-s2.0-85169561819 (Scopus ID)978-1-6654-8867-9 (ISBN)
Conference
International Joint Conference on Neural Networks (IJCNN 2023), Gold Coast, Australia, 18-23 June, 2023
Available from: 2023-08-10 Created: 2023-08-10 Last updated: 2023-12-05Bibliographically approved
Nilsson, F., Bouguelia, M.-R. & Rögnvaldsson, T. (2023). Practical Joint Human-Machine Exploration of Industrial Time Series Using the Matrix Profile. Data mining and knowledge discovery, 37, 1-38
Open this publication in new window or tab >>Practical Joint Human-Machine Exploration of Industrial Time Series Using the Matrix Profile
2023 (English)In: Data mining and knowledge discovery, ISSN 1384-5810, E-ISSN 1573-756X, Vol. 37, p. 1-38Article in journal (Refereed) Published
Abstract [en]

Technological advancements and widespread adaptation of new technology in industry have made industrial time series data more available than ever before. With this development grows the need for versatile methods for mining industrial time series data. This paper introduces a practical approach for joint human-machine exploration of industrial time series data using the Matrix Profile (MP), and presents some challenges involved. The approach is demonstrated on three real-life industrial data sets to show how it enables the user to quickly extract semantic information, detect cycles, find deviating patterns, and gain a deeper understanding of the time series. A benchmark test is also presented on ECG (electrocardiogram) data, showing that the approach works well in comparison to previously suggested methods for extracting relevant time series motifs. © 2022, The Author(s).

Place, publisher, year, edition, pages
New York, NY: Springer, 2023
Keywords
Time Series, Matrix Profile, Motif Discovery, Industry 4.0
National Category
Computer Sciences
Research subject
Smart Cities and Communities
Identifiers
urn:nbn:se:hh:diva-48164 (URN)10.1007/s10618-022-00871-y (DOI)000864041100001 ()2-s2.0-85139448908 (Scopus ID)
Note

Funding: Open access funding provided by Halmstad University.

Available from: 2022-09-28 Created: 2022-09-28 Last updated: 2023-01-12Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0001-5163-2997

Search in DiVA

Show all publications