hh.sePublications
Change search
Refine search result
123 1 - 50 of 123
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Abuella, Mohamed
    et al.
    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR).
    Atoui, M. Amine
    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR).
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR).
    Johansson, Simon
    CetaSol AB, Gothenburg, Sweden.
    Faghani, Ethan
    CetaSol AB, Gothenburg, Sweden.
    Data-Driven Explainable Artificial Intelligence for Energy Efficiency in Short-Sea Shipping2023In: Machine Learning and Knowledge Discovery in Databases: Applied Data Science and Demo Track: European Conference, ECML PKDD 2023, Turin, Italy, September 18–22, 2023, Proceedings, Part VII / [ed] Gianmarco De Francisci Morales; Claudia Perlich; Natali Ruchansky; Nicolas Kourtellis; Elena Baralis; Francesco Bonchi, Cham: Springer, 2023, Vol. 14175, p. 226-241Conference paper (Refereed)
    Abstract [en]

    The maritime industry is under pressure to increase energy efficiency for climate change mitigation. Navigational data, combining vessel operational and environmental measurements from onboard instruments and external sources, are critical for achieving this goal. Short-sea shipping presents a unique challenge due to the significant influence of surrounding landscape characteristics. With high-resolution onboard data increasingly accessible through IoT devices, appropriate data representations and AI/ML analytical tools are needed for effective decision support. The aim of this study is to investigate the fuel consumption estimation model’s role in developing an energy efficiency decision support tool. ML models that lacking explainability may neglect important factors and essential constraints, such as the need to meet arrival time requirements. Onboard weather measurements are compared to external forecasts, and our findings demonstrate the necessity of eXplainable Artificial Intelligence (XAI) techniques for effective decision support. Real-world data from a short-sea passenger vessel in southern Sweden, consisting of 1754 voyages over 15 months (More of data description and code sources of this study can be found in the GitHub repository at https://github.com/MohamedAbuella/ST4EESSS), are used to support our conclusions.  © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

  • 2.
    Altarabichi, Mohammed Ghaith
    et al.
    Halmstad University, School of Information Technology.
    Alabdallah, Abdallah
    Halmstad University, School of Information Technology.
    Pashami, Sepideh
    Halmstad University, School of Information Technology.
    Ohlsson, Mattias
    Halmstad University, School of Information Technology.
    Rögnvaldsson, Thorsteinn
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Improving Concordance Index in Regression-based Survival Analysis: Discovery of Loss Function for Neural Networks2024Manuscript (preprint) (Other academic)
    Abstract [en]

    In this work, we use an Evolutionary Algorithm (EA) to discover a novel Neural Network (NN) regression-based survival loss function with the aim of improving the C-index performance. Our contribution is threefold; firstly, we propose an evolutionary meta-learning algorithm SAGA$_{loss}$ for optimizing a neural-network regression-based loss function that maximizes the C-index; our algorithm consistently discovers specialized loss functions that outperform MSCE. Secondly, based on our analysis of the evolutionary search results, we highlight a non-intuitive insight that signifies the importance of the non-zero gradient for the censored cases part of the loss function, a property that is shown to be useful in improving concordance. Finally, based on this insight, we propose MSCE$_{Sp}$, a novel survival regression loss function that can be used off-the-shelf and generally performs better than the Mean Squared Error for censored cases. We performed extensive experiments on 19 benchmark datasets to validate our findings.

  • 3.
    Altarabichi, Mohammed Ghaith
    et al.
    Halmstad University, School of Information Technology.
    Fan, Yuantao
    Halmstad University, School of Information Technology.
    Pashami, Sepideh
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR).
    Rögnvaldsson, Thorsteinn
    Halmstad University, School of Information Technology.
    Predicting state of health and end of life for batteries in hybrid energy buses2020In: Proceedings of the 30th European Safety and Reliability Conference and the 15th Probabilistic Safety Assessment and Management Conference / [ed] Baraldi, Piero; Di Maio, Francesco; Zio, Enrico, Singapore: Research Publishing Services, 2020, p. 1231-1231Conference paper (Refereed)
    Abstract [en]

    There is a major ongoing transition from utilizing fossil fuel to electricity in buses for enabling a more sustainable, environmentally friendly, and connected transportation ecosystem. Batteries are expensive, up to 30% of the total cost for the vehicle (A. Fotouhi 2016), and considered safety-critical components for electric vehicles (EV). As they deteriorate over time, monitoring the health status and performing the maintenance accordingly in a proactive manner is crucial to achieving not only a safe and sustainable transportation system but also a cost-effective operation and thus a greater market satisfaction. As a widely used indicator, the State of Health (SOH) is a measurement that reflects the current capability of the battery in comparison to an ideal condition. Accurate estimation of SOH is important to evaluate the validity of the batteries for the intended application and can be utilized as a proxy to estimate the remaining useful life (RUL) and predict the end-of-life (EOL) of batteries for maintenance planning. The SOH is computed via an on-board computing device, i.e. battery management unit (BMU), which is commonly developed based on controlled experiments and many of them are physical-model based approaches that only depend on the internal parameters of the battery (B. Pattipati 2008; M. H. Lipu 2018). However, the deterioration processes of batteries in hybrid and full-electric buses depend not only on the designing parameters but also on the operating environment and usage patterns of the vehicle. Therefore, utilizing multiple data sources to estimate the health status and EOL of the batteries is of potential internet. In this study, a data-driven prognostic method is developed to estimate SOH and predict EOL for batteries in heterogeneous fleets of hybrid buses, using various types of data sources, e.g. physical configuration of the vehicle, deployment information, on-board sensor readings, and diagnostic fault codes. A set of new features was generated from the existing sensor readings by inducing artificial resets on each battery replacement. A neural network-based regression model achieved accurate estimates of battery SOH status. Another network was used to indicate the EOL of batteries and the result was evaluated using battery replacement based on the current maintenance strategy. © ESREL2020-PSAM15 Organizers. Published by Research Publishing, Singapore.

  • 4.
    Altarabichi, Mohammed Ghaith
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Fan, Yuantao
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Pashami, Sepideh
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Sheikholharam Mashhadi, Peyman
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Extracting Invariant Features for Predicting State of Health of Batteries in Hybrid Energy Buses2021In: 2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA), Porto, Portugal, 6-9 Oct., 2021, IEEE, 2021, p. 1-6Conference paper (Refereed)
    Abstract [en]

    Batteries are a safety-critical and the most expensive component for electric vehicles (EVs). To ensure the reliability of the EVs in operation, it is crucial to monitor the state of health of those batteries. Monitoring their deterioration is also relevant to the sustainability of the transport solutions, through creating an efficient strategy for utilizing the remaining capacity of the battery and its second life. Electric buses, similar to other EVs, come in many different variants, including different configurations and operating conditions. Developing new degradation models for each existing combination of settings can become challenging from different perspectives such as unavailability of failure data for novel settings, heterogeneity in data, low amount of data available for less popular configurations, and lack of sufficient engineering knowledge. Therefore, being able to automatically transfer a machine learning model to new settings is crucial. More concretely, the aim of this work is to extract features that are invariant across different settings.

    In this study, we propose an evolutionary method, called genetic algorithm for domain invariant features (GADIF), that selects a set of features to be used for training machine learning models, in such a way as to maximize the invariance across different settings. A Genetic Algorithm, with each chromosome being a binary vector signaling selection of features, is equipped with a specific fitness function encompassing both the task performance and domain shift. We contrast the performance, in migrating to unseen domains, of our method against a number of classical feature selection methods without any transfer learning mechanism. Moreover, in the experimental result section, we analyze how different features are selected under different settings. The results show that using invariant features leads to a better generalization of the machine learning models to an unseen domain.

  • 5.
    Altarabichi, Mohammed Ghaith
    et al.
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Pashami, Sepideh
    Halmstad University, School of Information Technology.
    Sheikholharam Mashhadi, Peyman
    Halmstad University, School of Information Technology.
    Fast Genetic Algorithm for feature selection — A qualitative approximation approach2023In: Expert systems with applications, ISSN 0957-4174, E-ISSN 1873-6793, Vol. 211, article id 118528Article in journal (Refereed)
    Abstract [en]

    Evolutionary Algorithms (EAs) are often challenging to apply in real-world settings since evolutionary computations involve a large number of evaluations of a typically expensive fitness function. For example, an evaluation could involve training a new machine learning model. An approximation (also known as meta-model or a surrogate) of the true function can be used in such applications to alleviate the computation cost. In this paper, we propose a two-stage surrogate-assisted evolutionary approach to address the computational issues arising from using Genetic Algorithm (GA) for feature selection in a wrapper setting for large datasets. We define “Approximation Usefulness” to capture the necessary conditions to ensure correctness of the EA computations when an approximation is used. Based on this definition, we propose a procedure to construct a lightweight qualitative meta-model by the active selection of data instances. We then use a meta-model to carry out the feature selection task. We apply this procedure to the GA-based algorithm CHC (Cross generational elitist selection, Heterogeneous recombination and Cataclysmic mutation) to create a Qualitative approXimations variant, CHCQX. We show that CHCQX converges faster to feature subset solutions of significantly higher accuracy (as compared to CHC), particularly for large datasets with over 100K instances. We also demonstrate the applicability of the thinking behind our approach more broadly to Swarm Intelligence (SI), another branch of the Evolutionary Computation (EC) paradigm with results of PSOQX, a qualitative approximation adaptation of the Particle Swarm Optimization (PSO) method. A GitHub repository with the complete implementation is available. © 2022 The Author(s)

  • 6.
    Altarabichi, Mohammed Ghaith
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Pashami, Sepideh
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Sheikholharam Mashhadi, Peyman
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Surrogate-Assisted Genetic Algorithm for Wrapper Feature Selection2021In: 2021 IEEE Congress on Evolutionary Computation (CEC), IEEE, 2021, p. 776-785Conference paper (Refereed)
    Abstract [en]

    Feature selection is an intractable problem, therefore practical algorithms often trade off the solution accuracy against the computation time. In this paper, we propose a novel multi-stage feature selection framework utilizing multiple levels of approximations, or surrogates. Such a framework allows for using wrapper approaches in a much more computationally efficient way, significantly increasing the quality of feature selection solutions achievable, especially on large datasets. We design and evaluate a Surrogate-Assisted Genetic Algorithm (SAGA) which utilizes this concept to guide the evolutionary search during the early phase of exploration. SAGA only switches to evaluating the original function at the final exploitation phase.

    We prove that the run-time upper bound of SAGA surrogate-assisted stage is at worse equal to the wrapper GA, and it scales better for induction algorithms of high order of complexity in number of instances. We demonstrate, using 14 datasets from the UCI ML repository, that in practice SAGA significantly reduces the computation time compared to a baseline wrapper Genetic Algorithm (GA), while converging to solutions of significantly higher accuracy. Our experiments show that SAGA can arrive at near-optimal solutions three times faster than a wrapper GA, on average. We also showcase the importance of evolution control approach designed to prevent surrogates from misleading the evolutionary search towards false optima.

  • 7.
    Altarabichi, Mohammed Ghaith
    et al.
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Pashami, Sepideh
    Halmstad University, School of Information Technology.
    Sheikholharam Mashhadi, Peyman
    Halmstad University, School of Information Technology.
    Handl, Julia
    University of Manchester, Manchester, United Kingdom.
    Rolling the Dice for Better Deep Learning Performance: A Study of Randomness Techniques in Deep Neural Networks2024Manuscript (preprint) (Other (popular science, discussion, etc.))
    Abstract [en]

    This paper presents a comprehensive empirical investigation into the interactions between various randomness techniques in Deep Neural Networks (DNNs) and how they contribute to network performance. It is well-established that injecting randomness into the training process of DNNs, through various approaches at different stages, is often beneficial for reducing overfitting and improving generalization. However, the interactions between randomness techniques such as weight noise, dropout, and many others remain poorly understood. Consequently, it is challenging to determine which methods can be effectively combined to optimize DNN performance. To address this issue, we categorize the existing randomness techniques into four key types: data, model, optimization, and learning. We use this classification to identify gaps in the current coverage of potential mechanisms for the introduction of noise, leading to proposing two new techniques: adding noise to the loss function and random masking of the gradient updates.

    In our empirical study, we employ a Particle Swarm Optimizer (PSO) to explore the space of possible configurations to answer where and how much randomness should be injected to maximize DNN performance. We assess the impact of various types and levels of randomness for DNN architectures applied to standard computer vision benchmarks: MNIST, FASHION-MNIST, CIFAR10, and CIFAR100. Across more than 30\,000 evaluated configurations, we perform a detailed examination of the interactions between randomness techniques and their combined impact on DNN performance. Our findings reveal that randomness in data augmentation and in weight initialization are the main contributors to performance improvement. Additionally, correlation analysis demonstrates that different optimizers, such as Adam and Gradient Descent with Momentum, prefer distinct types of randomization during the training process. A GitHub repository with the complete implementation and generated dataset is available\footnote[1]{https://github.com/Ghaith81/Radnomness\_in\_Neural\_Network}.

  • 8.
    Altarabichi, Mohammed Ghaith
    et al.
    Halmstad University, School of Information Technology.
    Pashami, Sepideh
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Sheikholharam Mashhadi, Peyman
    Halmstad University, School of Information Technology.
    Fast Genetic Algorithm For Feature Selection — A Qualitative Approximation Approach2023In: Evolutionary Computation Conference Companion (GECCO ’23 Companion), July 15–19, 2023, Lisbon, Portugal, New York, NY: Association for Computing Machinery (ACM), 2023, p. 11-12Conference paper (Refereed)
    Abstract [en]

    We propose a two-stage surrogate-assisted evolutionary approach to address the computational issues arising from using Genetic Algorithm (GA) for feature selection in a wrapper setting for large datasets. The proposed approach involves constructing a lightweight qualitative meta-model by sub-sampling data instances and then using this meta-model to carry out the feature selection task. We define "Approximation Usefulness" to capture the necessary conditions that allow the meta-model to lead the evolutionary computations to the correct maximum of the fitness function. Based on our procedure we create CHCQX a Qualitative approXimations variant of the GA-based algorithm CHC (Cross generational elitist selection, Heterogeneous recombination and Cataclysmic mutation). We show that CHCQX converges faster to feature subset solutions of significantly higher accuracy, particularly for large datasets with over 100K instances. We also demonstrate the applicability of our approach to Swarm Intelligence (SI), with results of PSOQX, a qualitative approximation adaptation of the Particle Swarm Optimization (PSO) method. A GitHub repository with the complete implementation is available2. This paper for the Hot-off-the-Press track at GECCO 2023 summarizes the original work published at [3].

    References

    [1] Mohammed Ghaith Altarabichi, Yuantao Fan, Sepideh Pashami, Peyman Sheikholharam Mashhadi, and Sławomir Nowaczyk. 2021. Extracting invariant features for predicting state of health of batteries in hybrid energy buses. In 2021 ieee 8th international conference on data science and advanced analytics (dsaa). IEEE, 1–6.

    [2] Mohammed Ghaith Altarabichi, Sławomir Nowaczyk, Sepideh Pashami, and Peyman Sheikholharam Mashhadi. 2021. Surrogate-assisted genetic algorithm for wrapper feature selection. In 2021 IEEE Congress on Evolutionary Computation (CEC). IEEE, 776–785.

    [3] Mohammed Ghaith Altarabichi, Sławomir Nowaczyk, Sepideh Pashami, and Peyman Sheikholharam Mashhadi. 2023. Fast Genetic Algorithm for feature selection—A qualitative approximation approach. Expert systems with applications 211 (2023), 118528.

    © 2023 Copyright held by the owner/author(s).

  • 9.
    Altarabichi, Mohammed Ghaith
    et al.
    Halmstad University, School of Information Technology.
    Sheikholharam Mashhadi, Peyman
    Halmstad University, School of Information Technology.
    Fan, Yuantao
    Halmstad University, School of Information Technology.
    Pashami, Sepideh
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Del Moral, Pablo
    Halmstad University, School of Information Technology.
    Rahat, Mahmoud
    Halmstad University, School of Information Technology.
    Rögnvaldsson, Thorsteinn
    Halmstad University, School of Information Technology.
    Stacking Ensembles of Heterogenous Classifiers for Fault Detection in Evolving Environments2020In: Proceedings of the 30th European Safety and Reliability Conference and the 15th Probabilistic Safety Assessment and Management Conference / [ed] Piero Baraldi; Francesco Di Maio; Enrico Zio, Singapore: Research Publishing Services, 2020, p. 1068-1068Conference paper (Refereed)
    Abstract [en]

    Monitoring the condition, detecting faults, and modeling the degradation of industrial equipment are important challenges in Prognostics and Health Management (PHM) field. Our solution to the challenge demonstrated a multi-stage approach for detecting faults in a group of identical industrial equipment, composed of four identical interconnected components, that have been deployed to the evolving environment with changes in operational and environmental conditions. In the first stage, a stacked ensemble of heterogeneous classifiers was applied to predict the state of each component of the equipment individually. In the second stage, a low pass filter was applied to smoothen the predictions cast by stacked ensembles, utilizing temporal information of the prediction sequence. © ESREL2020-PSAM15 Organizers. Published by Research Publishing, Singapore.

  • 10.
    Amirhossein, Berenji
    et al.
    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR).
    Taghiyarrenani, Zahra
    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR).
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR).
    curr2vib: Modality Embedding Translation for Broken-Rotor Bar Detection2023In: Machine Learning and Principles and Practice of Knowledge Discovery in Databases: International Workshops of ECML PKDD 2022, Grenoble, France, September 19–23, 2022, Proceedings, Part II / [ed] Irena Koprinska et al., Cham: Springer Nature, 2023, Vol. 1753, p. 423-437Conference paper (Refereed)
    Abstract [en]

    Recently and due to the advances in sensor technology and Internet-of-Things, the operation of machinery can be monitored, using a higher number of sources and modalities. In this study, we demonstrate that Multi-Modal Translation is capable of transferring knowledge from a modality with higher level of applicability (more usefulness to solve an specific task) but lower level of accessibility (how easy and affordable it is to collect information from this modality) to another one with higher level of accessibility but lower level of applicability. Unlike the fusion of multiple modalities which requires all of the modalities to be available during the deployment stage, our proposed method depends only on the more accessible one; which results in the reduction of the costs regarding instrumentation equipment. The presented case study demonstrates that by the employment of the proposed method we are capable of replacing five acceleration sensors with three current sensors, while the classification accuracy is also increased by more than 1%.

    Download full text (pdf)
    fulltext
  • 11.
    Angelsmark, Ola
    et al.
    Department of Computer Science, Lund University, Lund, Sweden.
    Malec, Jacek
    Department of Computer Science, Lund University, Lund, Sweden.
    Nilsson, Klas
    Department of Computer Science, Lund University, Lund, Sweden.
    Nowaczyk, Sławomir
    Department of Computer Science, Lund University, Lund, Sweden.
    Prosperi, Leonardo
    Department of Computer Science, Lund University, Lund, Sweden.
    Knowledge Representation for Reconfigurable Automation Systems2007Conference paper (Refereed)
    Abstract [en]

    This article describes the work in progress on knowledge representation formalisms chosen for use in the European project SIARAS. Skill-Based Inspection and Assembly for Reconfigurable Automation Systems has a goal of creating intelligent support system for reconfiguration and adaptation of assembly systems. Knowledge is represented in an ontology expressed in OWL, for generic reasoning in Description Logic, and in a number of special-purpose reasoning modules, specific for the application domain.

  • 12.
    Ashfaq, Awais
    et al.
    Halmstad University, School of Information Technology.
    Lingman, Markus
    Halland Hospital, Region Halland, Sweden.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    KAFE: Knowledge and Frequency Adapted Embeddings2022In: Machine Learning, Optimization, and Data Science: 7th International Conference, LOD 2021, Grasmere, UK, October 4–8, 2021, Revised Selected Papers, Part II / [ed] Giuseppe Nicosia; Varun Ojha; Emanuele La Malfa; Gabriele La Malfa; Giorgio Jansen; Panos M. Pardalos; Giovanni Giuffrida; Renato Umeton, Cham: Springer, 2022, Vol. 13164, p. 132-146Conference paper (Refereed)
    Abstract [en]

    Word embeddings are widely used in several Natural Language Processing (NLP) applications. The training process typically involves iterative gradient updates of each word vector. This makes word frequency a major factor in the quality of embedding, and in general the embedding of words with few training occurrences end up being of poor quality. This is problematic since rare and frequent words, albeit semantically similar, might end up far from each other in the embedding space.

    In this study, we develop KAFE (Knowledge And Frequency adapted Embeddings) which combines adversarial principles and knowledge graph to efficiently represent both frequent and rare words. The goal of adversarial training in KAFE is to minimize the spatial distinguishability (separability) of frequent and rare words in the embedding space. The knowledge graph encourages the embedding to follow the structure of the domain-specific hierarchy, providing an informative prior that is particularly important for words with low amount of training data. We demonstrate the performance of KAFE in representing clinical diagnoses using real-world Electronic Health Records (EHR) data coupled with a knowledge graph. EHRs are notorious for including ever-increasing numbers of rare concepts that are important to consider when defining the state of the patient for various downstream applications. Our experiments demonstrate better intelligibility through visualisation, as well as higher prediction and stability scores of KAFE over state-of-the-art. © Springer Nature Switzerland AG 2022

  • 13.
    Ashfaq, Awais
    et al.
    Halmstad University, School of Information Technology. Halland Hospital, Halmstad, Sweden.
    Lingman, Markus
    Halmstad University, School of Information Technology. Halland Hospital, Halmstad, Sweden; Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden.
    Sensoy, Murat
    Amazon, Seattle, United States.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    DEED: DEep Evidential Doctor2023In: Artificial Intelligence, ISSN 0004-3702, E-ISSN 1872-7921, Vol. 325, article id 104019Article in journal (Refereed)
    Abstract [en]

    As Deep Neural Networks (DNN) make their way into safety-critical decision processes, it becomes imperative to have robust and reliable uncertainty estimates for their predictions for both in-distribution and out-of-distribution (OOD) examples. This is particularly important in real-life high-risk settings such as healthcare, where OOD examples (e.g., patients with previously unseen or rare labels, i.e., diagnoses) are frequent, and an incorrect clinical decision might put human life in danger, in addition to having severe ethical and financial costs. While evidential uncertainty estimates for deep learning have been studied for multi-class problems, research in multi-label settings remains untapped. In this paper, we propose a DEep Evidential Doctor (DEED), which is a novel deterministic approach to estimate multi-label targets along with uncertainty. We achieve this by placing evidential priors over the original likelihood functions and directly estimating the parameters of the evidential distribution using a novel loss function. Additionally, we build a redundancy layer (particularly for high uncertainty and OOD examples) to minimize the risk associated with erroneous decisions based on dubious predictions. We achieve this by learning the mapping between the evidential space and a continuous semantic label embedding space via a recurrent decoder. Thereby inferring, even in the case of OOD examples, reasonably close predictions to avoid catastrophic consequences. We demonstrate the effectiveness of DEED on a digit classification task based on a modified multi-label MNIST dataset, and further evaluate it on a diagnosis prediction task from a real-life electronic health record dataset. We highlight that in terms of prediction scores, our approach is on par with the existing state-of-the-art having a clear advantage of generating reliable, memory and time-efficient uncertainty estimates with minimal changes to any multi-label DNN classifier. © 2023 The Author(s)

  • 14.
    Ashfaq, Awais
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research. Halland Hospital, Region Halland, Halmstad, Sweden.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Machine learning in healthcare - a system’s perspective2019In: Proceedings of the ACM SIGKDD Workshop on Epidemiology meets Data Mining and Knowledge Discovery (epiDAMIK) / [ed] B. Aditya Prakash, Anil Vullikanti, Shweta Bansal, Adam Sadelik, Arlington, 2019, p. 14-17Conference paper (Refereed)
    Abstract [en]

    A consequence of the fragmented and siloed healthcare landscape is that patient care (and data) is split along multitude of different facilities and computer systems and enabling interoperability between these systems is hard. The lack interoperability not only hinders continuity of care and burdens providers, but also hinders effective application of Machine Learning (ML) algorithms. Thus, most current ML algorithms, designed to understand patient care and facilitate clinical decision-support, are trained on limited datasets. This approach is analogous to the Newtonian paradigm of Reductionism in which a system is broken down into elementary components and a description of the whole is formed by understanding those components individually. A key limitation of the reductionist approach is that it ignores the component-component interactions and dynamics within the system which are often of prime significance in understanding the overall behaviour of complex adaptive systems (CAS). Healthcare is a CAS.

    Though the application of ML on health data have shown incremental improvements for clinical decision support, ML has a much a broader potential to restructure care delivery as a whole and maximize care value. However, this ML potential remains largely untapped: primarily due to functional limitations of Electronic Health Records (EHR) and the inability to see the healthcare system as a whole. This viewpoint (i) articulates the healthcare as a complex system which has a biological and an organizational perspective, (ii) motivates with examples, the need of a system's approach when addressing healthcare challenges via ML and, (iii) emphasizes to unleash EHR functionality - while duly respecting all ethical and legal concerns - to reap full benefits of ML.

    Download full text (pdf)
    Epidamik proceedings
  • 15.
    Ashfaq, Awais
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research. Halland Hospital, Region Halland, Sweden.
    Pinheiro Sant'Anna, Anita
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Lingman, Markus
    Halland Hospital, Region Halland, Sweden & Institute of Medicine, Dept. of Molecular and Clinical Medicine/Cardiology, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Readmission prediction using deep learning on electronic health records2019In: Journal of Biomedical Informatics, ISSN 1532-0464, E-ISSN 1532-0480, Vol. 97, article id 103256Article in journal (Refereed)
    Abstract [en]

    Unscheduled 30-day readmissions are a hallmark of Congestive Heart Failure (CHF) patients that pose significant health risks and escalate care cost. In order to reduce readmissions and curb the cost of care, it is important to initiate targeted intervention programs for patients at risk of readmission. This requires identifying high-risk patients at the time of discharge from hospital. Here, using real data from over 7,500 CHF patients hospitalized between 2012 and 2016 in Sweden, we built and tested a deep learning framework to predict 30-day unscheduled readmission. We present a cost-sensitive formulation of Long Short-Term Memory (LSTM) neural network using expert features and contextual embedding of clinical concepts. This study targets key elements of an Electronic Health Record (EHR) driven prediction model in a single framework: using both expert and machine derived features, incorporating sequential patterns and addressing the class imbalance problem. We show that the model with all key elements achieves a higher discrimination ability (AUC 0.77) compared to the rest. Additionally, we present a simple financial analysis to estimate annual savings if targeted interventions are offered to high risk patients. © 2019 The Authors

  • 16.
    Bae, Juhee
    et al.
    University of Skövde, Skövde, Sweden.
    Helldin, Tove
    University of Skövde, Skövde, Sweden.
    Riveiro, Maria
    Jönköping University, Jönköping, Sweden & University of Skövde, Skövde, Sweden.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Bouguelia, Mohamed-Rafik
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Falkman, Göran
    University of Skövde, Skövde, Sweden.
    Interactive Clustering: A Comprehensive Review2020In: ACM Computing Surveys, ISSN 0360-0300, E-ISSN 1557-7341, Vol. 53, no 1, article id 1Article in journal (Refereed)
    Abstract [en]

    In this survey, 105 papers related to interactive clustering were reviewed according to seven perspectives: (1) on what level is the interaction happening, (2) which interactive operations are involved, (3) how user feedback is incorporated, (4) how interactive clustering is evaluated, (5) which data and (6) which clustering methods have been used, and (7) what outlined challenges there are. This article serves as a comprehensive overview of the field and outlines the state of the art within the area as well as identifies challenges and future research needs. © 2020 Copyright held by the owner/author(s).

  • 17.
    Berenji, Amirhossein
    et al.
    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR).
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR).
    Taghiyarrenani, Zahra
    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR).
    Data-Centric Perspective on Explainability Versus Performance Trade-Off2023In: Advances in Intelligent Data Analysis XXI: 21st International Symposium on Intelligent Data Analysis, IDA 2023, Louvain-la-Neuve, Belgium, April 12–14, 2023, Proceedings / [ed] Bruno Crémilleux, Sibylle Hess, Siegfried Nijssen, Cham: Springer, 2023, Vol. 13876, p. 42-54Conference paper (Refereed)
    Abstract [en]

    The performance versus interpretability trade-off has been well-established in the literature for many years in the context of machine learning models. This paper demonstrates its twin, namely the data-centric performance versus interpretability trade-off. In a case study of bearing fault diagnosis, we found that substituting the original acceleration signal with a demodulated version offers a higher level of interpretability, but it comes at the cost of significantly lower classification performance. We demonstrate these results on two different datasets and across four different machine learning algorithms. Our results suggest that “there is no free lunch,” i.e., the contradictory relationship between interpretability and performance should be considered earlier in the analysis process than it is typically done in the literature today; in other words, already in the preprocessing and feature extraction step. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

    Download full text (pdf)
    fulltext
  • 18.
    Bobek, Szymon
    et al.
    Jagiellonian University, Krakow, Poland.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Pashami, Sepideh
    Halmstad University, School of Information Technology.
    Taghiyarrenani, Zahra
    Halmstad University, School of Information Technology.
    Nalepa, Grzegorz J.
    Jagiellonian University, Krakow, Poland.
    Towards Explainable Deep Domain Adaptation2023In: Artificial Intelligence. ECAI 2023 International Workshops: Part 1 / [ed] Sławomir Nowaczyk, Przemysław Biecek, Neo Christopher Chung, Mauro Vallati, Paweł Skruch, Joanna Jaworek-Korjakowska..., Cham: Springer, 2023, p. 101-113Conference paper (Refereed)
    Abstract [en]

    In many practical applications data used for training a machine learning model and the deployment data does not always preserve the same distribution. Transfer learning and, in particular, domain adaptation allows to overcome this issue, by adapting the source model to a new target data distribution and therefore generalizing the knowledge from source to target domain. In this work, we present a method that makes the adaptation process more transparent by providing two complementary explanation mechanisms. The first mechanism explains how the source and target distributions are aligned in the latent space of the domain adaptation model. The second mechanism provides descriptive explanations on how the decision boundary changes in the adapted model with respect to the source model. Along with a description of a method, we also provide initial results obtained on publicly available, real-life dataset.

  • 19.
    Bouguelia, Mohamed-Rafik
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Karlsson, Alexander
    University of Skövde, Skövde, Sweden.
    Pashami, Sepideh
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Holst, Anders
    Swedish Institute of Computer Science, Kista, Sweden.
    Mode tracking using multiple data streams2018In: Information Fusion, ISSN 1566-2535, E-ISSN 1872-6305, Vol. 43, p. 33-46Article in journal (Refereed)
    Abstract [en]

    Most existing work in information fusion focuses on combining information with well-defined meaning towards a concrete, pre-specified goal. In contradistinction, we instead aim for autonomous discovery of high-level knowledge from ubiquitous data streams. This paper introduces a method for recognition and tracking of hidden conceptual modes, which are essential to fully understand the operation of complex environments. We consider a scenario of analyzing usage of a fleet of city buses, where the objective is to automatically discover and track modes such as highway route, heavy traffic, or aggressive driver, based on available on-board signals. The method we propose is based on aggregating the data over time, since the high-level modes are only apparent in the longer perspective. We search through different features and subsets of the data, and identify those that lead to good clusterings, interpreting those clusters as initial, rough models of the prospective modes. We utilize Bayesian tracking in order to continuously improve the parameters of those models, based on the new data, while at the same time following how the modes evolve over time. Experiments with artificial data of varying degrees of complexity, as well as on real-world datasets, prove the effectiveness of the proposed method in accurately discovering the modes and in identifying which one best explains the current observations from multiple data streams. © 2017 Elsevier B.V. All rights reserved.

  • 20.
    Bouguelia, Mohamed-Rafik
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Payberah, Amir H.
    Swedish Institute of Computer Science, Stockholm, Sweden.
    An adaptive algorithm for anomaly and novelty detection in evolving data streams2018In: Data mining and knowledge discovery, ISSN 1384-5810, E-ISSN 1573-756X, Vol. 32, no 6, p. 1597-1633Article in journal (Refereed)
    Abstract [en]

    In the era of big data, considerable research focus is being put on designing efficient algorithms capable of learning and extracting high-level knowledge from ubiquitous data streams in an online fashion. While, most existing algorithms assume that data samples are drawn from a stationary distribution, several complex environments deal with data streams that are subject to change over time. Taking this aspect into consideration is an important step towards building truly aware and intelligent systems. In this paper, we propose GNG-A, an adaptive method for incremental unsupervised learning from evolving data streams experiencing various types of change. The proposed method maintains a continuously updated network (graph) of neurons by extending the Growing Neural Gas algorithm with three complementary mechanisms, allowing it to closely track both gradual and sudden changes in the data distribution. First, an adaptation mechanism handles local changes where the distribution is only non-stationary in some regions of the feature space. Second, an adaptive forgetting mechanism identifies and removes neurons that become irrelevant due to the evolving nature of the stream. Finally, a probabilistic evolution mechanism creates new neurons when there is a need to represent data in new regions of the feature space. The proposed method is demonstrated for anomaly and novelty detection in non-stationary environments. Results show that the method handles different data distributions and efficiently reacts to various types of change. © 2018 The Author(s)

    Download full text (pdf)
    bouguelia-dami-gnga
  • 21.
    Bouguelia, Mohamed-Rafik
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Santosh, K. C.
    The University of South Dakota, Vermillion, South Dakota, USA.
    Verikas, Antanas
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Agreeing to disagree: active learning with noisy labels without crowdsourcing2018In: International Journal of Machine Learning and Cybernetics, ISSN 1868-8071, E-ISSN 1868-808X, Vol. 9, no 8, p. 1307-1319Article in journal (Refereed)
    Abstract [en]

    We propose a new active learning method for classification, which handles label noise without relying on multiple oracles (i.e., crowdsourcing). We propose a strategy that selects (for labeling) instances with a high influence on the learned model. An instance x is said to have a high influence on the model h, if training h on x (with label y = h(x)) would result in a model that greatly disagrees with h on labeling other instances. Then, we propose another strategy that selects (for labeling) instances that are highly influenced by changes in the learned model. An instance x is said to be highly influenced, if training h with a set of instances would result in a committee of models that agree on a common label for x but disagree with h(x). We compare the two strategies and we show, on different publicly available datasets, that selecting instances according to the first strategy while eliminating noisy labels according to the second strategy, greatly improves the accuracy compared to several benchmarking methods, even when a significant amount of instances are mislabeled. © Springer-Verlag Berlin Heidelberg 2017

    Download full text (pdf)
    BougueliaAL
  • 22.
    Bouguelia, Mohamed-Rafik
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Pashami, Sepideh
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Multi-Task Representation Learning2017In: 30th Annual Workshop ofthe Swedish Artificial Intelligence Society SAIS 2017: May 15–16, 2017, Karlskrona, Sweden / [ed] Niklas Lavesson, Linköping: Linköping University Electronic Press, 2017, p. 53-59Conference paper (Refereed)
    Abstract [en]

    The majority of existing machine learning algorithms assume that training examples are already represented with sufficiently good features, in practice ones that are designed manually. This traditional way of preprocessing the data is not only tedious and time consuming, but also not sufficient to capture all the different aspects of the available information. With big data phenomenon, this issue is only going to grow, as the data is rarely collected and analyzed with a specific purpose in mind, and more often re-used for solving different problems. Moreover, the expert knowledge about the problem which allows them to come up with good representations does not necessarily generalize to other tasks. Therefore, much focus has been put on designing methods that can automatically learn features or representations of the data instead of learning from handcrafted features. However, a lot of this work used ad hoc methods and the theoretical understanding in this area is lacking.

    Download full text (pdf)
    fulltext
  • 23.
    Byttner, Stefan
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Prytz, Rune
    Volvo Group Trucks Technology, Gothenburg, Sweden.
    Rögnvaldsson, Thorsteinn
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    A field test with self-organized modeling for knowledge discovery in a fleet of city buses2013In: 2013 IEEE International Conference on Mechatronics and Automation (ICMA 2013) / [ed] Shuxiang Guo, Piscataway, NJ: IEEE Press, 2013, p. 896-901, article id 6618034Conference paper (Refereed)
    Abstract [en]

    Fleets of commercial vehicles represent an excellent real life setting for ubiquitous knowledge discovery. There are many electronic control units onboard a modern bus or truck, with hundreds of signals being transmitted between them on the controller area network. The growing complexity of the vehicles has lead to a significant desire to have systems for fault detection, remote diagnostics and maintenance prediction. This paper aims to show that it is possible to discover useful diagnostic knowledge by a self-organized algorithm in the scenario of a fleet of city buses. The approach is demonstrated as a process consisting of two parts; Unsupervised modeling (where interesting features are discovered) and Guided search (where the previously found features are coupled to additional information sources). The modeling part searches for simple linear models in a group of vehicles, where interesting features are selected based on both non-randomness in relations and variability in the group. It is shown in an eight months long data collection study that this approach was able to discover features related to broken wheelspeed sensors. Strikingly, deviations in these features (for the vehicles with broken sensors) can be observed up to several months before a breakdown occur. This potentially allows for sufficient time to schedule the vehicle for maintenance and prepare the workshop with relevant components. © 2013 IEEE.

  • 24.
    Cabunagan-Cinco, Gerjane Joy
    et al.
    Cape Breton University, Sydney, Canada.
    Rajabi, Enayat
    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR). Cape Breton University, Sydney, Canada.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Cluster Analysis on Sustainable Transportation: The Case of New York City Open Data2022In: 2022 International Conference on Applied Artificial Intelligence (ICAPAI), IEEE, 2022Conference paper (Refereed)
    Abstract [en]

    Artificial Intelligence (AI) provides the opportunity to analyze complex transportation domains from various perspectives. Sustainability is one of the important transportation factors vital for a robust, fair, and efficient living environment and the livability of a city. This article leverages different feature engineering techniques on the New York City mobility dataset to identify the significant sustainability factors and employ the k-means clustering technique to cluster the commuters based on their transportation modes and demographics. Cluster analysis is performed based on the specified features and sustainable mode of transportation. Our cluster analysis of commuters on the New York City dataset shows that demographic information such as gender or race does not influence the sustainable mode of transportation, while the "start location"of travellers and their car access are influencing factors on sustainability. © 2022 IEEE.

  • 25.
    Calikus, Ece
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Fan, Yuantao
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Pinheiro Sant'Anna, Anita
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Interactive-cosmo: Consensus self-organized models for fault detection with expert feedback2019In: Proceedings of the Workshop on Interactive Data Mining, WIDM 2019, New York: Association for Computing Machinery (ACM), 2019, p. 1-9Conference paper (Refereed)
    Abstract [en]

    Diagnosing deviations and predicting faults is an important task, especially given recent advances related to Internet of Things. However, the majority of the efforts for diagnostics are still carried out by human experts in a time-consuming and expensive manner. One promising approach towards self-monitoring systems is based on the "wisdom of the crowd" idea, where malfunctioning equipments are detected by understanding the similarities and differences in the operation of several alike systems.

    A fully autonomous fault detection, however, is not possible, since not all deviations or anomalies correspond to faulty behaviors; many can be explained by atypical usage or varying external conditions. In this work, we propose a method which gradually incorporates expert-provided feedback for more accurate self-monitoring. Our idea is to support model adaptation while allowing human feedback to persist over changes in data distribution, such as concept drift. © 2019 Association for Computing Machinery.

    Download full text (pdf)
    fulltext
  • 26.
    Calikus, Ece
    et al.
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Bouguelia, Mohamed-Rafik
    Halmstad University, School of Information Technology.
    Dikmen, Onur
    Halmstad University, School of Information Technology.
    Wisdom of the contexts: active ensemble learning for contextual anomaly detection2022In: Data mining and knowledge discovery, ISSN 1384-5810, E-ISSN 1573-756X, Vol. 36, p. 2410-2458Article in journal (Refereed)
    Abstract [en]

    In contextual anomaly detection, an object is only considered anomalous within a specific context. Most existing methods use a single context based on a set of user-specified contextual features. However, identifying the right context can be very challenging in practice, especially in datasets with a large number of attributes. Furthermore, in real-world systems, there might be multiple anomalies that occur in different contexts and, therefore, require a combination of several "useful" contexts to unveil them. In this work, we propose a novel approach, called WisCon (Wisdom of the Contexts), to effectively detect complex contextual anomalies in situations where the true contextual and behavioral attributes are unknown. Our method constructs an ensemble of multiple contexts, with varying importance scores, based on the assumption that not all useful contexts are equally so. We estimate the importance of each context using an active learning approach with a novel query strategy. Experiments show that WisCon significantly outperforms existing baselines in different categories (i.e., active classifiers, unsupervised contextual, and non-contextual anomaly detectors) on 18 datasets. Furthermore, the results support our initial hypothesis that there is no single perfect context that successfully uncovers all kinds of contextual anomalies, and leveraging the "wisdom" of multiple contexts is necessary. © 2022, The Author(s).

  • 27.
    Calikus, Ece
    et al.
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Dikmen, Onur
    Halmstad University, School of Information Technology.
    Context Discovery for Anomaly DetectionManuscript (preprint) (Other academic)
    Abstract [en]

    Contextual anomaly detection aims at identifying objects that are anomalous only within specific contexts. Most existing methods are limited to a single context defined by user-specified features. While identifying the right context is not trivial in practice, there is often more than just one context in real-world systems under which different anomalies naturally occur. In this work, we introduce ConQuest, a new unsupervised contextual anomaly detection approach that automatically discovers and incorporates multiple contexts useful for revealing contextual anomalies. In ConQuest, we search for relevant contexts by optimizing an unsupervised multi-objective function, where each objective is derived from desired properties of contextual anomaly detection. To effectively balance such (often competing) properties, we use a multi-objective genetic algorithm that returns a Pareto front comprising diverse, non-dominated solutions. Through experiments on various datasets, we show ConQuest outperforms state-of-the-art methods. Further, we showcase the advantage of using multiple objectives over single-objective context discovery strategies and demonstrate the interpretability aspect of ConQuest.

  • 28.
    Calikus, Ece
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Pinheiro Sant'Anna, Anita
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Byttner, Stefan
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Ranking Abnormal Substations by Power Signature Dispersion2018In: Energy Procedia, ISSN 1876-6102, Vol. 149, p. 345-353Article in journal (Refereed)
    Abstract [en]

    The relation between heat demand and outdoor temperature (heat power signature) is a typical feature used to diagnose abnormal heat demand. Prior work is mainly based on setting thresholds, either statistically or manually, in order to identify outliers in the power signature. However, setting the correct threshold is a difficult task since heat demand is unique for each building. Too loose thresholds may allow outliers to go unspotted, while too tight thresholds can cause too many false alarms.

    Moreover, just the number of outliers does not reflect the dispersion level in the power signature. However, high dispersion is often caused by fault or configuration problems and should be considered while modeling abnormal heat demand.

    In this work, we present a novel method for ranking substations by measuring both dispersion and outliers in the power signature. We use robust regression to estimate a linear regression model. Observations that fall outside of the threshold in this model are considered outliers. Dispersion is measured using coefficient of determination R2 which is a statistical measure of how close the data are to the fitted regression line.

    Our method first produces two different lists by ranking substations using number of outliers and dispersion separately. Then, we merge the two lists into one using the Borda Count method. Substations appearing on the top of the list should indicate higher abnormality in heat demand compared to the ones on the bottom. We have applied our model on data from substations connected to two district heating networks in the south of Sweden. Three different approaches i.e. outlier-based, dispersion-based and aggregated methods are compared against the rankings based on return temperatures. The results show that our method significantly outperforms the state-of-the-art outlier-based method. © 2018 The Authors. Published by Elsevier Ltd.

    Download full text (pdf)
    fulltext
  • 29.
    Calikus, Ece
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Pinheiro Sant'Anna, Anita
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Dikmen, Onur
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    No free lunch but a cheaper supper: A general framework for streaming anomaly detection2020In: Expert systems with applications, ISSN 0957-4174, E-ISSN 1873-6793, Vol. 155, article id 113453Article in journal (Refereed)
    Abstract [en]

    In recent years, there has been increased research interest in detecting anomalies in temporal streaming data. A variety of algorithms have been developed in the data mining community, which can be divided into two categories (i.e., general and ad hoc). In most cases, general approaches assume the one-size-fits-all solution model where a single anomaly detector can detect all anomalies in any domain.  To date, there exists no single general method that has been shown to outperform the others across different anomaly types, use cases and datasets. On the other hand, ad hoc approaches that are designed for a specific application lack flexibility. Adapting an existing algorithm is not straightforward if the specific constraints or requirements for the existing task change. In this paper, we propose SAFARI, a general framework formulated by abstracting and unifying the fundamental tasks in streaming anomaly detection, which provides a flexible and extensible anomaly detection procedure. SAFARI helps to facilitate more elaborate algorithm comparisons by allowing us to isolate the effects of shared and unique characteristics of different algorithms on detection performance. Using SAFARI, we have implemented various anomaly detectors and identified a research gap that motivates us to propose a novel learning strategy in this work. We conducted an extensive evaluation study of 20 detectors that are composed using SAFARI and compared their performances using real-world benchmark datasets with different properties. The results indicate that there is no single superior detector that works well for every case, proving our hypothesis that "there is no free lunch" in the streaming anomaly detection world. Finally, we discuss the benefits and drawbacks of each method in-depth and draw a set of conclusions to guide future users of SAFARI.

  • 30.
    Calikus, Ece
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Pinheiro Sant'Anna, Anita
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Gadd, Henrik
    Halmstad University, School of Business, Engineering and Science, The Rydberg Laboratory for Applied Sciences (RLAS). Öresundskraft, Helsingborg, Sweden.
    Werner, Sven
    Halmstad University, School of Business, Engineering and Science, The Rydberg Laboratory for Applied Sciences (RLAS).
    A data-driven approach for discovering heat load patterns in district heating2019In: Applied Energy, ISSN 0306-2619, E-ISSN 1872-9118, Vol. 252, article id 113409Article in journal (Refereed)
    Abstract [en]

    Understanding the heat usage of customers is crucial for effective district heating operations and management. Unfortunately, existing knowledge about customers and their heat load behaviors is quite scarce. Most previous studies are limited to small-scale analyses that are not representative enough to understand the behavior of the overall network. In this work, we propose a data-driven approach that enables large-scale automatic analysis of heat load patterns in district heating networks without requiring prior knowledge. Our method clusters the customer profiles into different groups, extracts their representative patterns, and detects unusual customers whose profiles deviate significantly from the rest of their group. Using our approach, we present the first large-scale, comprehensive analysis of the heat load patterns by conducting a case study on many buildings in six different customer categories connected to two district heating networks in the south of Sweden. The 1222 buildings had a total floor space of 3.4 million square meters and used 1540 TJ heat during 2016. The results show that the proposed method has a high potential to be deployed and used in practice to analyze and understand customers’ heat-use habits. © 2019 Calikus et al. Published by Elsevier Ltd.

  • 31.
    Carpatorea, Iulian
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Rögnvaldsson, Thorsteinn
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Elmer, Marcus
    Volvo Group Trucks Technology, Advanced Technology & Research, Göteborg, Sweden.
    APPES Maps as Tools for Quantifying Performance of Truck Drivers2014In: Proceedings of the 2014 International Conference on Data Mining, DMIN'14 / [ed] Robert Stahlbock & Gary M. Weiss, USA: CSREA Press, 2014, p. 10-16Conference paper (Refereed)
    Abstract [en]

    Understanding and quantifying drivers’ influence on fuel consumption is an important and challenging problem. A number of commonly used approaches are based on collection of Accelerator Pedal Position - Engine Speed (APPES) maps. Up until now, however, most publicly available results are based on limited amounts of data collected in experiments performed under well-controlled conditions. Before APPES maps can be considered a reliable solution, there is a need to evaluate the usefulness of those models on a larger and more representative data.

    In this paper we present analysis of APPES maps that were collected, under actual operating conditions, on more than 1200 trips performed by a fleet of 5 Volvo trucks owned by a commercial transporter in Europe. We use Gaussian Mixture Models to identify areas of those maps that correspond to different types of driver behaviour, and investigate how the parameters of those models relate to variables of interest such as vehicle weight or fuel consumption.

  • 32.
    Carpatorea, Iulian
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Rögnvaldsson, Thorsteinn
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Elmer, Marcus
    Volvo Group Trucks Technology, Advanced Technology & Research, Göteborg, Sweden.
    Towards Data Driven Method for Quantifying Performance of Truck Drivers2014In: The SAIS Workshop 2014 Proceedings, Swedish Artificial Intelligence Society (SAIS) , 2014, p. 133-142Conference paper (Refereed)
    Abstract [en]

    Understanding factors that influence fuel consumption is a very important task both for the OEMs in the automotive industry and for their customers. There is a lot of knowledge already available concerning this topic, but it is poorly organized and often more anecdotal than rigorously verified. Nowadays, however, rich datasets from actual vehicle usage are available and a data-mining approach can be used to not only validate earlier hypotheses, but also to discover unexpected influencing factors.

    In this paper we particularly focus on analyzing how behavior of drivers affects fuel consumption. To this end we introduce a concept of “Base Value”, a number that incorporates many constant, unmeasured factors. We show our initial results on how it allows us to categorize driver’s performance more accurately than previously used methods. We present a detailed analysis of 32 trips by Volvo trucks that we have selected from a larger database. Those trips have a large overlap in the route traveled, of over 100 km, and at the same time exhibit different driver and fuel consumption characteristics.

    Download full text (pdf)
    fulltext
  • 33.
    Carpatorea, Iulian
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Rögnvaldsson, Thorsteinn
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Lodin, Johan
    Volvo Group Trucks Technology, Göteborg, Sweden.
    Features extracted from APPES to enable the categorization of heavy-duty vehicle drivers2017In: 2017 Intelligent Systems Conference (IntelliSys), IEEE, 2017, p. 476-481Conference paper (Refereed)
    Abstract [en]

    Improving the performance of systems is a goal pursued in all areas and vehicles are no exception. In places like Europe, where the majority of goods are transported over land, it is imperative for fleet operators to have the best efficiency, which results in efforts to improve all aspects of truck operations. We focus on drivers and their performance with respect to fuel consumption. Some of relevant factors are not accounted for inavailable naturalistic data, since it is not feasible to measure them. An alternative is to set up experiments to investigate driver performance but these are expensive and the results are not always conclusive. For example, drivers are usually aware of the experiment’s parameters and adapt their behavior.

    This paper proposes a method that addresses some of the challenges related to categorizing driver performance with respect to fuel consumption in a naturalistic environment. We use expert knowledge to transform the data and explore the resulting structure in a new space. We also show that the regions found in APPES provide useful information related to fuel consumption. The connection between APPES patterns and fuel consumption can be used to, for example, cluster drivers in groups that correspond to high or low performance. © 2017 IEEE

    Download full text (pdf)
    fulltext
  • 34.
    Carpatorea, Iulian
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Slawomir, Nowaczyk
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Rögnvaldsson, Thorsteinn
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Elmer, Marcus
    Volvo Group Trucks Technology, Göteborg, Sweden.
    Lodin, Johan
    Volvo Group Trucks Technology, Göteborg, Sweden.
    Learning of Aggregate Features for Comparing Drivers Based on Naturalistic Data2016In: Proceedings: 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA) / [ed] Lisa O’Conner, Los Alamitos, CA: IEEE Computer Society, 2016, p. 1067-1072Conference paper (Refereed)
    Abstract [en]

    Fuel used by heavy duty trucks is a major cost for logistics companies, and therefore improvements in this area are highly desired. Many of the factors that influence fuel consumption, such as the road type, vehicle configuration or external environment, are difficult to influence. One of the most under-explored ways to lower the costs is training and incentivizing drivers. However, today it is difficult to measure driver performance in a comprehensive way outside of controlled, experimental setting.

    This paper proposes a machine learning methodology for quantifying and qualifying driver performance, with respect to fuel consumption, that is suitable for naturalistic driving situations. The approach is a knowledge-based feature extraction technique, constructing a normalizing fuel consumption value denoted Fuel under Predefined Conditions (FPC), which captures the effect of factors that are relevant but are not measured directly.

    The FPC, together with information available from truck sensors, is then compared against the actual fuel used on a given road segment, quantifying the effects associated with driver behavior or other variables of interest. We show that raw fuel consumption is a biased measure of driver performance, being heavily influenced by other factors such as high load or adversary weather conditions, and that using FPC leads to more accurate results. In this paper we also show evaluation the proposed method using large-scale, real-world, naturalistic database of heavy-duty vehicle operation.

    Download full text (pdf)
    fulltext
  • 35.
    Chen, Kunru
    et al.
    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR).
    Pashami, Sepideh
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Fan, Yuantao
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Predicting Air Compressor Failures Using Long Short Term Memory Networks2019In: Progress in Artificial Intelligence: 19th EPIA Conference on Artificial Intelligence, EPIA 2019, Vila Real, Portugal, September 3–6, 2019, Proceedings, Part I / [ed] Paulo Moura Oliveira, Paulo Novais, Luís Paulo Reis, Cham: Springer, 2019, p. 596-609Conference paper (Refereed)
    Abstract [en]

    We introduce an LSTM-based method for predicting compressor failures using aggregated sensory data, and evaluate it using historical information from over 1000 heavy duty vehicles during 2015 and 2016. The goal is to proactively identify trucks that will require maintenance in the near future, so that component replacement can be scheduled before the failure happens, translating into improved uptime. The problem is formulated as a classification task of whether a compressor failure will happen within the specified prediction horizon. A recurrent neural network using Long Short-Term Memory (LSTM) architecture is employed as the prediction model, and compared against Random Forest (RF), the solution used in industrial deployment at the moment. Experimental results show that while Random Forest slightly outperforms LSTM in terms of AUC score, the predictions of LSTM stay significantly more stable over time, showing a consistent trend from healthy to faulty class. Additionally, LSTM is also better at detecting the switch from faulty class to the healthy one after a repair. We demonstrate that this stability is important for making repair decisions, especially in questionable cases, and therefore LSTM model is likely to lead to better results in practice. © Springer Nature Switzerland AG 2019

  • 36.
    Chen, Kunru
    et al.
    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR).
    Pashami, Sepideh
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Johansson, Emilia
    Toyota Material Handling Europe, Mjölby, Sweden.
    Sternelöv, Gustav
    Toyota Material Handling Europe, Mjölby, Sweden.
    Rögnvaldsson, Thorsteinn
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Forklift Truck Activity Recognition from CAN Data2021In: IoT Streams for Data-Driven Predictive Maintenance and IoT, Edge, and Mobile for Embedded Machine Learning: Second International Workshop, IoT Streams 2020, and First International Workshop, ITEM 2020, Co-located with ECML/PKDD 2020, Ghent, Belgium, September 14-18, 2020, Revised Selected Papers / [ed] Joao Gama, Sepideh Pashami, Albert Bifet, Moamar Sayed-Mouchawe, Holger Fröning, Franz Pernkopf, Gregor Schiele, Michaela Blott, Heidelberg: Springer, 2021, p. 119-126Conference paper (Refereed)
    Abstract [en]

    Machine activity recognition is important for accurately esti- mating machine productivity and machine maintenance needs. In this paper, we present ongoing work on how to recognize activities of forklift trucks from on-board data streaming on the controller area network. We show that such recognition works across different sites. We first demon- strate the baseline classification performance of a Random Forest that uses 14 signals over 20 time steps, for a 280-dimensional input. Next, we show how a deep neural network can learn low-dimensional representa- tions that, with fine-tuning, achieve comparable accuracy. The proposed representation achieves machine activity recognition. Also, it visualizes the forklift operation over time and illustrates the relationships across different activities. © Springer Nature Switzerland AG 2020

    Download full text (pdf)
    fulltext
  • 37.
    Chen, Kunru
    et al.
    Halmstad University, School of Information Technology.
    Rögnvaldsson, Thorsteinn
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Pashami, Sepideh
    Halmstad University, School of Information Technology.
    Johansson, Emilia
    Toyota Material Handling Manufacturing Sweden AB, Mjölby, Sweden.
    Sternelöv, Gustav
    Toyota Material Handling Manufacturing Sweden AB, Mjölby, Sweden.
    Semi-Supervised Learning for Forklift Activity Recognition from Controller Area Network (CAN) Signals2022In: Sensors, E-ISSN 1424-8220, Vol. 22, no 11, article id 4170Article in journal (Refereed)
    Abstract [en]

    Machine Activity Recognition (MAR) can be used to monitor manufacturing processes and find bottlenecks and potential for improvement in production. Several interesting results on MAR techniques have been produced in the last decade, but mostly on construction equipment. Forklift trucks, which are ubiquitous and highly important industrial machines, have been missing from the MAR research. This paper presents a data-driven method for forklift activity recognition that uses Controller Area Network (CAN) signals and semi-supervised learning (SSL). The SSL enables the utilization of large quantities of unlabeled operation data to build better classifiers; after a two-step post-processing, the recognition results achieve balanced accuracy of 88% for driving activities and 95% for load-handling activities on a hold-out data set. In terms of the Matthews correlation coefficient for five activity classes, the final score is 0.82, which is equal to the recognition results of two non-domain experts who use videos of the activities. A particular success is that context can be used to capture the transport of small weight loads that are not detected by the forklift’s built-in weight sensor. © 2022 by the authors. Licensee MDPI, Basel, Switzerland.

    Download full text (pdf)
    fulltext
  • 38.
    Chen, Kunru
    et al.
    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR).
    Rögnvaldsson, Thorsteinn
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR).
    Pashami, Sepideh
    Halmstad University, School of Information Technology.
    Klang, Jonas
    Toyota Material Handling Manufacturing Sweden AB, Mjölby, Sweden.
    Sternelov, Gustav
    Toyota Material Handling Manufacturing Sweden AB, Mjölby, Sweden.
    Toward Solving Domain Adaptation with Limited Source Labeled Data2023In: 2023 IEEE International Conference on Data Mining Workshops (ICDMW) / [ed] Jihe Wang; Yi He, Thang N. Dinh; Christan Grant; Meikang Qiu; Witold Pedrycz, Piscataway, NJ: IEEE Computer Society, 2023, p. 1240-1246Conference paper (Refereed)
    Abstract [en]

    The success of domain adaptation relies on high-quality labeled data from the source domain, which is a luxury setup for applied machine learning problems. This article investigates a particular challenge: the source labeled data are neither plentiful nor sufficiently representative. We studied the challenge of limited data with an industrial application, i.e., forklift truck activity recognition. The task is to develop data-driven methods to recognize forklift usage performed in different warehouses with a large scale of signals collected from the onboard sensors. The preliminary results show that using pseudo-labeled data from the source domain can significantly improve classification performance on the target domain in some tasks. As the real-world problems are much more complex than typical research settings, it is not clearly understood in what circumstance the improvement may occur. Therefore, we provided discussions regarding this phenomenon and shared several inspirations on the difficulty of understanding and debugging domain adaptation problems in practice. © 2023 IEEE.

  • 39.
    Chen, Kunru
    et al.
    Halmstad University, School of Information Technology.
    Rögnvaldsson, Thorsteinn
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Pashami, Sepideh
    Halmstad University, School of Information Technology.
    Klang, Jonas
    Toyota Material Handling Manufacturing Sweden AB, Mjölby, Sweden.
    Sternelöv, Gustav
    Toyota Material Handling Manufacturing Sweden AB, Mjölby, Sweden.
    Material handling machine activity recognition by context ensemble with gated recurrent units2023In: Engineering applications of artificial intelligence, ISSN 0952-1976, E-ISSN 1873-6769, Vol. 126, no Part C, article id 106992Article in journal (Refereed)
    Abstract [en]

    Research on machine activity recognition (MAR) is drawing more attention because MAR can provide productivity monitoring for efficiency optimization, better maintenance scheduling, product design improvement, and potential material savings. A particular challenge of MAR for human-operated machines is the overlap when transiting from one activity to another: during transitions, operators often perform two activities simultaneously, e.g., lifting the fork already while approaching a rack, so the exact time when one activity ends and another begins is uncertain. Machine learning models are often uncertain during such activity transitions, and we propose a novel ensemble-based method adapted to fuzzy transitions in a forklift MAR problem. Unlike traditional ensembles, where models in the ensemble are trained on different subsets of data, or with costs that force them to be diverse in their responses, our approach is to train a single model that predicts several activity labels, each under a different context. These individual predictions are not made by independent networks but are made using a structure that allows for sharing important features, i.e., a context ensemble. The results show that the gated recurrent unit network can provide medium or strong confident context ensembles for 95% of the cases in the test set, and the final forklift MAR result achieves accuracies of 97% for driving and 90% for load-handling activities. This study is the first to highlight the overlapping activity issue in MAR problems and to demonstrate that the recognition results can be significantly improved by designing a machine learning framework that addresses this issue. © 2023 The Author(s)

  • 40.
    Cooney, Martin
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Pashami, Sepideh
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Pinheiro Sant'Anna, Anita
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Fan, Yuantao
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Pitfalls of Affective Computing: How can the automatic visual communication of emotions lead to harm, and what can be done to mitigate such risks?2018In: WWW '18 Companion Proceedings of the The Web Conference 2018, New York, NY: ACM Publications, 2018, p. 1563-1566Conference paper (Refereed)
    Abstract [en]

    What would happen in a world where people could "see'' others' hidden emotions directly through some visualizing technology Would lies become uncommon and would we understand each other better Or to the contrary, would such forced honesty make it impossible for a society to exist The science fiction television show Black Mirror has exposed a number of darker scenarios in which such futuristic technologies, by blurring the lines of what is private and what is not, could also catalyze suffering. Thus, the current paper first turns an eye towards identifying some potential pitfalls in emotion visualization which could lead to psychological or physical harm, miscommunication, and disempowerment. Then, some countermeasures are proposed and discussed--including some level of control over what is visualized and provision of suitably rich emotional information comprising intentions--toward facilitating a future in which emotion visualization could contribute toward people's well-being. The scenarios presented here are not limited to web technologies, since one typically thinks about emotion recognition primarily in the context of direct contact. However, as interfaces develop beyond today's keyboard and monitor, more information becomes available also at a distance--for example, speech-to-text software could evolve to annotate any dictated text with a speaker's emotional state.

    Download full text (pdf)
    fulltext
  • 41.
    Dahl, Oskar
    et al.
    Halmstad University, School of Information Technology.
    Johansson, Fredrik
    Halmstad University, School of Information Technology.
    Khoshkangini, Reza
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Pashami, Sepideh
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Pihl, Claes
    Volvo Group, QandCS, Gothenburg, Sweden.
    Understanding Association Between Logged Vehicle Data and Vehicle Marketing Parameters: Using Clustering and Rule-Based Machine Learning2020In: Proceedings of the 2020 3rd International Conference on Information Management and Management Science, IMMS 2020, New York: Association for Computing Machinery (ACM), 2020, p. 13-22Conference paper (Refereed)
    Abstract [en]

    Trucks are designed, configured and marketed for various working environments. There lies a concern whether trucks are used as intended by the manufacturer, as usage may impact the longevity, efficiency and productivity of the trucks. In this paper we propose a framework that aims to extract costumers' vehicle behaviours from Logged Vehicle Data (LVD) in order to evaluate whether they align with vehicle configurations, so-called Global Transport Application (GTA) parameters. Gaussian mixture model (GMM)s are employed to cluster and classify various vehicle behaviors from the LVD. Rule-based machine learning (RBML) was applied on the clusters to examine whether vehicle behaviors follow the GTA configuration. Particularly, we propose an approach based on studying associations that is able to extract insights on whether the trucks are used as intended. Experimental results shown that while for the vast majority of the trucks' behaviors seemingly follows their GTA configuration, there are also interesting outliers that warrant further analysis. © 2020 ACM.

  • 42.
    Davari, Narjes
    et al.
    INESC TEC, Porto, Portugal.
    Pashami, Sepideh
    Halmstad University, School of Information Technology. RISE Research Institute of Sweden, Kista, Sweden.
    Veloso, Bruno
    INESC TEC, Porto, Portugal; University of Porto, Porto, Portugal; University Portucalense, Porto, Portugal.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Fan, Yuantao
    Halmstad University, School of Information Technology.
    Mota Pereira, Pedro
    Metro of Porto, Porto, Portugal.
    Ribeiro, Rita P.
    INESC TEC, Porto, Portugal; University of Porto, Porto, Portugal.
    Gama, João
    INESC TEC, Porto, Portugal; University of Porto, Porto, Portugal.
    A Fault Detection Framework Based on LSTM Autoencoder: A Case Study for Volvo Bus Data Set2022In: Advances in Intelligent Data Analysis XX: 20th International Symposium on Intelligent Data Analysis, IDA 2022 Rennes, France, April 20–22, 2022: Proceedings / [ed] Tassadit Bouadi; Elisa Fromont; Eyke Hüllermeier, Cham: Springer, 2022, p. 39-52Conference paper (Refereed)
    Abstract [en]

    This study applies a data-driven anomaly detection framework based on a Long Short-Term Memory (LSTM) autoencoder network for several subsystems of a public transport bus. The proposed framework efficiently detects abnormal data, significantly reducing the false alarm rate compared to available alternatives. Using historical repair records, we demonstrate how detection of abnormal sequences in the signals can be used for predicting equipment failures. The deviations from normal operation patterns are detected by analysing the data collected from several on-board sensors (e.g., wet tank air pressure, engine speed, engine load) installed on the bus. The performance of LSTM autoencoder (LSTM-AE) is compared against the multi-layer autoencoder (mlAE) network in the same anomaly detection framework. The experimental results show that the performance indicators of the LSTM-AE network, in terms of F1 Score, Recall, and Precision, are better than those of the mlAE network. © 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

    Download full text (pdf)
    fulltext
  • 43.
    Del Moral, Pablo
    et al.
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Pashami, Sepideh
    Halmstad University, School of Information Technology. RISE Research Institutes of Sweden, Kista, Sweden.
    Filtering Misleading Repair Log Labels to Improve Predictive Maintenance Models2022In: Proceedings of the 7th European Conference of the Prognostics and Health Management Society 2022 / [ed] Phuc Do; Gabriel Michau; Cordelia Ezhilarasu, State College, PA: PHM Society , 2022, Vol. 7 (1), p. 110-117Conference paper (Refereed)
    Abstract [en]

    One of the main challenges for predictive maintenance in real applications is the quality of the data, especially the labels. In this paper, we propose a methodology to filter out the misleading labels that harm the performance of Machine Learning models. Ideally, predictive maintenance would be based on the information of when a fault has occurred in a machine and what specific type of fault it was. Then, we could train machine learning models to identify the symptoms of such fault before it leads to a breakdown. However, in many industrial applications, this information is not available. Instead, we approximate it using a log of component replacements, usually coming from the sales or maintenance departments. The repair history provides reliable labels for fault prediction models only if the replaced component was indeed faulty, with symptoms captured by collected data, and it was going to lead to a breakdown.

    However, very often, at least for complex equipment, this assumption does not hold. Models trained using unreliable labels will then, necessarily, fail. We demonstrate that filtering misleading labels leads to improved results. Our central claim is that the same fault, happening several times, should have similar symptoms in the data; thus, we can train a model to predict them. On the contrary, replacements of the same component that do not exhibit similar symptoms will be confusing and harm the ML models. Therefore, we aim to filter the maintenance operations, keeping only those that can be used to predict each other. Suppose we can train a successful model using the data before a component replacement to predict another component replacement. In that case, those maintenance operations must be motivated by the same, or a very similar, type of fault.

    We test this approach on a real scenario using data from a fleet of sterilizers deployed in hospitals. The data includes sensor readings from the machines describing their operations and the service logs indicating the replacement of components when the manufacturing company performs the service. Since sterilizers are complex machines consisting of many components and systems interacting with each other, there is the possibility of faults happening simultaneously.

    Download full text (pdf)
    fulltext
  • 44.
    Del Moral, Pablo
    et al.
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Pashami, Sepideh
    Halmstad University, School of Information Technology.
    Hierarchical Multi-class Classification for Fault Diagnosis2021In: Proceedings of the 31st European Safety and Reliability Conference (ESREL 2021) / [ed] Bruno Castanier; Marko Cepin; David Bigaud; Christophe Berenguer, Singapore: Research Publishing Services, 2021, p. 2457-2464Conference paper (Refereed)
    Abstract [en]

    This paper formulates the problem of predictive maintenance for complex systems as a hierarchical multi-class classification task. This formulation is useful for equipment with multiple sub-systems and components performing heterogeneous tasks. Often, the data available describes the whole system's operation and is not ideal for accurate condition monitoring. In this setup, specialized predictive models analyzing one component at a time rarely perform much better than random. However, using machine learning and hierarchical approaches, we can still exploit the data to build a fault isolation system that provides measurable benefits for technicians in the field. We propose a method for creating a taxonomy of components to train hierarchical classifiers that aim to identify the faulty component. The output of this model is a structured set of predictions with different probabilities for each component. In this setup, traditional machine learning metrics fail to capture the relationship between the performance of the models and its usefulness in the field.We introduce a new metric to evaluate our approach's benefits; it measures the number of tests a technician needs to perform before pinpointing the faulty component. Using a dataset from a real-case problem coming fro the automotive industry, we demonstrate how traditional machine learning performance metrics, like accuracy, fail to capture practical benefits. Our proposed hierarchical approach succeeds in exploiting the information in the data and outperforms non-hierarchical machine learning solutions. In addition, we can identify the weakest link of our fault isolation model, allowing us to improve it efficiently.

    Download full text (pdf)
    fulltext
  • 45.
    Del Moral, Pablo
    et al.
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Pashami, Sepideh
    Halmstad University, School of Information Technology.
    Hierarchical multi-fault prognostics for complex systemsManuscript (preprint) (Other academic)
    Abstract [en]

    The field of predictive maintenance for complex machinery with multiple possible faults is an important but largely unexplored area. In general, one assumes, often implicitly, the existence of monitoring data specific enough to capture every possible fault independently from all the others.

    In this paper, we focus on the problem of predicting time-to-failure, or remaining useful life, in situations where the above assumption does not hold. Specifically, what happens when the data is not good enough to uniquely predict every fault, and, more importantly, what happens when different faults share the same symptoms on the recorded data.

    We demonstrate that prognostics approaches learning independent models for each fault are inadequate. In particular, in the presence of faults that produce similar failure patterns, they produce false alarms disproportionately often or miss the majority of failures. 

    We propose the HMP framework (Hierarchical Multi-fault Prognosis) to solve this problem by extracting a hierarchy of faults based on the similarity of the data they produce. At each node of the hierarchy, we train a regression model to predict the time-to-failure for any of the faults contained in this node. The intuition is that while it might be impossible to predict individual time-to-failure in the presence of similar faults, a model trained on aggregated data can still provide useful information. We demonstrate through experiments the validity of our approach.

    Download full text (pdf)
    fulltext
  • 46.
    Del Moral, Pablo
    et al.
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Pashami, Sepideh
    Halmstad University, School of Information Technology.
    Why Is Multiclass Classification Hard?2022In: IEEE Access, E-ISSN 2169-3536, Vol. 10, p. 80448-80462Article in journal (Refereed)
    Abstract [en]

    In classification problems, as the number of classes increases, correctly classifying a new instance into one of them is assumed to be more challenging than making the same decision in the presence of fewer classes. The essence of the problem is that using the learning algorithm on each decision boundary individually is better than using the same learning algorithm on several of them simultaneously. However, why and when it happens is still not well-understood today. This work’s main contribution is to introduce the concept of heterogeneity of decision boundaries as an explanation of this phenomenon. Based on the definition of heterogeneity of decision boundaries, we analyze and explain the differences in the performance of state of the art approaches to solve multi-class classification. We demonstrate that as the heterogeneity increases, the performances of all approaches, except one-vs-one, decrease. We show that by correctly encoding the knowledge of the heterogeneity of decision boundaries in a decomposition of the multi-class problem, we can obtain better results than state of the art decompositions. The benefits can be an increase in classification performance or a decrease in the time it takes to train and evaluate the models. We first provide intuitions and illustrate the effects of the heterogeneity of decision boundaries using synthetic datasets and a simplistic classifier. Then, we demonstrate how a real dataset exhibits these same principles, also under realistic learning algorithms. In this setting, we devise a method to quantify the heterogeneity of different decision boundaries, and use it to decompose the multi-class problem. The results show significant improvements over state-of-the-art decompositions that do not take the heterogeneity of decision boundaries into account. © 2013 IEEE.

    Download full text (pdf)
    fulltext
  • 47.
    Del Moral, Pablo
    et al.
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Sant'Anna, Anita
    Halmstad University, School of Information Technology.
    Pashami, Sepideh
    Halmstad University, School of Information Technology.
    Pitfalls of Assessing Extracted Hierarchies for Multi-Class ClassificationManuscript (preprint) (Other academic)
    Abstract [en]

    Using hierarchies of classes is one of the standard methods to solve multi-class classification problems. In the literature, selecting the right hierarchy is considered to play a key role in improving classification performance. Although different methods have been proposed, there is still a lack of understanding of what makes a hierarchy good and what makes a method to extract hierarchies perform better or worse.

    To this effect, we analyze and compare some of the most popular approaches to extracting hierarchies. We identify some common pitfalls that may lead practitioners to make misleading conclusions about their methods.To address some of these problems, we demonstrate that using random hierarchies is an appropriate benchmark to assess how the hierarchy's quality affects the classification performance.

    In particular, we show how the hierarchy's quality can become irrelevant depending on the experimental setup: when using powerful enough classifiers, the final performance is not affected by the quality of the hierarchy. We also show how comparing the effect of the hierarchies against non-hierarchical approaches might incorrectly indicate their superiority.

    Our results confirm that datasets with a high number of classes generally present complex structures in how these classes relate to each other. In these datasets, the right hierarchy can dramatically improve classification performance.

    Download full text (pdf)
    fulltext
  • 48.
    Etminani, Kobra
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Engström, Arianna Tao
    Halmstad University, School of Information Technology.
    Göransson, Carina
    Halmstad University, School of Health and Welfare, Centre of Research on Welfare, Health and Sport (CVHI).
    Sant'Anna, Anita
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    How Behavior Change Strategies are Used to Design Digital Interventions to Improve Medication Adherence and Blood Pressure Among Patients With Hypertension: Systematic Review2020In: Journal of Medical Internet Research, E-ISSN 1438-8871, Vol. 22, no 4, article id e17201Article in journal (Refereed)
    Abstract [en]

    Background: Information on how behavior change strategies have been used to design digital interventions (DIs) to improve blood pressure (BP) control or medication adherence (MA) for patients with hypertension is currently limited.

    Objective: Hypertension is a major modifiable risk factor for cardiovascular diseases and can be controlled with appropriate medication. Many interventions that target MA to improve BP are increasingly using modern digital technologies. This systematic review was conducted to discover how DIs have been designed to improve MA and BP control among patients with hypertension in the recent 10 years. Results were mapped into a matrix of change objectives using the Intervention Mapping framework to guide future development of technologies to improve MA and BP control.

    Methods: We included all the studies regarding DI development to improve MA or BP control for patients with hypertension published in PubMed from 2008 to 2018. All the DI components were mapped into a matrix of change objectives using the Intervention Mapping technique by eliciting the key determinant factors (from patient and health care team and system levels) and targeted patient behaviors.

    Results: The analysis included 54 eligible studies. The determinants were considered at two levels: patient and health care team and system. The most commonly described determinants at the patient level were lack of education, lack of self-awareness, lack of self-efficacy, and forgetfulness. Clinical inertia and an inadequate health workforce were the most commonly targeted determinants at the health care team and system level. Taking medication, interactive patient-provider communication, self-measurement, and lifestyle management were the most cited patient behaviors at both levels. Most of the DIs did not include support from peers or family members, despite its reported effectiveness and the rate of social media penetration.

    Conclusions: This review highlights the need to design a multifaceted DI that can be personalized according to patient behavior(s) that need to be changed to overcome the key determinant(s) of low adherence to medication or uncontrolled BP among patients with hypertension, considering different levels including patient and healthcare team and system involvement. © Kobra Etminani, Arianna Tao Engström, Carina Göransson, Anita Sant’Anna, Sławomir Nowaczyk.

    Download full text (pdf)
    fulltext
  • 49.
    Etminani, Kobra
    et al.
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Göransson, Carina
    Halmstad University, School of Health and Welfare, Centre of Research on Welfare, Health and Sport (CVHI).
    Galozy, Alexander
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Norell Pejner, Margaretha
    Halmstad University, School of Health and Welfare, Centre of Research on Welfare, Health and Sport (CVHI).
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.
    Improving Medication Adherence Through Adaptive Digital Interventions (iMedA) in Patients With Hypertension: Protocol for an Interrupted Time Series Study2021In: JMIR Research Protocols, E-ISSN 1929-0748, Vol. 10, no 5, article id e24494Article in journal (Refereed)
    Abstract [en]

    Background: There is a strong need to improve medication adherence (MA) for individuals with hypertension in order to reduce long-term hospitalization costs. We believe this can be achieved through an artificial intelligence agent that helps the patient in understanding key individual adherence risk factors and designing an appropriate intervention plan. The incidence of hypertension in Sweden is estimated at approximately 27%. Although blood pressure control has increased in Sweden, barely half of the treated patients achieved adequate blood pressure levels. It is a major risk factor for coronary heart disease and stroke as well as heart failure. MA is a key factor for good clinical outcomes in persons with hypertension.

    Objective: The overall aim of this study is to design, develop, test, and evaluate an adaptive digital intervention called iMedA, delivered via a mobile app to improve MA, self-care management, and blood pressure control for persons with hypertension.

    Methods: The study design is an interrupted time series. We will collect data on a daily basis, 14 days before, during 6 months of delivering digital interventions through the mobile app, and 14 days after. The effect will be analyzed using segmented regression analysis. The participants will be recruited in Region Halland, Sweden. The design of the digital interventions follows the just-in-time adaptive intervention framework. The primary (distal) outcome is MA, and the secondary outcome is blood pressure. The design of the digital intervention is developed based on a needs assessment process including a systematic review, focus group interviews, and a pilot study, before conducting the longitudinal interrupted time series study.

    Results: The focus groups of persons with hypertension have been conducted to perform the needs assessment in a Swedish context. The design and development of digital interventions are in progress, and the interventions are planned to be ready in November 2020. Then, the 2-week pilot study for usability evaluation will start, and the interrupted time series study, which we plan to start in February 2021, will follow it.

    Conclusions: We hypothesize that iMedA will improve medication adherence and self-care management. This study could illustrate how self-care management tools can be an additional (digital) treatment support to a clinical one without increasing burden on health care staff. © Kobra Etminani, Carina Göransson, Alexander Galozy, Margaretha Norell Pejner, Sławomir Nowaczyk.

  • 50.
    Fan, Yuantao
    et al.
    Halmstad University, School of Information Technology.
    Hamid, Sarmadi
    Halmstad University, School of Information Technology.
    Nowaczyk, Sławomir
    Halmstad University, School of Information Technology.
    Incorporating Physics-based Models into Data-Driven Approaches for Air Leak Detection in City Buses2023In: Machine Learning and Principles and Practice of Knowledge Discovery in Databases: International Workshops of ECML PKDD 2022, Grenoble, France, September 19–23, 2022, Proceedings, Part II / [ed] Koprinska et al., Cham: Springer, 2023, p. 438-450Conference paper (Refereed)
    Abstract [en]

    In this work-in-progress paper two types of physics-based models, for accessing elastic and non-elastic air leakage processes, were evaluated and compared with conventional statistical methods to detect air leaks in city buses, via a data-driven approach. We have access to data streamed from a pressure sensor located in the air tanks of a few city buses, during their daily operations. The air tank in these buses supplies compressed air to drive various components, e.g. air brake, suspension, doors, gearbox, etc. We fitted three physics-based models only to the leakage segments extracted from the air pressure signal and used fitted model parameters as expert features for detecting air leaks. Furthermore, statistical moments of these fitted parameters, over predetermined time intervals, were compared to conventional statistical features on raw pressure values, under a classification setting in discriminating samples before and after the repair of air leak problems. The result of this exploratory study, on six air leak cases, shows that the fitted parameters of the physics-based models are useful for discriminating samples with air leak faults from the fault-free samples, which were observed right after the repair was performed to deal with the air leak problem. The comparison based on ANOVA F-score shows that the proposed features based on fitted parameters of physics-based models outrank the conventional features. It is observed that features of a non-elastic leakage model perform the best. © 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

123 1 - 50 of 123
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf