Graph neural networks (GNNs), especially dynamic GNNs, have become a research hotspot in spatiotemporal forecasting problems. While many dynamic graph construction methods have been developed, relatively few of them explore the causal relationship between neighbor nodes. Thus, the resulting models lack strong explainability for the causal relationship between the neighbor nodes of the dynamically generated graphs, which can easily lead to a risk in subsequent decisions. Moreover, few of them consider the uncertainty and noise of dynamic graphs based on the time series datasets, which are ubiquitous in real-world graph structure networks. In this article, we propose a novel dynamic diffusion-variational GNN (DVGNN) for spatiotemporal forecasting. For dynamic graph construction, an unsupervised generative model is devised. Two layers of graph convolutional network (GCN) are applied to calculate the posterior distribution of the latent node embeddings in the encoder stage. Then, a diffusion model is used to infer the dynamic link probability and reconstruct causal graphs (CGs) in the decoder stage adaptively. The new loss function is derived theoretically, and the reparameterization trick is adopted in estimating the probability distribution of the dynamic graphs by evidence lower bound (ELBO) during the backpropagation period. After obtaining the generated graphs, dynamic GCN and temporal attention are applied to predict future states. Experiments are conducted on four real-world datasets of different graph structures in different domains. The results demonstrate that the proposed DVGNN model outperforms state-of-the-art approaches and achieves outstanding root mean square error (RMSE) results while exhibiting higher robustness. Also, by F1-score and probability distribution analysis, we demonstrate that DVGNN better reflects the causal relationship and uncertainty of dynamic graphs. The website of the code is https://github.com/gorgen2020/DVGNN.

Objective: This study aims to comprehensively review the use of graph neural networks (GNNs) for clinical risk prediction based on electronic health records (EHRs). The primary goal is to provide an overview of the state-of-the-art of this subject, highlighting ongoing research efforts and identifying existing challenges in developing effective GNNs for improved prediction of clinical risks. Methods: A search was conducted in the Scopus, PubMed, ACM Digital Library, and Embase databases to identify relevant English-language papers that used GNNs for clinical risk prediction based on EHR data. The study includes original research papers published between January 2009 and May 2023. Results: Following the initial screening process, 50 articles were included in the data collection. A significant increase in publications from 2020 was observed, with most selected papers focusing on diagnosis prediction (n = 36). The study revealed that the graph attention network (GAT) (n = 19) was the most prevalent architecture, and MIMIC-III (n = 23) was the most common data resource. Conclusion: GNNs are relevant tools for predicting clinical risk by accounting for the relational aspects among medical events and entities and managing large volumes of EHR data. Future studies in this area may address challenges such as EHR data heterogeneity, multimodality, and model interpretability, aiming to develop more holistic GNN models that can produce more accurate predictions, be effectively implemented in clinical settings, and ultimately improve patient care. © 2024 The Authors

In recent years, the concept of lifelong learning has been emphasized in relation to higher education, with a bearing idea of the possibility for the individual for a continuous, self-motivated pursuit of gaining knowledge for both personal and professional reasons, provided by higher education institutions (HEI:s). But how can this actually be done in practice? In this paper we present an ongoing project called MAISTR, which is a collaboration between Swedish HEI:s and industry with the aim of providing a number of flexible courses within the subjects of Artificial intelligence (AI), Design, and Innovation management, for professionals. Our aim is to describe how the project is setup to create new learning opportunities, including the development process and co-creation with industry, the core structure and the pedagogical design. Furthermore, we would like to discuss both challenges and opportunities that come with this kind of project, as well as reflecting on early stage outcomes. © 2023, ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering.

Purpose: The purpose of this study is to develop and validate a 3D deep learning model that predicts the final clinical diagnosis of Alzheimer's disease (AD), dementia with Lewy bodies (DLB), mild cognitive impairment due to Alzheimer's disease (MCI-AD), and cognitively normal (CN) using fluorine 18 fluorodeoxyglucose PET (18F-FDG PET) and compare model's performance to that of multiple expert nuclear medicine physicians' readers.

Materials and methods: Retrospective 18F-FDG PET scans for AD, MCI-AD, and CN were collected from Alzheimer's disease neuroimaging initiative (556 patients from 2005 to 2020), and CN and DLB cases were from European DLB Consortium (201 patients from 2005 to 2018). The introduced 3D convolutional neural network was trained using 90% of the data and externally tested using 10% as well as comparison to human readers on the same independent test set. The model's performance was analyzed with sensitivity, specificity, precision, F1 score, receiver operating characteristic (ROC). The regional metabolic changes driving classification were visualized using uniform manifold approximation and projection (UMAP) and network attention.

Results: The proposed model achieved area under the ROC curve of 96.2% (95% confidence interval: 90.6-100) on predicting the final diagnosis of DLB in the independent test set, 96.4% (92.7-100) in AD, 71.4% (51.6-91.2) in MCI-AD, and 94.7% (90-99.5) in CN, which in ROC space outperformed human readers performance. The network attention depicted the posterior cingulate cortex is important for each neurodegenerative disease, and the UMAP visualization of the extracted features by the proposed model demonstrates the reality of development of the given disorders.

Conclusion: Using only 18F-FDG PET of the brain, a 3D deep learning model could predict the final diagnosis of the most common neurodegenerative disorders which achieved a competitive performance compared to the human readers as well as their consensus. © 2021. The Author(s).

The monitoring infrastructure of an industrial fleet can rely on the so-called unit-level and subfleet-level models to observe the behavior of a target unit. However, such infrastructure has to confront several challenges. First, from an anomaly detection perspective of monitoring a target unit, unit-level and subfleet-level models can give different information about the nature of an anomaly, and which approach or level model is appropriate is not always clear. Second, in the absence of well-understood prior models of unit and subfleet behavior, the choice of a base model at their respective levels, especially in an online/streaming setting, may not be clear. Third, managing false alarms is a major problem. To deal with these challenges, we proposed to rely on the conformal anomaly detection framework. In addition, an ensemble approach was deployed to mitigate the knowledge gap in understanding the underlying data-generating process at the unit and subfleet levels. Therefore, to monitor the behavior of a target unit, a unit-level ensemble model (ULEM) and a subfleet-level ensemble model (SLEM) were constructed, where each member of the respective ensemble is based on a conformal anomaly detector (CAD). However, since the information obtained by these two ensemble models through their p-values may not always agree, a combined ensemble model (CEM) was proposed. The results are based on real-world operational data obtained from district heating (DH) substations. Here, it was observed that CEM reduces the overall false alarms compared to ULEM or SLEM, albeit at the cost of some detection delay. The analysis demonstrated the advantages and limitations of ULEM, SLEM, and CEM. Furthermore, discords obtained from the state-of-the-art matrix-profile (MP) method and the combined calibration scores obtained from ULEM and SLEM were compared in an offline setting. Here, it was observed that SLEM achieved a better overall precision and detection delay. Finally, the different components related to ULEM, SLEM, and CEM were put together into what we refer to as TRANTOR: a conformal anomaly detection based industrial fleet monitoring framework. The proposed framework is expected to enable fleet operators in various domains to improve their monitoring infrastructure by efficiently detecting anomalous behavior and controlling false alarms at the target units. © 2022

Background: In recent years, neuroimaging with deep learning (DL) algorithms have made remarkable advances in the diagnosis of neurodegenerative disorders. However, applying DL in different medical domains is usually challenged by lack of labeled data. To address this challenge, transfer learning (TL) has been applied to use state-of-the-art convolution neural networks pre-trained on natural images. Yet, there are differences in characteristics between medical and natural images, also image classification and targeted medical diagnosis tasks. The purpose of this study is to investigate the performance of specialized and TL in the classification of neurodegenerative disorders using 3D volumes of 18F-FDG-PET brain scans. Results: Results show that TL models are suboptimal for classification of neurodegenerative disorders, especially when the objective is to separate more than two disorders. Additionally, specialized CNN model provides better interpretations of predicted diagnosis. Conclusions: TL can indeed lead to superior performance on binary classification in timely and data efficient manner, yet for detecting more than a single disorder, TL models do not perform well. Additionally, custom 3D model performs comparably to TL models for binary classification, and interestingly perform better for diagnosis of multiple disorders. The results confirm the superiority of the custom 3D-CNN in providing better explainable model compared to TL adopted ones. © 2022, The Author(s).

We considered the case of monitoring a large fleet where heterogeneity in the operational behavior among its constituent units (i.e., systems or machines) is non-negligible, and no labeled data is available. Each unit in the fleet, referred to as a target, is tracked by its sub-fleet. A conformal sub-fleet (CSF) is a set of units that act as a proxy for the normal operational behavior of a target unit by relying on the Mondrian conformal anomaly detection framework. Two approaches, the k-nearest neighbors and conformal clustering, were investigated for constructing such a sub-fleet by formulating a stability criterion. Moreover, it is important to discover the sub-sequence of events that describes an anomalous behavior in a target unit. Hence, we proposed to extract such sub-sequences for further investigation without pre-specifying their length. We refer to it as a conformal anomaly sequence (CAS). Furthermore, different nonconformity measures were evaluated for their efficiency, i.e., their ability to detect anomalous behavior in a target unit, based on the length of the observed CAS and the S-criterion value. The CSF approach was evaluated in the context of monitoring district heating substations. Anomalous behavior sub-sequences were corroborated with the domain expert leading to the conclusion that the proposed approach has the potential to be useful for both diagnostic and knowledge extraction purposes, especially in domains where labeled data is not available or hard to obtain. © 2021

Convolutional Neural Networks (CNNs) have shown their effectiveness in a variety of imaging applications including medical imaging diagnostics. However, these deep learning models are data-hungry and need enough labeled samples for the training phase which is limited in the medical domain. Transfer learning is one possible solution to this challenge with training a new model. Assessing model performance should be done not only based on criteria like accuracy, and area under the ROC curve, but also it is important to investigate what regions were of most interest for the classification decisions, especially for medical applications. We performed a case study on neurodegenerative disorders, in specific Alzheimer’s disease, mild cognitive im- pairment, dementia with lewy bodies and cognitively normal brains using 3D 18F-FDG-PET brain scans. Two transfer learning models, InceptionV3 and ResNet50, as well as a custom 3D-CNN that is trained from scratch are compared. Two XAI methods, occlusion and Grad-CAM are chosen to visualize the important brain regions using correctly classified cases. We found that the TL models learn significantly different decision surfaces than the 3D-CNN model. The 3D spatial structure of the brain regions are better kept in the 3D-CNN model, and that might explain the higher performance of this model over 2D-TL models. Moreover, we found out the two XAI methods provide different results, where occlusion method focused more on specific brain regions.

A typical district heating (DH) network consists of hundreds, sometimes thousands, of substations. In the absence of a well-understood prior model or data labels about each substation, the overall monitoring of such large number of substations can be challenging. To overcome the challenge, an approach based on the collective operational monitoring of each substation by a local group (i.e., the reference-group) of other similar substations in the network was formulated. Herein, if a substation of interest (i.e., the target) starts to behave differently in comparison to those in its reference-group, then it was designated as an outlier. The approach was demonstrated on the monitoring of the return temperature variable for atypical and faulty operational behavior in 778 substations associated with multi-dwelling buildings. The choice of an appropriate similarity measure along with its size k were the two important factors that enables a reference-group to effectively detect an outlier target. Thus, different similarity measures and size k for the construction of the reference-groups were investigated, which led to the selection of the Euclidean distance with k = 80. This setup resulted in the detection of 77 target substations that were outliers, i.e., the behavior of their return temperature changed in comparison to the majority of those in their respective reference-groups. Of these, 44 were detected due to the local construction of the reference-groups. In addition, six frequent patterns of deviating behavior in the return temperature of the substations were identified using the reference-group based approach, which were then further corroborated by the feedback from a DH domain expert. © 2020 Elsevier Ltd

The ever increasing complexity of modern systems and equipment make the task of monitoring their health quite challenging. Traditional methods such as expert defined thresholds, physics based models and process history based techniques have certain drawbacks. Thresholds defined by experts require deep knowledge about the system and are often too conservative. Physics driven approaches are costly to develop and maintain. Finally, process history based models require large amount of data that may not be available at design time of a system. Moreover, the focus of these traditional approaches has been system specific. Hence, when industrial systems are deployed on a large scale, their monitoring becomes a new challenge. Under these conditions, this paper demonstrates the use of a group-based selfmonitoring approach that learns over time from similar systems subject to similar conditions. The approach is based on conformal anomaly detection coupled with an exchangeability test that uses martingales. This allows setting a threshold value based on sound theoretical justification. A hypothesis test based on this threshold is used to decide on if a system has deviated from its group. We demonstrate the feasibility of this approach through a real case study of monitoring a group of heat-pumps where it can detect a faulty hot-water switch-valve and a broken outdoor temperature sensor without previously observing these faults.