Enhancing original LiDAR point cloud features with virtual points has gained widespread attention in multimodal information fusion. However, existing methods struggle to leverage image depth information due to the sparse nature of point clouds, hindering proper alignment with camera-derived features. We propose a novel 3D object detection method that refines virtual point clouds using a coarse-to-fine approach, incorporating a dynamic 2D Gaussian distribution for better matching and a dynamic posterior density-aware RoI network for refined feature extraction. Our method achieves an average precision (AP) of 90.02% for moderate car detection on the KITTI validation set, outperforming state-of-the-art methods. Additionally, our approach yields AP scores of 86.58% and 82.16% for moderate and hard car detection categories on the KITTI test set, respectively. These results underscore the effectiveness of our method in addressing point cloud sparsity and enhancing 3D object detection performance. The code is available at https://github.com/ZhongkangZ/LidarIG. © 2024 Elsevier B.V.

Traffic flow forecasting is a challenging task due to its spatio-temporal nature and the stochastic features underlying complex traffic situations. Currently, Graph Convolutional Network (GCN) methods are among the most successful and promising approaches. However, most GCNs methods rely on a static graph structure, which is generally unable to extract the dynamic spatio-temporal relationships of traffic data and to interpret trip patterns or motivation behind traffic flows. In this paper, we propose a novel Semantics-aware Dynamic Graph Convolutional Network (SDGCN) for traffic flow forecasting. A sparse, state-sharing, hidden Markov model is applied to capture the patterns of traffic flows from sparse trajectory data; this way, latent states, as well as transition matrices that govern the observed trajectory, can be learned. Consequently, we can build dynamic Laplacian matrices adaptively by jointly considering the trip pattern and motivation of traffic flows. Moreover, high-order Laplacian matrices can be obtained by a newly designed forward algorithm of low time complexity. GCN is then employed to exploit spatial features, and Gated Recurrent Unit (GRU) is applied to exploit temporal features. We conduct extensive experiments on three real-world traffic datasets. Experimental results demonstrate that the prediction accuracy of SDGCN outperforms existing traffic flow forecasting methods. In addition, it provides better explanations of the generative Laplace matrices, making it suitable for traffic flow forecasting in large cities and providing insight into the causes of various phenomena such as traffic congestion. The code is publicly available at https://github.com/gorgen2020/SDGCN. © 2023 IEEE.

Over the last decade, with the increasing popularity and usage of the internet of things worldwide, Internet of Medical Things (IoMT) has emerged as a key technology of the modern era. IoMT uses Artificial Intelligence, 5G, big data, edge computing, and blockchain to provide users with electronic medical services. However, it may face several security threats and attacks over an insecure public network. Therefore, to protect sensitive medical data in IoMT, it is necessary to design a secure and efficient authentication protocol. In this study, we propose a privacy-preserving authentication management protocol based on blockchain. The protocol uses a blockchain to store identities and related parameters to assist communication entities in authentication. In addition, the protocol adopts a three-factor authentication method and introduces Chebyshev chaotic map to ensure the security of user login and authentication. Formal security proof and analysis using the random oracle model and Burrows-Abadi-Needham logic show that the proposed protocol is secure. Moreover, we use informal security analysis to demonstrate that the protocol can resist various security attacks. The functional comparison shows that the protocol has high security. Through performance analysis and comparison with other protocols, the proposed protocol can increase computation overhead, communication overhead, and storage overhead by up to 39.8%, 93.6%, and 86.7%,respectively. © 2023 Elsevier Ltd

Zero-shot 3D shape understanding aims to recognize “unseen” 3D categories that are not present in training data. Recently, Contrastive Language–Image Pre-training (CLIP) has shown promising open-world performance in zero-shot 3D shape understanding tasks by information fusion among language and 3D modality. It first renders 3D objects into multiple 2D image views and then learns to understand the semantic relationships between the textual descriptions and images, enabling the model to generalize to new and unseen categories. However, existing studies in zero-shot 3D shape understanding rely on predefined rendering parameters, resulting in repetitive, redundant, and low-quality views. This limitation hinders the model's ability to fully comprehend 3D shapes and adversely impacts the text–image fusion in a shared latent space. To this end, we propose a novel approach called Differentiable rendering-based multi-view Image–Language Fusion (DILF) for zero-shot 3D shape understanding. Specifically, DILF leverages large-scale language models (LLMs) to generate textual prompts enriched with 3D semantics and designs a differentiable renderer with learnable rendering parameters to produce representative multi-view images. These rendering parameters can be iteratively updated using a text–image fusion loss, which aids in parameters’ regression, allowing the model to determine the optimal viewpoint positions for each 3D object. Then a group-view mechanism is introduced to model interdependencies across views, enabling efficient information fusion to achieve a more comprehensive 3D shape understanding. Experimental results can demonstrate that DILF outperforms state-of-the-art methods for zero-shot 3D classification while maintaining competitive performance for standard 3D classification. The code is available at https://github.com/yuzaiyang123/DILP. © 2023 The Author(s)

No system entity within a contemporary distributed cyber system can be entirely trusted. Hence, the classic centralized trust management method cannot be directly applied to it. Blockchain technology is essential to achieving decentralized trust management, its consensus mechanism is useful in addressing large-scale data sharing and data consensus challenges. Herein, an n-party quantum detectable Byzantine agreement (DBA) based on the GHZ state to realize the data consensus in a quantum blockchain is proposed, considering the threat posed by the growth of quantum information technology on the traditional blockchain. Relying on the nonlocality of the GHZ state, the proposed protocol detects the honesty of nodes by allocating the entanglement resources between different nodes. The GHZ state is notably simpler to prepare than other multi-particle entangled states, thus reducing preparation consumption and increasing practicality. When the number of network nodes increases, the proposed protocol provides better scalability and stronger practicability than the current quantum DBA. In addition, the proposed protocol has the optimal fault-tolerant found and does not rely on any other presumptions. A consensus can be reached even when there are n−2 traitors. The performance analysis confirms viability and effectiveness through exemplification. The security analysis also demonstrates that the quantum DBA protocol is unconditionally secure, effectively ensuring the security of data and realizing data consistency in the quantum blockchain. © 2023 The Authors

Few-Shot Class-Incremental Learning (FSCIL) aims to learn new classes incrementally with a limited number of samples per class. It faces issues of forgetting previously learned classes and overfitting on few-shot classes. An efficient strategy is to learn features that are discriminative in both base and incremental sessions. Current methods improve discriminability by manually designing inter-class margins based on empirical observations, which can be suboptimal. The emerging Neural Collapse (NC) theory provides a theoretically optimal inter-class margin for classification, serving as a basis for adaptively computing the margin. Yet, it is designed for closed, balanced data, not for sequential or few-shot imbalanced data. To address this gap, we propose a Meta-learning- and NC-based FSCIL method, MetaNC-FSCIL, to compute the optimal margin adaptively and maintain it at each incremental session. Specifically, we first compute the theoretically optimal margin based on the NC theory. Then we introduce a novel loss function to ensure that the loss value is minimized precisely when the inter-class margin reaches its theoretically best. Motivated by the intuition that “learn how to preserve the margin” matches the meta-learning's goal of “learn how to learn”, we embed the loss function in base-session meta-training to preserve the margin for future meta-testing sessions. Experimental results demonstrate the effectiveness of MetaNC-FSCIL, achieving superior performance on multiple datasets. The code is available at https://github.com/qihangran/metaNC-FSCIL. © 2024 The Author(s)

Three-dimensional human pose and shape estimation is to compute a full human 3D mesh given a single image. The contamination of features caused by occlusion usually degrades its performance significantly. Recent progress in this field typically addressed the occlusion problem implicitly. By contrast, in this paper, we address it explicitly using a simple yet effective de-occlusion multi-task learning network. Our key insight is that feature for mesh parameter regression should be noiseless. Thus, in the feature space, our method disentangles the occludee that represents the noiseless human feature from the occluder. Specifically, a spatial regularization and an attention mechanism are imposed in the backbone of our network to disentangle the features into different channels. Furthermore, two segmentation tasks are proposed to supervise the de-occlusion process. The final mesh model is regressed by the disentangled occlusion-aware features. Experiments on both occlusion and non-occlusion datasets are conducted, and the results prove that our method is superior to the state-of-the-art methods on two occlusion datasets, while achieving competitive performance on a non-occlusion dataset. We also demonstrate that the proposed de-occlusion strategy is the main factor to improve the robustness against occlusion. The code is available at https://github.com/qihangran/De-occlusion_MTL_HMR. © 2023

Nanorobots have been used in smart health to collect time series data such as electrocardiograms and electroencephalograms. Real-time classification of dynamic time series signals in nanorobots is a challenging task. Nanorobots in the nanoscale range require a classification algorithm with low computational complexity. First, the classification algorithm should be able to dynamically analyze time series signals and update itself to process the concept drifts (CD). Second, the classification algorithm should have the ability to handle catastrophic forgetting (CF) and classify historical data. Most importantly, the classification algorithm should be energy-efficient to use less computing power and memory to classify signals in real-time on a smart nanorobot. To solve these challenges, we design an algorithm that can Prevent Concept Drift in Online continual Learning for time series classification (PCDOL). The prototype suppression item in PCDOL can reduce the impact caused by CD. It also solves the CF problem through the replay feature. The computation per second and the memory consumed by PCDOL are only 3.572M and 1KB, respectively. The experimental results show that PCDOL is better than several state-of-the-art methods for dealing with CD and CF in energy-efficient nanorobots. © IEEE

Large deep learning models are impressive, but they struggle when real-time data is not available. Few-shot class-incremental learning (FSCIL) poses a significant challenge for deep neural networks to learn new tasks from just a few labeled samples without forgetting the previously learned ones. This setup can easily leads to catastrophic forgetting and overfitting problems, severely affecting model performance. Studying FSCIL helps overcome deep learning model limitations on data volume and acquisition time, while improving practicality and adaptability of machine learning models. This paper provides a comprehensive survey on FSCIL. Unlike previous surveys, we aim to synthesize few-shot learning and incremental learning, focusing on introducing FSCIL from two perspectives, while reviewing over 30 theoretical research studies and more than 20 applied research studies. From the theoretical perspective, we provide a novel categorization approach that divides the field into five subcategories, including traditional machine learning methods, meta learning-based methods, feature and feature space-based methods, replay-based methods, and dynamic network structure-based methods. We also evaluate the performance of recent theoretical research on benchmark datasets of FSCIL. From the application perspective, FSCIL has achieved impressive achievements in various fields of computer vision such as image classification, object detection, and image segmentation, as well as in natural language processing and graph. We summarize the important applications. Finally, we point out potential future research directions, including applications, problem setups, and theory development. Overall, this paper offers a comprehensive analysis of the latest advances in FSCIL from a methodological, performance, and application perspective. © 2023 The Author(s)

The incremental learning paradigm in machine learning has consistently been a focus of academic research. It is similar to the way in which biological systems learn, and reduces energy consumption by avoiding excessive retraining. Existing studies utilize the powerful feature extraction capabilities of pre-trained models to address incremental learning, but there remains a problem of insufficient utilization of neural network feature knowledge. To address this issue, this paper proposes a novel method called Pre-trained Model Knowledge Distillation (PMKD) which combines knowledge distillation of neural network representations and replay. This paper designs a loss function based on centered kernel alignment to transfer neural network representations knowledge from the pre-trained model to the incremental model layer-by-layer. Additionally, the use of memory buffer for Dark Experience Replay helps the model retain past knowledge better. Experiments show that PMKD achieved superior performance on various datasets and different buffer sizes. Compared to other methods, our class incremental learning accuracy reached the best performance. The open-source code is published athttps://github.com/TianSongS/PMKD-IL. © 2023 The Author(s)

Cardiac ultrasound (US) image segmentation is vital for evaluating clinical indices, but it often demands a large dataset and expert annotations, resulting in high costs for deep learning algorithms. To address this, our study presents a framework utilizing artificial intelligence generation technology to produce multi-class RGB masks for cardiac US image segmentation. The proposed approach directly performs semantic segmentation of the heart's main structures in US images from various scanning modes. Additionally, we introduce a novel learning approach based on conditional generative adversarial networks (CGAN) for cardiac US image segmentation, incorporating a conditional input and paired RGB masks. Experimental results from three cardiac US image datasets with diverse scan modes demonstrate that our approach outperforms several state-of-the-art models, showcasing improvements in five commonly used segmentation metrics, with lower noise sensitivity. Source code is available at https://github.com/energy588/US2mask. © 2024 Elsevier Ltd

Re-identification (ReID) of occluded persons is a challenging task due to the loss of information in scenes with occlusions. Most existing methods for occluded ReID use 2D-based network structures to directly extract representations from 2D RGB (red, green, and blue) images, which can result in reduced performance in occluded scenes. However, since a person is a 3D non-grid object, learning semantic representations in a 2D space can limit the ability to accurately profile an occluded person. Therefore, it is crucial to explore alternative approaches that can effectively handle occlusions and leverage the full 3D nature of a person. To tackle these challenges, in this study, we employ a 3D view-based approach that fully utilizes the geometric information of 3D objects while leveraging advancements in 2D-based networks for feature extraction. Our study is the first to introduce a 3D view-based method in the areas of holistic and occluded ReID. To implement this approach, we propose a random rendering strategy that converts 2D RGB images into 3D multi-view images. We then use a 3D Multi-View Transformation Network for ReID (MV-ReID) to group and aggregate these images into a unified feature space. Compared to 2D RGB images, multi-view images can reconstruct occluded portions of a person in 3D space, enabling a more comprehensive understanding of occluded individuals. The experiments on benchmark datasets demonstrate that the proposed method achieves state-of-the-art results on occluded ReID tasks and exhibits competitive performance on holistic ReID tasks. These results also suggest that our approach has the potential to solve occlusion problems and contribute to the field of ReID. The source code and dataset are available at https://github.com/yuzaiyang123/MV-Reid. © 2023 Elsevier B.V.

Cognitive computing endeavors to construct models that emulate brain functions, which can be explored through electroencephalography (EEG). Developing precise and robust EEG classification models is crucial for advancing cognitive computing. Despite the high accuracy of supervised EEG classification models, they are constrained by labor-intensive annotations and poor generalization. Self-supervised models address these issues but encounter difficulties in matching the accuracy of supervised learning. Three challenges persist: 1) capturing temporal dependencies in EEG; 2) adapting loss functions to describe feature similarities in self-supervised models; and 3) addressing the prevalent issue of data imbalance in EEG. This study introduces the DreamCatcher Network (DCNet), a self-supervised EEG classification framework with a two-stage training strategy. The first stage extracts robust representations through contrastive learning, and the second stage transfers the representation encoder to a supervised EEG classification task. DCNet utilizes time-series contrastive learning to autonomously construct representations that comprehensively capture temporal correlations. A novel loss function, SelfDreamCatcherLoss, is proposed to evaluate the similarities between these representations and enhance the performance of DCNet. Additionally, two data augmentation methods are integrated to alleviate class imbalances. Extensive experiments show the superiority of DCNet over the current state-of-the-art models, achieving high accuracy on both the Sleep-EDF and HAR datasets. It holds substantial promise for revolutionizing sleep disorder detection and expediting the development of advanced healthcare systems driven by cognitive computing. © Copyright 2024 IEEE