Multi-Sensor Fusion for Classifying Challenging Weather Conditions in Autonomous Driving
2025 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE credits
Student thesis
Abstract [en]
This thesis explores multimodal deep learning approaches for weathertype classification in autonomous systems by fusing LiDAR,radar, and RGB sensor data. A range of fusion strategies were implemented and evaluated across two distinct frameworks: one based on the 3D point cloud encoder PointPillars, and the other on the lightweight image-based EfficientNet-B0 architecture. Both early and mid-level fusion combinations were tested to assess the complementary value of spatial and semantic features. Experiments were conducted on the Michigan State University Four Seasons (MSU-4S) dataset, which provides synchronized data from all three modalities. Among several fusion configurations, the most effective results were achieved through early fusion of LiDAR and radar features, followed by gated mid-level fusion with RGB. The PointPillars-based model attained the highest performance with 87.77% test accuracy and a macro-averaged F1 score of 0.870, while the EfficientNet-B0-based model achieved comparable accuracy(86.77%) and F1 score (0.8452) with significantly lower computational cost and faster inference.These results highlight the effectiveness of multimodal gated fusion and underscore the trade-offs between spatial precision and computational efficiency, offering valuable insights for real-time weather classification in autonomous perception systems.
Place, publisher, year, edition, pages
2025. , p. 110
Keywords [en]
Multi-sensor fusion, Early fusion, gated fusion, EfficientNet, PointPillars, LiDAR, Radar, RGB, Bird’s Eye View, Autonomous vehicles, Weather classification
National Category
Computer Vision and Learning Systems
Identifiers
URN: urn:nbn:se:hh:diva-57333OAI: oai:DiVA.org:hh-57333DiVA, id: diva2:1997440
Subject / course
Computer science and engineering
Educational program
Computer Science and Engineering, 300 credits
Presentation
2025-06-04, F506, Halmstad, 13:00 (English)
Supervisors
Examiners
2025-09-122025-09-122025-10-01Bibliographically approved