Hyperspectral Imaging-Based Perception in Autonomous Driving Scenarios: Benchmarking Baseline Semantic Segmentation Models

Hyperspectral Imaging (HSI), known for its advantages over traditional RGB imaging in remote sensing, agriculture, and medicine, has recently gained attention for enhancing Advanced Driving Assistance Systems (ADAS) perception. A few HSI datasets such as HyKo, HSI-Drive, HSI-Road, and Hyperspectral City have been made available. However, a comprehensive evaluation of semantic segmentation models (SSM) using these datasets is lacking. To address this gap, we evaluated the available annotated HSI datasets on four deep learning-based baseline SSMs i.e. DeepLab v3+, HRNet, PSPNet, and U-Net along with its two variants: Coordinate Attention (UNet-CA) and Convolutional Block-Attention Module (UNet-CBAM). The original models' architectures were adapted to handle the varying spatial and spectral dimensions of the datasets. These baseline SSMs were trained using class-weighted loss function for individual HSI datasets and were evaluated over mean-based metrics i.e. intersection over union (IoU), recall, precision, F1 score, specificity, and accuracy. Our results indicate that UNet-CBAM which extracts channel-wise feature extraction, outperforms other SSMs and shows the potential to leverage spectral information for enhanced semantic segmentation. This study establishes a baseline SSM-based benchmark on available annotated datasets for future evaluation of HSI-based ADAS perception. However, the limitations of current HSI datasets, such as limited dataset size, high class imbalance, and lack of fine-grained annotations, remain significant constraints for developing robust SSMs for ADAS applications.