To address the perception inaccuracies of autonomous mining trucks in open-pit mines under low-visibility and low-illumination conditions—issues that may lead to obstacle collisions. This paper was proposed a multimodal fusion-based obstacle detection method to enhance detection accuracy and operational safety in complex environments. Firstly, local Feature matching at light speed (LightGlue), was employed to achieve spatial alignment between thermal infrared and visible light images, thereby avoiding spatial misalignment and geometric distortion prior to fusion. Secondly, in the modality feature extraction and fusion stage, a Dual-Modality Feature Fusion (DMFF) module was incorporated into the improved dual-branch backbone network. The extraction capability of dual-modality features was enhanced and fusion was performed through feature compression and cross-modal feature enhancement. An iterative learning method was then introduced to effectively match the complementary information between modalities, generating a fused dual-modality feature map and improving multimodal detection performance. Finally, the fused feature maps at multiple scales were input into the detection head. They were combined with bounding box regression and classification prediction for precise detection. Experimental results demonstrate that the proposed method achieves excellent obstacle detection performance in challenging scenarios with low visibility. Specifically, it achieves a mean Average Precision (mAP@0.5) of 90.8%, and an F1-score of 0.887, outperforming existing methods in both accuracy and speed. Moreover, the proposed approach exhibits lower false positive and miss detection rates, effectively ensuring the safe navigation of autonomous mining trucks in complex operational environments.
| 科 Family | 属数 Number of genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) | 属 Genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) |
|---|---|---|---|---|---|---|
| 鹅膏菌科Amanitaceae | 2 | 11 | 5.26 | 鹅膏菌属 Amanita | 10 | 4.78 |
| 小菇科 Mycenaceae | 2 | 12 | 5.74 | 丝盖伞属 Inocybe | 5 | 2.39 |
| 多孔菌科 Polyporaceae | 8 | 14 | 6.70 | 蜡蘑属 Laccaria | 5 | 2.39 |
| 红菇科 Russulaceae | 3 | 23 | 11.00 | 小皮伞属 Marasmius | 6 | 2.87 |
| 小菇属 Mycena | 11 | 5.26 | ||||
| 光柄菇属 Pluteus | 5 | 2.39 | ||||
| 红菇属 Russula | 17 | 8.13 | ||||
| 栓菌属 Trametes | 5 | 2.39 |