Full
Outline
To address the issues of noise sensitivity, ambiguous intent, and missing directional information in single-modality voice interaction for in-vehicle scenarios, the paper proposes a multimodal Point-of-Interest (POI) intelligent recognition method that fuses speech, vision, and gesture. The method establishes a four-layer system architecture comprising data collection, preprocessing, core algorithm, and interaction output. For speech intent recognition, Low-Rank Adaptation (LoRA) fine-tuned Qwen-7B is employed to generate structured intent output. For spatiotemporal localization, a fan-shaped region-based multi-strategy retrieval mechanism is designed, which integrates speech directional words with gaze direction, gesture pointing or navigation route, reducing the candidate POI scale by 74.47%. For comprehensive prediction, Qwen-VL is utilized to achieve cross-modal attention fusion for POI ranking. Experimental results on a self-built in-vehicle multimodal dataset demonstrate that the proposed method achieves an accuracy of 85.4% and an F1-score of 84.3%, improving by 8.6 and 7.7 percentage points respectively over the best baseline. Under 0 dB SNR conditions, the accuracy remains at 70.1%, confirming the robust effectiveness of multimodal fusion in vehicular noise environments.
48
9
| 科 Family | 属数 Number of genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) | 属 Genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) |
|---|---|---|---|---|---|---|
| 鹅膏菌科Amanitaceae | 2 | 11 | 5.26 | 鹅膏菌属 Amanita | 10 | 4.78 |
| 小菇科 Mycenaceae | 2 | 12 | 5.74 | 丝盖伞属 Inocybe | 5 | 2.39 |
| 多孔菌科 Polyporaceae | 8 | 14 | 6.70 | 蜡蘑属 Laccaria | 5 | 2.39 |
| 红菇科 Russulaceae | 3 | 23 | 11.00 | 小皮伞属 Marasmius | 6 | 2.87 |
| 小菇属 Mycena | 11 | 5.26 | ||||
| 光柄菇属 Pluteus | 5 | 2.39 | ||||
| 红菇属 Russula | 17 | 8.13 | ||||
| 栓菌属 Trametes | 5 | 2.39 |
关闭全屏
No. 86 Xueyuan South Road, Haidian District, Beijing
010-62662699(Operation and Technical Support)
qkjq@cast.org.cn
Copyright © 2025 China Association for Science and Technology. All rights reserved. For all open access content, the relevant licensing terms apply.
Sponsored by the Office of the Leading Group for Cybersecurity and Informatization of CAST, and supported by Science and Technology Review Publishing House
