PDF
Full
Outline
Existing methods for audio-visual cross-modal association learning often adopt a dual-stream network structure, but they still face challenges in reducing computational complexity, model light weighting, and efficient feature fusion. To improve model performance and enhance the efficiency of cross-modal learning, a single-stream network-based approach for audio-visual cross-modal learning was proposed. Firstly, preprocessed data from both modalities were fed into a single-stream feature extraction network, where a class-information-based loss function was employed to learn and extract feature vectors from both modalities. Subsequently, attention-based feature fusion was performed on the extracted feature vectors from both modalities. Finally, a combination of cosine similarity algorithm and cross-entropy loss was used to learn the association between the two modalities, thus completing the cross-modal association learning task. Experimental results demonstrate that the proposed method achieves promising performance in audio-visual cross-modal verification, matching, and retrieval tasks, ensuring excellent performance while considering the lightness and flexibility of the network structure.
PDF
297
113
| 科 Family | 属数 Number of genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) | 属 Genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) |
|---|---|---|---|---|---|---|
| 鹅膏菌科Amanitaceae | 2 | 11 | 5.26 | 鹅膏菌属 Amanita | 10 | 4.78 |
| 小菇科 Mycenaceae | 2 | 12 | 5.74 | 丝盖伞属 Inocybe | 5 | 2.39 |
| 多孔菌科 Polyporaceae | 8 | 14 | 6.70 | 蜡蘑属 Laccaria | 5 | 2.39 |
| 红菇科 Russulaceae | 3 | 23 | 11.00 | 小皮伞属 Marasmius | 6 | 2.87 |
| 小菇属 Mycena | 11 | 5.26 | ||||
| 光柄菇属 Pluteus | 5 | 2.39 | ||||
| 红菇属 Russula | 17 | 8.13 | ||||
| 栓菌属 Trametes | 5 | 2.39 |
关闭全屏
No. 86 Xueyuan South Road, Haidian District, Beijing
010-62199257
qkjq@cast.org.cn
Copyright © 2025 China Association for Science and Technology. All rights reserved. For all open access content, the relevant licensing terms apply.
Sponsored by the Office of the Leading Group for Cybersecurity and Informatization of CAST, and supported by Science and Technology Review Publishing House
