Chinese Traditional and Herbal Drugs
|
2025, 56(4): 1310-1317
Bimodal feature fusion algorithm of Tibetan medicinal materials recognition based on hypergraph neural network
Full
CHEN Jianfu, SUN Yan, PEI Jiuchang
Affiliations
doi: 10.7501/j.issn.0253-2670.2025.04.019
Outline
Objective To identify Tibetan medicinal materials accurately and realize the intelligent mining and management of Tibetan medicinal materials. Methods HerbiFusionNet, a hypergraph-based dual-modal feature fusion model for Tibetan medicinal plants recognition, was proposed. Firstly, the improved ResNet152-CA model was used to extract the spatial features of Tibetan medicinal materials images, and the Transformer architecture based BERT model was used to extract the semantic features of Tibetan medicinal materials texts to realize the complementarity and fusion of the two modal features. Then, the similarity of the fused feature vectors was calculated to construct a hypergraph network. Finally, the complex association relationship of Tibetan medicinal plants was captured by hypergraph neural network, and the accurate classification of Tibetan medicinal materials was obtained. Results The experimental results show that, compared with the single-modal ResNet-152-CA model, the HerbiFusionNe model based on hypergraph neural network introduced by fusing dual-modal features has an accuracy rate of 96.28%, which is increased by 4.40%. The HerbiFusionNet model proposed in this study demonstrates the effectiveness of using hypergraph structure to mine complex relationships in Tibetan medicinal materials data by fusing dual-modal features of image and text. Conclusion The HerbiFusionNet model improves the accuracy of Tibetan medicinal materials recognition, can effectively capture the high-order relationship between Tibetan medicinal materials images and texts, and shows the advantages of hypergraph neural network in dealing with the complex data structure of Tibetan medicinal plants. It lays a standardized foundation for further exploration of the “symptoms-prescription-medicinal materials” relationship and safe use, and promotes the development of Tibetan medicine research and application.
Tibetan medicinal materials
/
hypergraph
/
bimodal feature fusion
/
convolutional neural networks
/
HerbiFusionNet model
陈健福, 孙燕, 裴九场.
基于超图神经网络的双模态特征融合藏药材植株识别算法.
中草药,
2025
, 56
(4)
: 1310
-1317
.
DOI: 10.7501/j.issn.0253-2670.2025.04.019
CHEN Jianfu, SUN Yan, PEI Jiuchang.
Bimodal feature fusion algorithm of Tibetan medicinal materials recognition based on hypergraph neural network[J].
Chinese Traditional and Herbal Drugs,
2025
, 56
(4)
: 1310
-1317
.
DOI: 10.7501/j.issn.0253-2670.2025.04.019
Year 2025 volume 56 Issue 4
38
9
Cite this Article
BibTeX
Article Info
doi: 10.7501/j.issn.0253-2670.2025.04.019
- Receive Date:2024-10-02
- Online Date:2026-11-13