收藏切换
Skeleton-Based Human Action Recognition with Fusion of Motion Domain Knowledge and Adaptive Spatio-Temporal Transformer
收藏切换
Chengwu Liang1, 2, Songqi Jiang1, 2, Jie Yang1, 2, 3, Peiwang Zhu1, 2, Yun Tie5, Lei Gao6, Wei Hu1, 2, 4, Wenbo Guo1
Journal of Computer Research and Development | 2025, 62(12) : 3028 - 3041
Less
收藏切换
Journal of Computer Research and Development | 2025, 62(12): 3028-3041
• Artificial Intelligence •
Skeleton-Based Human Action Recognition with Fusion of Motion Domain Knowledge and Adaptive Spatio-Temporal Transformer
Full
Chengwu Liang1, 2, Songqi Jiang1, 2, Jie Yang1, 2, 3, Peiwang Zhu1, 2, Yun Tie5, Lei Gao6, Wei Hu1, 2, 4, Wenbo Guo1
Affiliations
  • 1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041
  • 2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002
  • 3State Grid Hunan Electric Power Co., Ltd. Huaihua Power Supply Branch, Huaihua, Hunan 418006
  • 4State Grid Anhui Electric Power Co., Ltd. Tongling Power Supply Branch, Tongling, Anhui 244151
  • 5School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001
  • 6Department of Electrical, Computer and Biomedical Engineering, Metropolitan University of Toronto, Toronto, Canada M5B2K3
  • born in 1982. PhD, professor, master supervisor. Member of CCF. His main research interests include computer vision, machine learning, and multimedia processing

    born in 1999. Master candidate. His main research interests include target detection and human action recognition

    born in 1998. Master. His main research interests include computer vision and multi-modality information fusion

    born in 2000. Master candidate. His main research interest includes computer vision

    born in 1974. PhD, professor, PhD supervisor. His main research interests include intelligent multimedia, and virtual reality and interaction

    born in 1983. PhD, associate professor. His main research interests include computer vision and multimedia processing

    born in 1999. Master. His main research interest includes smart grid

    born in 1994. PhD, lecturer. His main research interests include computer vision and image processing

About Author:

born in 1982. PhD, professor, master supervisor. Member of CCF. His main research interests include computer vision, machine learning, and multimedia processing

born in 1999. Master candidate. His main research interests include target detection and human action recognition

born in 1998. Master. His main research interests include computer vision and multi-modality information fusion

born in 2000. Master candidate. His main research interest includes computer vision

born in 1974. PhD, professor, PhD supervisor. His main research interests include intelligent multimedia, and virtual reality and interaction

born in 1983. PhD, associate professor. His main research interests include computer vision and multimedia processing

born in 1999. Master. His main research interest includes smart grid

born in 1994. PhD, lecturer. His main research interests include computer vision and image processing

Published: 2025-12-01 doi: 10.7544/issn1000-1239.202440558
Outline
收藏切换

Existing methods for skeleton-based human action recognition always ignore motion domain knowledge, resulting in the lack of interpretability of logical decision-making that human can understand. In this paper, we propose a novel skeleton-based human action recognition method by fusion of domain knowledge and adaptive spatio-temporal Transformer, to improve recognition performance and interpretability. Firstly, inspired by the short-term motion knowledge, a temporal multi-branch structure is designed to learn and capture the characteristics of short-term sub-acitons. Secondly, a dynamic information fusion module is proposed to learn the weight vectors of different temporal branches, and then fuse multiscale short-term motion features. Finally, to learn the relationship between different sub-actions and facilitate the motion information interaction between skeleton joints, a multiscale temporal convolution feature fusion module is proposed to capture the long-term motion correlations, by integrating the domain knowledge of the long-term motion. Experimental evaluations are conducted on four large action datasets, including NTU RGB+D, NTU RGB+D 120, FineGym, and InHARD. The experimental results show that the recognition performance of the proposed method is superior to several data-driven methods, effectively improving the modelling ability of short-term motion feature learning and information interaction between skeleton joints, with the interpretability.

skeleton-based action recognition  /  domain knowledge  /  temporal segment  /  sub-aciton  /  Transformer  /  self-attention mechanism
梁成武, 蒋松琪, 杨杰, 朱培旺, 帖云, 高磊, 胡伟, 郭文博. 融合运动领域知识与自适应时空Transformer的人体骨架行为识别. 计算机研究与发展, 2025 , 62 (12) : 3028 -3041 . DOI: 10.7544/issn1000-1239.202440558
Chengwu Liang, Songqi Jiang, Jie Yang, Peiwang Zhu, Yun Tie, Lei Gao, Wei Hu, Wenbo Guo. Skeleton-Based Human Action Recognition with Fusion of Motion Domain Knowledge and Adaptive Spatio-Temporal Transformer[J]. Journal of Computer Research and Development, 2025 , 62 (12) : 3028 -3041 . DOI: 10.7544/issn1000-1239.202440558
Year 2025 volume 62 Issue 12
138
62
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202440558
  • Receive Date:2024-06-21
  • Online Date:2026-11-10
  • Published:2025-12-01
Article Data
Affiliations
History
  • Received:2024-06-21
  • Revised:2025-04-22
  • Accepted:2025-06-11
Affiliations
    1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041
    2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002
    3State Grid Hunan Electric Power Co., Ltd. Huaihua Power Supply Branch, Huaihua, Hunan 418006
    4State Grid Anhui Electric Power Co., Ltd. Tongling Power Supply Branch, Tongling, Anhui 244151
    5School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001
    6Department of Electrical, Computer and Biomedical Engineering, Metropolitan University of Toronto, Toronto, Canada M5B2K3
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202440558
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT