Article(id=1327017541538526154, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327017337032651028, articleNumber=null, orderNo=null, doi=10.7544/issn1000-1239.202440558, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=research-article, receivedDate=1718899200000, receivedDateStr=2024-06-21, revisedDate=1745251200000, revisedDateStr=2025-04-22, acceptedDate=1749571200000, acceptedDateStr=2025-06-11, onlineDate=1794315236695, onlineDateStr=2026-11-10, pubDate=1764518400000, pubDateStr=2025-12-01, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1794315236695, onlineIssueDateStr=2026-11-10, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1794315236695, creator=13701087609, updateTime=1794315236695, updator=13701087609, issue=Issue{id=1327017337032651028, tenantId=1146029695717560320, journalId=1324759914146529478, year='2025', volume='62', issue='12', pageStart='2889', pageEnd='3146', issueExtLink='null', onlineDate='null', pubDate='1764518400000', pubDateStr='2025-12-01', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1794315187938, creator='13701087609', updateTime=1794377025235, updator='13701087609', preIssue=null, nextIssue=null, articleTotal=null, ext={EN=IssueExt(id=1327276701995872766, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327017337032651028, language=EN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=), CN=IssueExt(id=1327276701995872767, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327017337032651028, language=CN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=)}, issueFiles=null, downloadFileDto=null}, startPage=3028, endPage=3041, ext={EN=ArticleExt(id=1327017541765018571, articleId=1327017541538526154, tenantId=1146029695717560320, journalId=1324759914146529478, language=EN, title=Skeleton-Based Human Action Recognition with Fusion of Motion Domain Knowledge and Adaptive Spatio-Temporal Transformer, columnId=1325489891789910943, journalTitle=Journal of Computer Research and Development, columnName=Artificial Intelligence, runingTitle=null, highlight=null, articleAbstract=

Existing methods for skeleton-based human action recognition always ignore motion domain knowledge, resulting in the lack of interpretability of logical decision-making that human can understand. In this paper, we propose a novel skeleton-based human action recognition method by fusion of domain knowledge and adaptive spatio-temporal Transformer, to improve recognition performance and interpretability. Firstly, inspired by the short-term motion knowledge, a temporal multi-branch structure is designed to learn and capture the characteristics of short-term sub-acitons. Secondly, a dynamic information fusion module is proposed to learn the weight vectors of different temporal branches, and then fuse multiscale short-term motion features. Finally, to learn the relationship between different sub-actions and facilitate the motion information interaction between skeleton joints, a multiscale temporal convolution feature fusion module is proposed to capture the long-term motion correlations, by integrating the domain knowledge of the long-term motion. Experimental evaluations are conducted on four large action datasets, including NTU RGB+D, NTU RGB+D 120, FineGym, and InHARD. The experimental results show that the recognition performance of the proposed method is superior to several data-driven methods, effectively improving the modelling ability of short-term motion feature learning and information interaction between skeleton joints, with the interpretability.

, authors=Chengwu Liang1, 2, Songqi Jiang1, 2, Jie Yang1, 2, 3, Peiwang Zhu1, 2, Yun Tie5, Lei Gao6, Wei Hu1, 2, 4, Wenbo Guo1, authorsList=Chengwu Liang, Songqi Jiang, Jie Yang, Peiwang Zhu, Yun Tie, Lei Gao, Wei Hu, Wenbo Guo, authorCompany=null, correspAuthors=null, authorNote=

born in 1982. PhD, professor, master supervisor. Member of CCF. His main research interests include computer vision, machine learning, and multimedia processing

born in 1999. Master candidate. His main research interests include target detection and human action recognition

born in 1998. Master. His main research interests include computer vision and multi-modality information fusion

born in 2000. Master candidate. His main research interest includes computer vision

born in 1974. PhD, professor, PhD supervisor. His main research interests include intelligent multimedia, and virtual reality and interaction

born in 1983. PhD, associate professor. His main research interests include computer vision and multimedia processing

born in 1999. Master. His main research interest includes smart grid

born in 1994. PhD, lecturer. His main research interests include computer vision and image processing

, correspAuthorsNote=null, copyrightStatement=Copyright © 2025 Journal of Computer Research and Development. All rights reserved., copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, fund=null), CN=ArticleExt(id=1327017544763945966, articleId=1327017541538526154, tenantId=1146029695717560320, journalId=1324759914146529478, language=CN, title=融合运动领域知识与自适应时空Transformer的人体骨架行为识别, columnId=1325489892104483745, journalTitle=计算机研究与发展, columnName=人工智能, runingTitle=null, highlight=null, articleAbstract=

现有人体骨架行为识别方法往往忽略运动学领域知识,造成模型内在人类可理解的逻辑决策解释性不足. 基于此,提出一种融合领域知识与自适应时空片段Transformer的骨架行为识别方法,以提高骨架行为识别模型的性能和可解释性. 首先,受短时运动领域知识启发,设计多时间分支结构用于学习和捕捉多时间尺度的短时子动作特征. 其次,提出一种动态信息融合模块,学习不同时间分支的权重向量进而动态融合多时间分支、多尺度短时运动特征. 最后,融合长时运动领域知识,提出多尺度时间卷积特征融合模块捕捉长时运动关联,用于学习不同子动作片段之间的关系并促进不同骨架关节点间的运动信息交互. 在4个数据集上进行评估与实验,包括人体日常行为数据集NTU RGB+D和NTU RGB+D 120、体育行为数据集FineGym,以及工业场景行为数据集InHARD. 结果表明所提方法的行为识别性能,优于包括基准Transformer方法在内的多个先进方法,可有效提升骨架序列短时运动特征学习和关节点之间信息交互的建模能力,并具有一定可解释性.

, authors=梁成武1, 2, 蒋松琪1, 2, 杨杰1, 2, 3, 朱培旺1, 2, 帖云5, 高磊6, 胡伟1, 2, 4, 郭文博1, authorsList=梁成武, 蒋松琪, 杨杰, 朱培旺, 帖云, 高磊, 胡伟, 郭文博, authorCompany=null, correspAuthors=null, authorNote=

1982年生. 博士,教授,硕士生导师,CCF会员. 主要研究方向为计算机视觉、机器学习、多媒体处理

1999年生. 硕士研究生. 主要研究方向为目标检测、人体动作识别

1998年生. 硕士. 主要研究方向为计算机视觉、多模态信息融合

2000年生. 硕士研究生. 主要研究方向为计算机视觉

1974年生. 博士,教授,博士生导师. 主要研究方向为智能多媒体、虚拟现实及交互

1983年生. 博士,副教授. 主要研究方向为计算机视觉、多媒体处理

1999年生. 硕士. 主要研究方向为智能电网

1994年生. 博士,讲师. 主要研究方向为计算机视觉、图像处理

, correspAuthorsNote=null, copyrightStatement=版权所有 © 2025 计算机研究与发展编辑部, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=ESCveA1Vwg414zQ5Ow8wQQ==, magXml=1XZoAKmStQwKUGGQ/2RUFw==, pdfUrl=null, pdf=oeqdJIkYxZvnDaaUmoXweQ==, pdfFileSize=3472322, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=bcSwVTni7Ib2OSsvkDCrqQ==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=a8h4UGKIo0AfaZI15spVPg==, mapNumber=null, fund=null)}, authors=[Author(id=1327017545636360193, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=liangchengwu0615@126.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017545741217796, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017545636360193, language=EN, stringName=Chengwu Liang, firstName=Chengwu, middleName=null, lastName=Liang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041
2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002, bio={"content":"

born in 1982. PhD, professor, master supervisor. Member of CCF. His main research interests include computer vision, machine learning, and multimedia processing

"}, bioImg=null, bioContent=

born in 1982. PhD, professor, master supervisor. Member of CCF. His main research interests include computer vision, machine learning, and multimedia processing

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017545825103877, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017545636360193, language=CN, stringName=梁成武, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1河南城建学院电气与控制工程学院 河南平顶山 467041
2三峡大学电气与新能源学院 湖北宜昌 443002, bio={"img":"70+g9qsK40VgrcoR0ZnweQ==","content":"

1982年生. 博士,教授,硕士生导师,CCF会员. 主要研究方向为计算机视觉、机器学习、多媒体处理

"}, bioImg=70+g9qsK40VgrcoR0ZnweQ==, bioContent=

1982年生. 博士,教授,硕士生导师,CCF会员. 主要研究方向为计算机视觉、机器学习、多媒体处理

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017544990438383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=1, ext=[AuthorCompanyExt(id=1327017545003021296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041), AuthorCompanyExt(id=1327017545011409905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1河南城建学院电气与控制工程学院 河南平顶山 467041)]), AuthorCompany(id=1327017545099490290, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=2, ext=[AuthorCompanyExt(id=1327017545107878899, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002), AuthorCompanyExt(id=1327017545116267508, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2三峡大学电气与新能源学院 湖北宜昌 443002)])]), Author(id=1327017545908989959, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017546030624778, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017545908989959, language=EN, stringName=Songqi Jiang, firstName=Songqi, middleName=null, lastName=Jiang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041
2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002, bio={"content":"

born in 1999. Master candidate. His main research interests include target detection and human action recognition

"}, bioImg=null, bioContent=

born in 1999. Master candidate. His main research interests include target detection and human action recognition

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017546101927947, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017545908989959, language=CN, stringName=蒋松琪, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1河南城建学院电气与控制工程学院 河南平顶山 467041
2三峡大学电气与新能源学院 湖北宜昌 443002, bio={"img":"gBi8qMFemoD5ZJJ+HBp0Bw==","content":"

1999年生. 硕士研究生. 主要研究方向为目标检测、人体动作识别

"}, bioImg=gBi8qMFemoD5ZJJ+HBp0Bw==, bioContent=

1999年生. 硕士研究生. 主要研究方向为目标检测、人体动作识别

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017544990438383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=1, ext=[AuthorCompanyExt(id=1327017545003021296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041), AuthorCompanyExt(id=1327017545011409905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1河南城建学院电气与控制工程学院 河南平顶山 467041)]), AuthorCompany(id=1327017545099490290, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=2, ext=[AuthorCompanyExt(id=1327017545107878899, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002), AuthorCompanyExt(id=1327017545116267508, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2三峡大学电气与新能源学院 湖北宜昌 443002)])]), Author(id=1327017546177425421, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017546265505809, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017546177425421, language=EN, stringName=Jie Yang, firstName=Jie, middleName=null, lastName=Yang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 3, address=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041
2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002
3State Grid Hunan Electric Power Co., Ltd. Huaihua Power Supply Branch, Huaihua, Hunan 418006, bio={"content":"

born in 1998. Master. His main research interests include computer vision and multi-modality information fusion

"}, bioImg=null, bioContent=

born in 1998. Master. His main research interests include computer vision and multi-modality information fusion

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017546332614674, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017546177425421, language=CN, stringName=杨杰, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 3, address=1河南城建学院电气与控制工程学院 河南平顶山 467041
2三峡大学电气与新能源学院 湖北宜昌 443002
3国网湖南省电力有限公司怀化供电分公司 湖南怀化 418006, bio={"img":"kvByoCjYoOGdxmocGWksHA==","content":"

1998年生. 硕士. 主要研究方向为计算机视觉、多模态信息融合

"}, bioImg=kvByoCjYoOGdxmocGWksHA==, bioContent=

1998年生. 硕士. 主要研究方向为计算机视觉、多模态信息融合

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017544990438383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=1, ext=[AuthorCompanyExt(id=1327017545003021296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041), AuthorCompanyExt(id=1327017545011409905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1河南城建学院电气与控制工程学院 河南平顶山 467041)]), AuthorCompany(id=1327017545099490290, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=2, ext=[AuthorCompanyExt(id=1327017545107878899, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002), AuthorCompanyExt(id=1327017545116267508, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2三峡大学电气与新能源学院 湖北宜昌 443002)]), AuthorCompany(id=1327017545195959285, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=3, ext=[AuthorCompanyExt(id=1327017545212736502, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545195959285, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3State Grid Hunan Electric Power Co., Ltd. Huaihua Power Supply Branch, Huaihua, Hunan 418006), AuthorCompanyExt(id=1327017545221125111, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545195959285, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3国网湖南省电力有限公司怀化供电分公司 湖南怀化 418006)])]), Author(id=1327017546399723540, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017546487803927, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017546399723540, language=EN, stringName=Peiwang Zhu, firstName=Peiwang, middleName=null, lastName=Zhu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041
2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002, bio={"content":"

born in 2000. Master candidate. His main research interest includes computer vision

"}, bioImg=null, bioContent=

born in 2000. Master candidate. His main research interest includes computer vision

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017546601050136, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017546399723540, language=CN, stringName=朱培旺, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1河南城建学院电气与控制工程学院 河南平顶山 467041
2三峡大学电气与新能源学院 湖北宜昌 443002, bio={"img":"B+K2/8TziFnt+PqvPbbZNQ==","content":"

2000年生. 硕士研究生. 主要研究方向为计算机视觉

"}, bioImg=B+K2/8TziFnt+PqvPbbZNQ==, bioContent=

2000年生. 硕士研究生. 主要研究方向为计算机视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017544990438383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=1, ext=[AuthorCompanyExt(id=1327017545003021296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041), AuthorCompanyExt(id=1327017545011409905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1河南城建学院电气与控制工程学院 河南平顶山 467041)]), AuthorCompany(id=1327017545099490290, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=2, ext=[AuthorCompanyExt(id=1327017545107878899, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002), AuthorCompanyExt(id=1327017545116267508, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2三峡大学电气与新能源学院 湖北宜昌 443002)])]), Author(id=1327017546668159002, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017546865291292, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017546668159002, language=EN, stringName=Yun Tie, firstName=Yun, middleName=null, lastName=Tie, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=5, address=5School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001, bio={"content":"

born in 1974. PhD, professor, PhD supervisor. His main research interests include intelligent multimedia, and virtual reality and interaction

"}, bioImg=null, bioContent=

born in 1974. PhD, professor, PhD supervisor. His main research interests include intelligent multimedia, and virtual reality and interaction

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017546953371677, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017546668159002, language=CN, stringName=帖云, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=5, address=5郑州大学电气与信息工程学院 郑州 450001, bio={"img":"NWsL6rgINqQnGzElnsLY7w==","content":"

1974年生. 博士,教授,博士生导师. 主要研究方向为智能多媒体、虚拟现实及交互

"}, bioImg=NWsL6rgINqQnGzElnsLY7w==, bioContent=

1974年生. 博士,教授,博士生导师. 主要研究方向为智能多媒体、虚拟现实及交互

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017545376314363, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=5, ext=[AuthorCompanyExt(id=1327017545388897276, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545376314363, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=5School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001), AuthorCompanyExt(id=1327017545422451709, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545376314363, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=5郑州大学电气与信息工程学院 郑州 450001)])]), Author(id=1327017547054034975, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017547150503969, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017547054034975, language=EN, stringName=Lei Gao, firstName=Lei, middleName=null, lastName=Gao, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=6, address=6Department of Electrical, Computer and Biomedical Engineering, Metropolitan University of Toronto, Toronto, Canada M5B2K3, bio={"content":"

born in 1983. PhD, associate professor. His main research interests include computer vision and multimedia processing

"}, bioImg=null, bioContent=

born in 1983. PhD, associate professor. His main research interests include computer vision and multimedia processing

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017547246972962, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017547054034975, language=CN, stringName=高磊, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=6, address=6多伦多都会大学电气计算机和生物医学工程系 加拿大多伦多 M5B2K3, bio={"img":"hY5tYxfAg4kyShey4NxHhw==","content":"

1983年生. 博士,副教授. 主要研究方向为计算机视觉、多媒体处理

"}, bioImg=hY5tYxfAg4kyShey4NxHhw==, bioContent=

1983年生. 博士,副教授. 主要研究方向为计算机视觉、多媒体处理

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017545535697918, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=6, ext=[AuthorCompanyExt(id=1327017545548280831, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545535697918, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=6Department of Electrical, Computer and Biomedical Engineering, Metropolitan University of Toronto, Toronto, Canada M5B2K3), AuthorCompanyExt(id=1327017545556669440, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545535697918, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=6多伦多都会大学电气计算机和生物医学工程系 加拿大多伦多 M5B2K3)])]), Author(id=1327017547322470436, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=6, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017547435716648, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017547322470436, language=EN, stringName=Wei Hu, firstName=Wei, middleName=null, lastName=Hu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 4, address=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041
2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002
4State Grid Anhui Electric Power Co., Ltd. Tongling Power Supply Branch, Tongling, Anhui 244151, bio={"content":"

born in 1999. Master. His main research interest includes smart grid

"}, bioImg=null, bioContent=

born in 1999. Master. His main research interest includes smart grid

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017547523797033, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017547322470436, language=CN, stringName=胡伟, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 4, address=1河南城建学院电气与控制工程学院 河南平顶山 467041
2三峡大学电气与新能源学院 湖北宜昌 443002
4国网安徽省电力有限公司铜陵供电分公司 安徽铜陵 244151, bio={"img":"zI+GUbNEDdJUYVUNVgjdvg==","content":"

1999年生. 硕士. 主要研究方向为智能电网

"}, bioImg=zI+GUbNEDdJUYVUNVgjdvg==, bioContent=

1999年生. 硕士. 主要研究方向为智能电网

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017544990438383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=1, ext=[AuthorCompanyExt(id=1327017545003021296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041), AuthorCompanyExt(id=1327017545011409905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1河南城建学院电气与控制工程学院 河南平顶山 467041)]), AuthorCompany(id=1327017545099490290, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=2, ext=[AuthorCompanyExt(id=1327017545107878899, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002), AuthorCompanyExt(id=1327017545116267508, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2三峡大学电气与新能源学院 湖北宜昌 443002)]), AuthorCompany(id=1327017545288233976, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=4, ext=[AuthorCompanyExt(id=1327017545296622585, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545288233976, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4State Grid Anhui Electric Power Co., Ltd. Tongling Power Supply Branch, Tongling, Anhui 244151), AuthorCompanyExt(id=1327017545309205498, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545288233976, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4国网安徽省电力有限公司铜陵供电分公司 安徽铜陵 244151)])]), Author(id=1327017547607683115, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=7, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017547725123629, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017547607683115, language=EN, stringName=Wenbo Guo, firstName=Wenbo, middleName=null, lastName=Guo, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041, bio={"content":"

born in 1994. PhD, lecturer. His main research interests include computer vision and image processing

"}, bioImg=null, bioContent=

born in 1994. PhD, lecturer. His main research interests include computer vision and image processing

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017547800621102, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017547607683115, language=CN, stringName=郭文博, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1河南城建学院电气与控制工程学院 河南平顶山 467041, bio={"img":"WATKphBeMbHlFn9/3Hh/zQ==","content":"

1994年生. 博士,讲师. 主要研究方向为计算机视觉、图像处理

"}, bioImg=WATKphBeMbHlFn9/3Hh/zQ==, bioContent=

1994年生. 博士,讲师. 主要研究方向为计算机视觉、图像处理

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017544990438383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=1, ext=[AuthorCompanyExt(id=1327017545003021296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041), AuthorCompanyExt(id=1327017545011409905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1河南城建学院电气与控制工程学院 河南平顶山 467041)])])], keywords=[Keyword(id=1327017547918061615, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, orderNo=1, keyword=skeleton-based action recognition), Keyword(id=1327017547985170480, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, orderNo=2, keyword=domain knowledge), Keyword(id=1327017548048085041, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, orderNo=3, keyword=temporal segment), Keyword(id=1327017548127776818, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, orderNo=4, keyword=sub-aciton), Keyword(id=1327017548199079987, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, orderNo=5, keyword=Transformer), Keyword(id=1327017548257800244, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, orderNo=6, keyword=self-attention mechanism), Keyword(id=1327017548324909109, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, orderNo=1, keyword=骨架行为识别), Keyword(id=1327017548392017974, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, orderNo=2, keyword=领域知识), Keyword(id=1327017548584955959, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, orderNo=3, keyword=时间片段), Keyword(id=1327017548689813561, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, orderNo=4, keyword=子动作), Keyword(id=1327017548811448378, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, orderNo=5, keyword=Transformer), Keyword(id=1327017548886945851, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, orderNo=6, keyword=自注意力机制)], refs=[Reference(id=1327017551894261852, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=0, authorNames=null, journalName=null, refType=null, unstructuredReference=裴利沈, 刘少博, 赵雪专. 人体行为识别研究综述[J]. 计算机科学与探索, 2022, 16(2): 305−322, articleTitle=null, refAbstract=null), Reference(id=1327017551961370717, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=1, authorNames=null, journalName=null, refType=null, unstructuredReference=Pei Lishen, Liu Shaobo, Zhao Xuezhuan. Review of human behavior recognition research[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(2): 305−322 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327017552049451102, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=2, rfOrder=2, authorNames=null, journalName=null, refType=null, unstructuredReference=李洪均, 丁宇鹏, 李超波, 等. 基于特征融合时序分割网络的行为识别研究[J]. 计算机研究与发展, 2020, 57(1): 145−158, articleTitle=null, refAbstract=null), Reference(id=1327017552124948575, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=2, rfOrder=3, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Hongjun, Ding Yupeng, Li Chaobo, et al. Action recognition of temporal segment network based on feature fusion[J]. Journal of Computer Research and Development, 2020, 57(1): 145−158 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327017552200446048, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=4, authorNames=null, journalName=null, refType=null, unstructuredReference=卢健, 李萱峰, 赵博, 等. 骨骼信息的人体行为识别综述[J]. 中国图象图形学报, 2023, 28(12): 3651−3669, articleTitle=null, refAbstract=null), Reference(id=1327017552263360609, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=5, authorNames=null, journalName=null, refType=null, unstructuredReference=Lu Jian, Li Xuanfeng, Zhao Bo, et al. A review of skeleton-based human action recognition[J]. Journal of Image and Graphics, 2023, 28(12): 3651−3669 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327017552334663778, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=6, authorNames=null, journalName=null, refType=null, unstructuredReference=Yan Sijie, Xiong Yuanjun, Lin Dahua. Spatial temporal graph convolutional networks for skeleton-based action recognition[C]//Proc of the 32nd AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2018: 7444−7452, articleTitle=null, refAbstract=null), Reference(id=1327017552410161251, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=7, authorNames=null, journalName=null, refType=null, unstructuredReference=Shi Lei, Zhang Yifan, Cheng Jian, et al. Two-stream adaptive graph convolutional networks for skeleton-based action recognition[C]//Proc of the 28th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2019: 12026−12035, articleTitle=null, refAbstract=null), Reference(id=1327017552494047332, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=6, rfOrder=8, authorNames=null, journalName=null, refType=null, unstructuredReference=Cheng Ke, Zhang Yifan, He Xiangyu, et al. Skeleton-based action recognition with shift graph convolutional network[C]//Proc of the 29th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 183−192, articleTitle=null, refAbstract=null), Reference(id=1327017552577933413, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=7, rfOrder=9, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Yuxin, Zhang Ziqi, Yuan Chunfeng, et al. Channel-wise topology refinement graph convolution for skeleton-based action recognition[C]//Proc of the 18th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2021: 13359−13368, articleTitle=null, refAbstract=null), Reference(id=1327017552661819494, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=10, authorNames=null, journalName=null, refType=null, unstructuredReference=曹毅, 吴伟官, 张小勇, 等. 基于自校准机制的时空采样图卷积行为识别模型[J]. 工程科学学报, 2024, 46(3): 480−490, articleTitle=null, refAbstract=null), Reference(id=1327017552733122663, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=11, authorNames=null, journalName=null, refType=null, unstructuredReference=Cao Yi, Wu Weiguan, Zhang Xiaoyong, et al. Action recognition model based on the spatiotemporal sampling graph convolutional network and self-calibration mechanism[J]. Chinese Journal of Engineering, 2024, 46(3): 480−490 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327017552808620136, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=12, authorNames=null, journalName=null, refType=null, unstructuredReference=Plizzari C, Cannici M, Matteucci M. Skeleton-based action recognition via spatial and temporal transformer networks[J]. Computer Vision and Image Understanding, 2021, 208/209: 103219−103228, articleTitle=null, refAbstract=null), Reference(id=1327017552892506217, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=10, rfOrder=13, authorNames=null, journalName=null, refType=null, unstructuredReference=Shi Lei, Zhang Yifan, Cheng Jian, et al. Decoupled spatial-temporal attention network for skeleton-based action-gesture recognition[C]//Proc of the 15th Asian Conf on Computer Vision. Berlin: Springer, 2020: 38−53, articleTitle=null, refAbstract=null), Reference(id=1327017552963809386, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=11, rfOrder=14, authorNames=null, journalName=null, refType=null, unstructuredReference=石跃祥, 朱茂清. 基于骨架动作识别的协作卷积Transformer网络[J]. 电子与信息学报, 2019, 45(4): 1485−1493, articleTitle=null, refAbstract=null), Reference(id=1327017553026723947, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=11, rfOrder=15, authorNames=null, journalName=null, refType=null, unstructuredReference=Shi Yuexiang, Zhu Maoqing. Collaborative convolutional Transformer network based on skeleton action recognition[J]. Journal of Electronics and Information Technology, 2019, 45(4): 1485−1493 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327017553106415724, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=12, rfOrder=16, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Shiwen, Guo Sheng, Wang Liming, et al. Knowledge integration networks for action recognition[C]//Proc of the 34th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2020: 12862−12869, articleTitle=null, refAbstract=null), Reference(id=1327017553194496109, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=17, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Liming, Tong Zhan, Ji Bin, et al. TDN: Temporal difference networks for efficient action recognition[C]//Proc of the 30th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2021: 1895−1904, articleTitle=null, refAbstract=null), Reference(id=1327017553269993582, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=18, authorNames=null, journalName=null, refType=null, unstructuredReference=梁成武, 胡伟, 杨杰, 等. 融合时空领域知识与数据驱动的骨架行为识别[J]. 计算机工程与应用, 2025, 61(5): 165−176, articleTitle=null, refAbstract=null), Reference(id=1327017553337102447, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=19, authorNames=null, journalName=null, refType=null, unstructuredReference=Liang Chengwu, Hu Wei, Yang Jie, et al. Integration of spatiotemporal domain knowledge and data-driven skeleton behavior recognition[J]. Computer Engineering and Applications, 2025, 61(5): 165−176 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327017553412599920, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=15, rfOrder=20, authorNames=null, journalName=null, refType=null, unstructuredReference=Qiu Helei, Hou Biao, Ren Bo, et al. Spatio-temporal segments attention for skeleton-based action recognition[J]. Neurocomputing, 2023, 518: 30−38, articleTitle=null, refAbstract=null), Reference(id=1327017553479708785, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=21, authorNames=null, journalName=null, refType=null, unstructuredReference=Duan Haodong, Wang Jiaqi, Chen Kai, et al. DG-STGCN: Dynamic spatial-temporal modeling for skeleton-based action recognition[J]. arXiv preprint, arXiv: 2210.05895, 2022, articleTitle=null, refAbstract=null), Reference(id=1327017553551011954, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=17, rfOrder=22, authorNames=null, journalName=null, refType=null, unstructuredReference=Shahroudy A, Liu Jun, Ng T T, et al. NTU RGB+D: A large scale dataset for 3D human activity analysis[C]//Proc of the 25th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2016: 1010−1019, articleTitle=null, refAbstract=null), Reference(id=1327017553639092339, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=18, rfOrder=23, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Jun, Shahroudy A, Perez M, et al. NTU RGB+D 120: A large-scale benchmark for 3D human activity understanding[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 42(10): 2684−2701, articleTitle=null, refAbstract=null), Reference(id=1327017553731367028, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=19, rfOrder=24, authorNames=null, journalName=null, refType=null, unstructuredReference=Shao Dian, Zhao Yue, Dai Bo, et al. FineGym: A hierarchical video dataset for fine-grained action understanding[C]//Proc of the 29th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 2616−2625, articleTitle=null, refAbstract=null), Reference(id=1327017553811058805, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=20, rfOrder=25, authorNames=null, journalName=null, refType=null, unstructuredReference=Dallel M, Havard V, Baudry D, et al. Inhard-industrial human action recognition dataset in the context of industrial collaborative robotics[C]//Proc of the 1st IEEE Int Conf on Human-Machine Systems (ICHMS). Piscataway, NJ: IEEE, 2020: 1−6, articleTitle=null, refAbstract=null), Reference(id=1327017553865584758, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=21, rfOrder=26, authorNames=null, journalName=null, refType=null, unstructuredReference=Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[J] arXiv preprint, arXiv: 2010.11929, 2021, articleTitle=null, refAbstract=null), Reference(id=1327017553924305015, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=22, rfOrder=27, authorNames=null, journalName=null, refType=null, unstructuredReference=Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proc of the 31st Int Conf on Neural Information Processing Systems. Long Beach, CA: Curran Associates Inc, 2017: 6000−6010, articleTitle=null, refAbstract=null), Reference(id=1327017554012385400, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=23, rfOrder=28, authorNames=null, journalName=null, refType=null, unstructuredReference=Bertasius G, Wang Heng, Torresani L. Is space-time attention all you need for video understanding?[C/OL]//Proc of the 38th Int Conf on Machine Learning. 2021[2025-04-10]. https://proceedings.mlr.press/v139/bertasius21a.html, articleTitle=null, refAbstract=null), Reference(id=1327017554075299961, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=24, rfOrder=29, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Xiang, Wang Wenhai, Hu Xiaolin, et al. Selective kernel networks[C]//Proc of the 28th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2019: 510−519, articleTitle=null, refAbstract=null), Reference(id=1327017554138214522, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=25, rfOrder=30, authorNames=null, journalName=null, refType=null, unstructuredReference=MMAction2 Contributors. Openmmlab’s next generation video understanding toolbox and benchmark[CP/OL]. 2020[2025-04-10]. https://github.com/open-mmlab/mmaction2, articleTitle=null, refAbstract=null), Reference(id=1327017554196934779, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=31, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Maosen, Chen Siheng, Chen Xu, et al. Actional-structural graph convolutional networks for skeleton-based action recognition[C]//Proc of the 28th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2019: 3595−3603, articleTitle=null, refAbstract=null), Reference(id=1327017554259849340, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=27, rfOrder=32, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Ziyu, Zhang Hongwen, Chen Zhenghao, et al. Disentangling and unifying graph convolutions for skeleton-based action recognition[C]//Proc of the 29th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 143−152, articleTitle=null, refAbstract=null), Reference(id=1327017554326958205, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=28, rfOrder=33, authorNames=null, journalName=null, refType=null, unstructuredReference=Chi H, Ha M H, Chi S, et al. InfoGCN: Representation learning for human skeleton-based action recognition[C]//Proc of the 31st IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 20186−20196, articleTitle=null, refAbstract=null), Reference(id=1327017554410844286, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=34, authorNames=null, journalName=null, refType=null, unstructuredReference=曹毅, 夏宇, 高清源, 等. 基于超连接图卷积网络的骨架行为识别方法[J/OL]. 吉林大学学报: 工学版, 2025, [2025-04-12]. https://doi.org/10.13229/j.cnki.jdxbgxb.20230440, articleTitle=null, refAbstract=null), Reference(id=1327017554473758847, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=35, authorNames=null, journalName=null, refType=null, unstructuredReference=Cao Yi, Xia Yu, Gao Qingyuan, et al. Based on super connection diagram convolution network skeleton behavior recognition method[J/OL]. Journal of Jilin University: Engineering Science, 2025,[2025-04-12]. https://doi.org/10.13229/j.cnki.jdxbgxb.20230440 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327017554557644928, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=30, rfOrder=36, authorNames=null, journalName=null, refType=null, unstructuredReference=Jang S, Lee H, Kim W J, et al. Multi-scale structural graph convolutional network for skeleton-based action recognition[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2024, 34(8): 7244−7258, articleTitle=null, refAbstract=null), Reference(id=1327017554628948097, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=31, rfOrder=37, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhou Huanyu, Liu Qingjie, Wang Yunhong. Learning discriminative representations for skeleton based action recognition[C]//Proc of the 32nd IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023: 10608−10617, articleTitle=null, refAbstract=null), Reference(id=1327017556294086786, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=32, rfOrder=38, authorNames=null, journalName=null, refType=null, unstructuredReference=Lee J, Lee M, Lee D, et al. Hierarchically decomposed graph convolutional networks for skeleton-based action recognition[C]//Proc of the 19th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2023: 10444−10453, articleTitle=null, refAbstract=null), Reference(id=1327017556377972868, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=33, rfOrder=39, authorNames=null, journalName=null, refType=null, unstructuredReference=Myung W, Su Nan, Xue Jinhao, et al. DEGCN: Deformable graph convolutional networks for skeleton-based action recognition[J]. IEEE Transactions on Image Processing, 2024, 33: 2477−2490, articleTitle=null, refAbstract=null), Reference(id=1327017556520579205, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=34, rfOrder=40, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Haowei, Liu Yongcheng, Chen Yuxin, et al. TranSkeleton: Hierarchical spatia-temporal Transformer for skeleton-based action recognition[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(8): 4137−4148, articleTitle=null, refAbstract=null), Reference(id=1327017556675768454, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=35, rfOrder=41, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Shaojie, Yin Jianqi, Dang Yonghao, et al. SiT-MLP: A simple MLP with point-wise topology feature learning for skeleton-based action recognition[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2024, 34(9): 8122−8134, articleTitle=null, refAbstract=null), Reference(id=1327017556734488711, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=36, rfOrder=42, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Shenlan, Ding Yuning, Zhang Jingro, et al. Multidimensional refinement graph convolutional network with robust decouple loss for fine-grained skeleton-based action recognition[J]. IEEE Transactions on Neural Networks and Learning Systems, 2025, 36(4): 7615−7626, articleTitle=null, refAbstract=null), Reference(id=1327017556797403272, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=37, rfOrder=43, authorNames=null, journalName=null, refType=null, unstructuredReference=Girdhar R, Ramanan D, Gupta A, et al. ActionVLAD: Learning spatio-temporal aggregation for action classification[C]//Proc of the 26th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2017: 971−980, articleTitle=null, refAbstract=null), Reference(id=1327017556860317833, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=38, rfOrder=44, authorNames=null, journalName=null, refType=null, unstructuredReference=Carreira J, Zisserman A. Quo vadis, action recognition? A new model and the kinetics dataset[C]//Proc of the 26th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2017: 6299−6308, articleTitle=null, refAbstract=null), Reference(id=1327017556931621002, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=39, rfOrder=45, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Liming, Xiong Yuanjun, Wang Zhe, et al. Temporal segment networks: Towards good practices for deep action recognition[C]//Proc of the 11th European Conf on Computer Vision. Berlin: Springer, 2016: 20−36, articleTitle=null, refAbstract=null), Reference(id=1327017556998729867, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=40, rfOrder=46, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhou B, Andonian A, Oliva A, et al. Temporal relational reasoning in videos[C]//Proc of the 13th European Conf on Computer Vision. Berlin: Springer, 2018: 803−818, articleTitle=null, refAbstract=null), Reference(id=1327017557074227340, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=41, rfOrder=47, authorNames=null, journalName=null, refType=null, unstructuredReference=Lin Ji, Gan Chuang, Han Song. TSM: Temporal shift module for efficient video understanding[C]//Proc of the 17th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2019: 7083−7093, articleTitle=null, refAbstract=null), Reference(id=1327017557145530509, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=42, rfOrder=48, authorNames=null, journalName=null, refType=null, unstructuredReference=Kim M, Kwon H, Wang Chunyu, et al. Relational self-attention: What’s missing in attention for video under-standing[C]//Proc of the 35th Int Conf on Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc, 2021: 8046−8059, articleTitle=null, refAbstract=null), Reference(id=1327017557221027982, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=43, rfOrder=49, authorNames=null, journalName=null, refType=null, unstructuredReference=Shi Jing, Zhang Yuanyuan, Wang Weihang, et al. A novel two-stream Transformer-based framework for multi-modality human action recognition[J]. Applied Sciences, 2023, 13(4): 2058, articleTitle=null, refAbstract=null), Reference(id=1327017557288136847, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=44, rfOrder=50, authorNames=null, journalName=null, refType=null, unstructuredReference=Kim S, Ahn D, Ko B C. Cross-modal learning with 3D deformable attention for action recognition[C]//Proc of the 19th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2023: 10265−10275, articleTitle=null, refAbstract=null), Reference(id=1327017557359440016, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=45, rfOrder=51, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Xiaolong, Girshick R, Gupta A, et al. Non-local neural networks[C]//Proc of the 27th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2018: 7794−7803, articleTitle=null, refAbstract=null), Reference(id=1327017557413965969, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=46, rfOrder=52, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Ze, Ning Jia, Cao Yue, et al. Video swin Transformer[C]//Proc of the 31st IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 3202−3211, articleTitle=null, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1327017544990438383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=1, ext=[AuthorCompanyExt(id=1327017545003021296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041), AuthorCompanyExt(id=1327017545011409905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1河南城建学院电气与控制工程学院 河南平顶山 467041)]), AuthorCompany(id=1327017545099490290, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=2, ext=[AuthorCompanyExt(id=1327017545107878899, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002), AuthorCompanyExt(id=1327017545116267508, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2三峡大学电气与新能源学院 湖北宜昌 443002)]), AuthorCompany(id=1327017545195959285, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=3, ext=[AuthorCompanyExt(id=1327017545212736502, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545195959285, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3State Grid Hunan Electric Power Co., Ltd. Huaihua Power Supply Branch, Huaihua, Hunan 418006), AuthorCompanyExt(id=1327017545221125111, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545195959285, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3国网湖南省电力有限公司怀化供电分公司 湖南怀化 418006)]), AuthorCompany(id=1327017545288233976, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=4, ext=[AuthorCompanyExt(id=1327017545296622585, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545288233976, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4State Grid Anhui Electric Power Co., Ltd. Tongling Power Supply Branch, Tongling, Anhui 244151), AuthorCompanyExt(id=1327017545309205498, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545288233976, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4国网安徽省电力有限公司铜陵供电分公司 安徽铜陵 244151)]), AuthorCompany(id=1327017545376314363, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=5, ext=[AuthorCompanyExt(id=1327017545388897276, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545376314363, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=5School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001), AuthorCompanyExt(id=1327017545422451709, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545376314363, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=5郑州大学电气与信息工程学院 郑州 450001)]), AuthorCompany(id=1327017545535697918, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=6, ext=[AuthorCompanyExt(id=1327017545548280831, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545535697918, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=6Department of Electrical, Computer and Biomedical Engineering, Metropolitan University of Toronto, Toronto, Canada M5B2K3), AuthorCompanyExt(id=1327017545556669440, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545535697918, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=6多伦多都会大学电气计算机和生物医学工程系 加拿大多伦多 M5B2K3)])], figs=[ArticleFig(id=1327017549016969276, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Figure 1, caption=Fusion of motion domain knowledge and adaptive spatio-temporal Transformer model, figureFileSmall=r4qihi9Q5on9mQ1ulmy2jw==, figureFileBig=bcSwVTni7Ib2OSsvkDCrqQ==, tableContent=null), ArticleFig(id=1327017549084078141, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=图1, caption=融合运动领域知识与自适应时空Transformer模型, figureFileSmall=r4qihi9Q5on9mQ1ulmy2jw==, figureFileBig=bcSwVTni7Ib2OSsvkDCrqQ==, tableContent=null), ArticleFig(id=1327017549432205374, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Figure 2, caption=Inter-segment self-attention mechanism, figureFileSmall=sBKN3usSB+GGZCYXrdLk2w==, figureFileBig=T838JlTsjYp0XMUc1X8f8A==, tableContent=null), ArticleFig(id=1327017549528674367, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=图2, caption=段间自注意力机制, figureFileSmall=sBKN3usSB+GGZCYXrdLk2w==, figureFileBig=T838JlTsjYp0XMUc1X8f8A==, tableContent=null), ArticleFig(id=1327017549620949056, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Figure 3, caption=ATSA module structure, figureFileSmall=i3ysTHCPNIp4kNKLMXlyVQ==, figureFileBig=IJdELj1MxHoL9ZGkfPJuSQ==, tableContent=null), ArticleFig(id=1327017549713223745, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=图3, caption=ATSA模块结构, figureFileSmall=i3ysTHCPNIp4kNKLMXlyVQ==, figureFileBig=IJdELj1MxHoL9ZGkfPJuSQ==, tableContent=null), ArticleFig(id=1327017549830664258, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Figure 4, caption=MTCF module structure, figureFileSmall=bGLKarlWRBeZH5uOFnCCVw==, figureFileBig=lzpawQS2swoWV2W0usngZA==, tableContent=null), ArticleFig(id=1327017549914550339, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=图4, caption=MTCF模块结构, figureFileSmall=bGLKarlWRBeZH5uOFnCCVw==, figureFileBig=lzpawQS2swoWV2W0usngZA==, tableContent=null), ArticleFig(id=1327017549985853508, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Figure 5, caption=Example of NTU RGB+D and NTU RGB+D 120 dataset samples, figureFileSmall=sP+SlGhT8UUeQOIm7m6MBw==, figureFileBig=nM5NTwlJdwkyE2MMEHbgNQ==, tableContent=null), ArticleFig(id=1327017550111682629, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=图5, caption=NTU RGB+D和NTU RGB+D 120数据集样本示例, figureFileSmall=sP+SlGhT8UUeQOIm7m6MBw==, figureFileBig=nM5NTwlJdwkyE2MMEHbgNQ==, tableContent=null), ArticleFig(id=1327017550191374406, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Figure 6, caption=Example of FineGym dataset samples, figureFileSmall=lt6fGDHIyD+UQhPVnFKSvg==, figureFileBig=0NkDn2Kz6PzilXzVmTzRmQ==, tableContent=null), ArticleFig(id=1327017550250094663, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=图6, caption=FineGym数据集样本示例, figureFileSmall=lt6fGDHIyD+UQhPVnFKSvg==, figureFileBig=0NkDn2Kz6PzilXzVmTzRmQ==, tableContent=null), ArticleFig(id=1327017550317203528, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Figure 7, caption=Example of InHARD dataset samples, figureFileSmall=M91mY/PonZbHP/kWtVVBNg==, figureFileBig=H4zBp4IVItPFkfOR5bkRXQ==, tableContent=null), ArticleFig(id=1327017550384312393, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=图7, caption=InHARD数据集样本示例, figureFileSmall=M91mY/PonZbHP/kWtVVBNg==, figureFileBig=H4zBp4IVItPFkfOR5bkRXQ==, tableContent=null), ArticleFig(id=1327017550455615562, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Figure 8, caption=The normalized confusion matrix on NTU RGB + D 120 dataset, figureFileSmall=ktPBL2tlcl2r8rVcBMVr3Q==, figureFileBig=PEFrKqj2HhJVYDE5AxeQgw==, tableContent=null), ArticleFig(id=1327017550547890251, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=图8, caption=NTU RGB+D 120数据集上的归一化混淆矩阵, figureFileSmall=ktPBL2tlcl2r8rVcBMVr3Q==, figureFileBig=PEFrKqj2HhJVYDE5AxeQgw==, tableContent=null), ArticleFig(id=1327017550614999116, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Figure 9, caption=Visualization of output features of different ISSA branches on NTU RGB+D dataset, figureFileSmall=Wtg9cisMgc9Nu24krC7WkA==, figureFileBig=rOQ1pP2wbLReYesCfi8y4w==, tableContent=null), ArticleFig(id=1327017550682107981, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=图9, caption=NTU RGB+D数据集中不同ISSA分支输出特征的可视化, figureFileSmall=Wtg9cisMgc9Nu24krC7WkA==, figureFileBig=rOQ1pP2wbLReYesCfi8y4w==, tableContent=null), ArticleFig(id=1327017550749216846, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Table 1, caption=Ablation Experiment of ISSA Module %, figureFileSmall=null, figureFileBig=null, tableContent=
方法XsubXview
ISSA(n=1)89.093.8
ISSA(n=2)89.894.2
ISSA(n=3)90.395.0
ISSA(n=4)88.594.9
ISSA(n=5)87.694.9
ISSA(n=6)87.194.7
), ArticleFig(id=1327017550828908623, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=表1, caption=

ISSA模块的消融实验

, figureFileSmall=null, figureFileBig=null, tableContent=
方法XsubXview
ISSA(n=1)89.093.8
ISSA(n=2)89.894.2
ISSA(n=3)90.395.0
ISSA(n=4)88.594.9
ISSA(n=5)87.694.9
ISSA(n=6)87.194.7
), ArticleFig(id=1327017550900211792, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Table 2, caption=Recognition Performance Comparison of Different Information Fusion Strategies %, figureFileSmall=null, figureFileBig=null, tableContent=
方法融合策略XsubXview
ATSA
(1,2,3分支)
平均融合90.895.6
线性融合90.695.3
DIF91.095.8
), ArticleFig(id=1327017550975709265, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=表2, caption=

不同信息融合策略的识别性能对比

, figureFileSmall=null, figureFileBig=null, tableContent=
方法融合策略XsubXview
ATSA
(1,2,3分支)
平均融合90.895.6
线性融合90.695.3
DIF91.095.8
), ArticleFig(id=1327017551067983954, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Table 3, caption=Ablation Experiment of MTCF Module %, figureFileSmall=null, figureFileBig=null, tableContent=
方法XsubXview
MTC(4分支)90.294.2
MTC(6分支)90.594.9
MTCF (4分支+固定权重)90.795.4
MTCF (4分支+动态权重)90.895.4
MTCF (6分支+动态权重)91.095.8
), ArticleFig(id=1327017551156064339, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=表3, caption=

MTCF模块的消融实验

, figureFileSmall=null, figureFileBig=null, tableContent=
方法XsubXview
MTC(4分支)90.294.2
MTC(6分支)90.594.9
MTCF (4分支+固定权重)90.795.4
MTCF (4分支+动态权重)90.895.4
MTCF (6分支+动态权重)91.095.8
), ArticleFig(id=1327017551227367508, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Table 4, caption=Ablation Experiment of Multi-Modal Data %, figureFileSmall=null, figureFileBig=null, tableContent=
方法XsubXview
关节点模态91.095.8
骨骼模态91.295.2
关节点运动模态88.894.4
骨骼运动模态88.392.9
4种模态融合93.297.1
), ArticleFig(id=1327017551294476373, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=表4, caption=

多模态数据的消融实验

, figureFileSmall=null, figureFileBig=null, tableContent=
方法XsubXview
关节点模态91.095.8
骨骼模态91.295.2
关节点运动模态88.894.4
骨骼运动模态88.392.9
4种模态融合93.297.1
), ArticleFig(id=1327017551374168150, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Table 5, caption=Recognition Accuracy Comparison of Different Methods on NTU RGB+D and NTU RGB+D 120 Datasets, figureFileSmall=null, figureFileBig=null, tableContent=
类别方法参数量计算量/
GFLOPs
NTU RGB+DNTU RGB+D 120
Xsub/%Xview/%Xsub/%Xset/%
注:模型参数量、计算量参考自文献[15, 33, 35].
GCNST-GCN[4]3.1×10681.588.370.773.2
2s-AGCN[5]3.5×1063.988.595.179.281.5
Shift-GCN[6]0.7×1062.590.796.585.987.6
CTR-GCN[7]1.4×1061.892.496.888.990.6
AS-GCN[26]86.894.278.379.8
MS-G3D[27]2.8×1065.291.596.286.988.4
InfoGCN[28]1.5×1061.792.796.989.490.6
HC-GCN[29]91.296.787.689.0
MSS-GCN[30]92.796.988.990.6
FR-Head[31]2.0×10692.896.889.590.9
HD-GCN[32]1.7×1061.693.097.089.891.2
DEGCN[33]5.6×10693.697.491.092.1
TransformerST-TR[9]12.1×106259.489.996.182.784.7
DSTA[10]4.1×10664.791.596.486.689.0
STSA-Net[15]1.9×10692.796.788.590.7
TranSkeleton[34]2.2×1069.292.897.089.490.5
MLPSiT-MLP[35]0.6×1060.792.396.889.090.5
Transformer本文方法2.5×1065.293.297.189.791.4
), ArticleFig(id=1327017551453859927, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=表5, caption=

不同方法在NTU RGB+D和NTU RGB+D 120数据集的识别准确率对比

, figureFileSmall=null, figureFileBig=null, tableContent=
类别方法参数量计算量/
GFLOPs
NTU RGB+DNTU RGB+D 120
Xsub/%Xview/%Xsub/%Xset/%
注:模型参数量、计算量参考自文献[15, 33, 35].
GCNST-GCN[4]3.1×10681.588.370.773.2
2s-AGCN[5]3.5×1063.988.595.179.281.5
Shift-GCN[6]0.7×1062.590.796.585.987.6
CTR-GCN[7]1.4×1061.892.496.888.990.6
AS-GCN[26]86.894.278.379.8
MS-G3D[27]2.8×1065.291.596.286.988.4
InfoGCN[28]1.5×1061.792.796.989.490.6
HC-GCN[29]91.296.787.689.0
MSS-GCN[30]92.796.988.990.6
FR-Head[31]2.0×10692.896.889.590.9
HD-GCN[32]1.7×1061.693.097.089.891.2
DEGCN[33]5.6×10693.697.491.092.1
TransformerST-TR[9]12.1×106259.489.996.182.784.7
DSTA[10]4.1×10664.791.596.486.689.0
STSA-Net[15]1.9×10692.796.788.590.7
TranSkeleton[34]2.2×1069.292.897.089.490.5
MLPSiT-MLP[35]0.6×1060.792.396.889.090.5
Transformer本文方法2.5×1065.293.297.189.791.4
), ArticleFig(id=1327017551537746008, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Table 6, caption=Recognition Accuracy Comparison of Different Methods on FineGym Dataset %, figureFileSmall=null, figureFileBig=null, tableContent=
类别方法Top1平均Top1
GCNST-GCN[4]36.425.2
MDR-GCN[36]94.5
CNNActionVLAD[37]69.550.1
I3D[38]74.863.2
TSN[39]86.076.4
TRN[40]87.479.8
TRNms[40]87.880.2
TSM[41]88.481.2
TransformerRSANet[42]86.4
RGBSformer[43]86.7
Deformable[44]90.3
Transformer本文方法95.093.1
), ArticleFig(id=1327017551642603609, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=表6, caption=

不同方法在FineGym数据集的识别准确率对比

, figureFileSmall=null, figureFileBig=null, tableContent=
类别方法Top1平均Top1
GCNST-GCN[4]36.425.2
MDR-GCN[36]94.5
CNNActionVLAD[37]69.550.1
I3D[38]74.863.2
TSN[39]86.076.4
TRN[40]87.479.8
TRNms[40]87.880.2
TSM[41]88.481.2
TransformerRSANet[42]86.4
RGBSformer[43]86.7
Deformable[44]90.3
Transformer本文方法95.093.1
), ArticleFig(id=1327017551722295386, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=EN, label=Table 7, caption=Recognition Accuracy Comparison of Different Methods on InHARD Dataset %, figureFileSmall=null, figureFileBig=null, tableContent=
类别方法数据模态Top1
CNNTSN[39]RGB84.4
TSM[41]RGB88.1
I3D[38]RGB82.1
Non Local[45]RGB86.3
TransformerTimesformer[21]RGB85.2
VideoSwin[46]RGB87.6
Transformer本文方法骨架88.3>
), ArticleFig(id=1327017551789404251, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, language=CN, label=表7, caption=

不同方法在InHARD数据集的识别准确率对比

, figureFileSmall=null, figureFileBig=null, tableContent=
类别方法数据模态Top1
CNNTSN[39]RGB84.4
TSM[41]RGB88.1
I3D[38]RGB82.1
Non Local[45]RGB86.3
TransformerTimesformer[21]RGB85.2
VideoSwin[46]RGB87.6
Transformer本文方法骨架88.3>
)], attaches=null, journal=Journal(id=1324759606947315908, delFlag=0, nameCn=计算机研究与发展, nameEn=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, issn=1000-1239, eissn=null, cn=11-1777/TP, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=k2l8zgqwstuxxevFX2qxVA==, journalPrice=null, startedYear=null, abbrevIsoEn=Journal of Computer Research and Development, journalRemark=null, publicationField=null, createdTime=1793776903141, updatedTime=1793850698612, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=J, firstLetterEn=J, subjectCode=Electronic Information, subjectName=null, subjectCodeEn=Electronic Information, subjectNameEn=null, picCn=k2l8zgqwstuxxevFX2qxVA==, picEn=r6k0Ry+RD+c/VXj5QarmDw==, jcr=null, cjcr=null, exts=[JournalExt(id=1325069127710286688, language=CN, name=计算机研究与发展, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698640, updatedTime=1793850698640, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1325069127777395553, language=EN, name=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698656, updatedTime=1793850698656, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1324759914146529478, websiteList=[Website(id=1325070300139892771, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN, language=CN, createTime=1793850978169, createBy=18614031015, updateTime=1793851040259, updateBy=18614031015, name=计算机研究与发展-中文, tplId=1146099689490845704, title=计算机研究与发展, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070893751349360, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=articleTextType, value=kx, createTime=1793851119697, updateTime=1793851119697, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893730377837, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=banner, value=null, createTime=1793851119692, updateTime=1793851119692, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893772320883, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=grayFlag, value=0, createTime=1793851119702, updateTime=1793851119702, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893721989228, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851119690, updateTime=1793851119690, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893784903797, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=minRunFlag, value=0, createTime=1793851119705, updateTime=1793851119705, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893742960751, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic, createTime=1793851119695, updateTime=1793851119695, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893780709492, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=silenceFlag, value=0, createTime=1793851119704, updateTime=1793851119704, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893738766446, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1793851119694, updateTime=1793851119694, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893755543665, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeColor, value=null, createTime=1793851119698, updateTime=1793851119698, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893763932274, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeStyle, value=null, createTime=1793851119700, updateTime=1793851119700, creator=18614031015, updator=18614031015)]), Website(id=1325070300580294698, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN, language=EN, createTime=1793850978274, createBy=18614031015, updateTime=1793851036638, updateBy=18614031015, name=计算机研究与发展-英文, tplId=1146101810881728533, title=Journal of Computer Research and Development, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070925288321148, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=articleTextType, value=kx, createTime=1793851127216, updateTime=1793851127216, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925263155321, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=banner, value=null, createTime=1793851127210, updateTime=1793851127210, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925309292671, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=grayFlag, value=0, createTime=1793851127221, updateTime=1793851127221, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925254766712, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851127208, updateTime=1793851127208, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925326069889, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=minRunFlag, value=0, createTime=1793851127225, updateTime=1793851127225, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925279932539, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic, createTime=1793851127214, updateTime=1793851127214, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925317681280, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=silenceFlag, value=0, createTime=1793851127223, updateTime=1793851127223, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925271543930, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1793851127212, updateTime=1793851127212, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925292515453, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeColor, value=null, createTime=1793851127217, updateTime=1793851127217, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925300904062, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeStyle, value=null, createTime=1793851127219, updateTime=1793851127219, creator=18614031015, updator=18614031015)])], journalTitle=计算机研究与发展, weixinUrl=null, journalUrl=https://crad.ict.ac.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Journal of Computer Research and Development, journalPhotoCn=k2l8zgqwstuxxevFX2qxVA==, journalPhotoEn=r6k0Ry+RD+c/VXj5QarmDw==, journalFirstLetter=J, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false, interPubPlatform=, interPubPlatformUrl=null), detailUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202440558, detailUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202440558, pdfUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/PDF/10.7544/issn1000-1239.202440558, pdfUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/PDF/10.7544/issn1000-1239.202440558, aliStartDate=0, aliEndDate=0, collectionFlag=false, citedCount=null, citedUrl=null, previewStatus=0, delFlag=0, hasFullText=1, orderTime=1764518400000, fullTextJson=null, articleText=null, reference=null)
收藏切换
融合运动领域知识与自适应时空Transformer的人体骨架行为识别
梁成武 1, 2 , 蒋松琪 1, 2 , 杨杰 1, 2, 3 , 朱培旺 1, 2 , 帖云 5 , 高磊 6 , 胡伟 1, 2, 4 , 郭文博 1
计算机研究与发展 | 人工智能 2025,62(12): 3028-3041
收起
收藏切换
计算机研究与发展 |人工智能 2025 , 62 (12) : 3028 -3041
融合运动领域知识与自适应时空Transformer的人体骨架行为识别
全屏
[Author(id=1327017545636360193, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=liangchengwu0615@126.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017545741217796, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017545636360193, language=EN, stringName=Chengwu Liang, firstName=Chengwu, middleName=null, lastName=Liang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041
2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002, bio={"content":"

born in 1982. PhD, professor, master supervisor. Member of CCF. His main research interests include computer vision, machine learning, and multimedia processing

"}, bioImg=null, bioContent=

born in 1982. PhD, professor, master supervisor. Member of CCF. His main research interests include computer vision, machine learning, and multimedia processing

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017545825103877, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017545636360193, language=CN, stringName=梁成武, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1河南城建学院电气与控制工程学院 河南平顶山 467041
2三峡大学电气与新能源学院 湖北宜昌 443002, bio={"img":"70+g9qsK40VgrcoR0ZnweQ==","content":"

1982年生. 博士,教授,硕士生导师,CCF会员. 主要研究方向为计算机视觉、机器学习、多媒体处理

"}, bioImg=70+g9qsK40VgrcoR0ZnweQ==, bioContent=

1982年生. 博士,教授,硕士生导师,CCF会员. 主要研究方向为计算机视觉、机器学习、多媒体处理

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017544990438383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=1, ext=[AuthorCompanyExt(id=1327017545003021296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041), AuthorCompanyExt(id=1327017545011409905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1河南城建学院电气与控制工程学院 河南平顶山 467041)]), AuthorCompany(id=1327017545099490290, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=2, ext=[AuthorCompanyExt(id=1327017545107878899, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002), AuthorCompanyExt(id=1327017545116267508, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2三峡大学电气与新能源学院 湖北宜昌 443002)])]), Author(id=1327017545908989959, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017546030624778, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017545908989959, language=EN, stringName=Songqi Jiang, firstName=Songqi, middleName=null, lastName=Jiang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041
2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002, bio={"content":"

born in 1999. Master candidate. His main research interests include target detection and human action recognition

"}, bioImg=null, bioContent=

born in 1999. Master candidate. His main research interests include target detection and human action recognition

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017546101927947, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017545908989959, language=CN, stringName=蒋松琪, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1河南城建学院电气与控制工程学院 河南平顶山 467041
2三峡大学电气与新能源学院 湖北宜昌 443002, bio={"img":"gBi8qMFemoD5ZJJ+HBp0Bw==","content":"

1999年生. 硕士研究生. 主要研究方向为目标检测、人体动作识别

"}, bioImg=gBi8qMFemoD5ZJJ+HBp0Bw==, bioContent=

1999年生. 硕士研究生. 主要研究方向为目标检测、人体动作识别

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017544990438383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=1, ext=[AuthorCompanyExt(id=1327017545003021296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041), AuthorCompanyExt(id=1327017545011409905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1河南城建学院电气与控制工程学院 河南平顶山 467041)]), AuthorCompany(id=1327017545099490290, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=2, ext=[AuthorCompanyExt(id=1327017545107878899, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002), AuthorCompanyExt(id=1327017545116267508, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2三峡大学电气与新能源学院 湖北宜昌 443002)])]), Author(id=1327017546177425421, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017546265505809, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017546177425421, language=EN, stringName=Jie Yang, firstName=Jie, middleName=null, lastName=Yang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 3, address=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041
2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002
3State Grid Hunan Electric Power Co., Ltd. Huaihua Power Supply Branch, Huaihua, Hunan 418006, bio={"content":"

born in 1998. Master. His main research interests include computer vision and multi-modality information fusion

"}, bioImg=null, bioContent=

born in 1998. Master. His main research interests include computer vision and multi-modality information fusion

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017546332614674, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017546177425421, language=CN, stringName=杨杰, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 3, address=1河南城建学院电气与控制工程学院 河南平顶山 467041
2三峡大学电气与新能源学院 湖北宜昌 443002
3国网湖南省电力有限公司怀化供电分公司 湖南怀化 418006, bio={"img":"kvByoCjYoOGdxmocGWksHA==","content":"

1998年生. 硕士. 主要研究方向为计算机视觉、多模态信息融合

"}, bioImg=kvByoCjYoOGdxmocGWksHA==, bioContent=

1998年生. 硕士. 主要研究方向为计算机视觉、多模态信息融合

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017544990438383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=1, ext=[AuthorCompanyExt(id=1327017545003021296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041), AuthorCompanyExt(id=1327017545011409905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1河南城建学院电气与控制工程学院 河南平顶山 467041)]), AuthorCompany(id=1327017545099490290, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=2, ext=[AuthorCompanyExt(id=1327017545107878899, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002), AuthorCompanyExt(id=1327017545116267508, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2三峡大学电气与新能源学院 湖北宜昌 443002)]), AuthorCompany(id=1327017545195959285, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=3, ext=[AuthorCompanyExt(id=1327017545212736502, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545195959285, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3State Grid Hunan Electric Power Co., Ltd. Huaihua Power Supply Branch, Huaihua, Hunan 418006), AuthorCompanyExt(id=1327017545221125111, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545195959285, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3国网湖南省电力有限公司怀化供电分公司 湖南怀化 418006)])]), Author(id=1327017546399723540, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017546487803927, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017546399723540, language=EN, stringName=Peiwang Zhu, firstName=Peiwang, middleName=null, lastName=Zhu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041
2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002, bio={"content":"

born in 2000. Master candidate. His main research interest includes computer vision

"}, bioImg=null, bioContent=

born in 2000. Master candidate. His main research interest includes computer vision

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017546601050136, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017546399723540, language=CN, stringName=朱培旺, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1河南城建学院电气与控制工程学院 河南平顶山 467041
2三峡大学电气与新能源学院 湖北宜昌 443002, bio={"img":"B+K2/8TziFnt+PqvPbbZNQ==","content":"

2000年生. 硕士研究生. 主要研究方向为计算机视觉

"}, bioImg=B+K2/8TziFnt+PqvPbbZNQ==, bioContent=

2000年生. 硕士研究生. 主要研究方向为计算机视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017544990438383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=1, ext=[AuthorCompanyExt(id=1327017545003021296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041), AuthorCompanyExt(id=1327017545011409905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1河南城建学院电气与控制工程学院 河南平顶山 467041)]), AuthorCompany(id=1327017545099490290, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=2, ext=[AuthorCompanyExt(id=1327017545107878899, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002), AuthorCompanyExt(id=1327017545116267508, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2三峡大学电气与新能源学院 湖北宜昌 443002)])]), Author(id=1327017546668159002, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017546865291292, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017546668159002, language=EN, stringName=Yun Tie, firstName=Yun, middleName=null, lastName=Tie, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=5, address=5School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001, bio={"content":"

born in 1974. PhD, professor, PhD supervisor. His main research interests include intelligent multimedia, and virtual reality and interaction

"}, bioImg=null, bioContent=

born in 1974. PhD, professor, PhD supervisor. His main research interests include intelligent multimedia, and virtual reality and interaction

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017546953371677, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017546668159002, language=CN, stringName=帖云, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=5, address=5郑州大学电气与信息工程学院 郑州 450001, bio={"img":"NWsL6rgINqQnGzElnsLY7w==","content":"

1974年生. 博士,教授,博士生导师. 主要研究方向为智能多媒体、虚拟现实及交互

"}, bioImg=NWsL6rgINqQnGzElnsLY7w==, bioContent=

1974年生. 博士,教授,博士生导师. 主要研究方向为智能多媒体、虚拟现实及交互

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017545376314363, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=5, ext=[AuthorCompanyExt(id=1327017545388897276, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545376314363, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=5School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001), AuthorCompanyExt(id=1327017545422451709, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545376314363, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=5郑州大学电气与信息工程学院 郑州 450001)])]), Author(id=1327017547054034975, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017547150503969, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017547054034975, language=EN, stringName=Lei Gao, firstName=Lei, middleName=null, lastName=Gao, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=6, address=6Department of Electrical, Computer and Biomedical Engineering, Metropolitan University of Toronto, Toronto, Canada M5B2K3, bio={"content":"

born in 1983. PhD, associate professor. His main research interests include computer vision and multimedia processing

"}, bioImg=null, bioContent=

born in 1983. PhD, associate professor. His main research interests include computer vision and multimedia processing

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017547246972962, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017547054034975, language=CN, stringName=高磊, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=6, address=6多伦多都会大学电气计算机和生物医学工程系 加拿大多伦多 M5B2K3, bio={"img":"hY5tYxfAg4kyShey4NxHhw==","content":"

1983年生. 博士,副教授. 主要研究方向为计算机视觉、多媒体处理

"}, bioImg=hY5tYxfAg4kyShey4NxHhw==, bioContent=

1983年生. 博士,副教授. 主要研究方向为计算机视觉、多媒体处理

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017545535697918, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=6, ext=[AuthorCompanyExt(id=1327017545548280831, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545535697918, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=6Department of Electrical, Computer and Biomedical Engineering, Metropolitan University of Toronto, Toronto, Canada M5B2K3), AuthorCompanyExt(id=1327017545556669440, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545535697918, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=6多伦多都会大学电气计算机和生物医学工程系 加拿大多伦多 M5B2K3)])]), Author(id=1327017547322470436, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=6, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017547435716648, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017547322470436, language=EN, stringName=Wei Hu, firstName=Wei, middleName=null, lastName=Hu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 4, address=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041
2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002
4State Grid Anhui Electric Power Co., Ltd. Tongling Power Supply Branch, Tongling, Anhui 244151, bio={"content":"

born in 1999. Master. His main research interest includes smart grid

"}, bioImg=null, bioContent=

born in 1999. Master. His main research interest includes smart grid

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017547523797033, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017547322470436, language=CN, stringName=胡伟, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 4, address=1河南城建学院电气与控制工程学院 河南平顶山 467041
2三峡大学电气与新能源学院 湖北宜昌 443002
4国网安徽省电力有限公司铜陵供电分公司 安徽铜陵 244151, bio={"img":"zI+GUbNEDdJUYVUNVgjdvg==","content":"

1999年生. 硕士. 主要研究方向为智能电网

"}, bioImg=zI+GUbNEDdJUYVUNVgjdvg==, bioContent=

1999年生. 硕士. 主要研究方向为智能电网

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017544990438383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=1, ext=[AuthorCompanyExt(id=1327017545003021296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041), AuthorCompanyExt(id=1327017545011409905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1河南城建学院电气与控制工程学院 河南平顶山 467041)]), AuthorCompany(id=1327017545099490290, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=2, ext=[AuthorCompanyExt(id=1327017545107878899, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002), AuthorCompanyExt(id=1327017545116267508, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545099490290, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2三峡大学电气与新能源学院 湖北宜昌 443002)]), AuthorCompany(id=1327017545288233976, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=4, ext=[AuthorCompanyExt(id=1327017545296622585, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545288233976, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4State Grid Anhui Electric Power Co., Ltd. Tongling Power Supply Branch, Tongling, Anhui 244151), AuthorCompanyExt(id=1327017545309205498, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017545288233976, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4国网安徽省电力有限公司铜陵供电分公司 安徽铜陵 244151)])]), Author(id=1327017547607683115, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, orderNo=7, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017547725123629, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017547607683115, language=EN, stringName=Wenbo Guo, firstName=Wenbo, middleName=null, lastName=Guo, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041, bio={"content":"

born in 1994. PhD, lecturer. His main research interests include computer vision and image processing

"}, bioImg=null, bioContent=

born in 1994. PhD, lecturer. His main research interests include computer vision and image processing

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017547800621102, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, authorId=1327017547607683115, language=CN, stringName=郭文博, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1河南城建学院电气与控制工程学院 河南平顶山 467041, bio={"img":"WATKphBeMbHlFn9/3Hh/zQ==","content":"

1994年生. 博士,讲师. 主要研究方向为计算机视觉、图像处理

"}, bioImg=WATKphBeMbHlFn9/3Hh/zQ==, bioContent=

1994年生. 博士,讲师. 主要研究方向为计算机视觉、图像处理

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017544990438383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, xref=1, ext=[AuthorCompanyExt(id=1327017545003021296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041), AuthorCompanyExt(id=1327017545011409905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017541538526154, companyId=1327017544990438383, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1河南城建学院电气与控制工程学院 河南平顶山 467041)])])]
梁成武1, 2 , 蒋松琪1, 2, 杨杰1, 2, 3, 朱培旺1, 2, 帖云5, 高磊6, 胡伟1, 2, 4, 郭文博1
作者信息
  • 1河南城建学院电气与控制工程学院 河南平顶山 467041
  • 2三峡大学电气与新能源学院 湖北宜昌 443002
  • 3国网湖南省电力有限公司怀化供电分公司 湖南怀化 418006
  • 4国网安徽省电力有限公司铜陵供电分公司 安徽铜陵 244151
  • 5郑州大学电气与信息工程学院 郑州 450001
  • 6多伦多都会大学电气计算机和生物医学工程系 加拿大多伦多 M5B2K3
作者简介:

1982年生. 博士,教授,硕士生导师,CCF会员. 主要研究方向为计算机视觉、机器学习、多媒体处理

1999年生. 硕士研究生. 主要研究方向为目标检测、人体动作识别

1998年生. 硕士. 主要研究方向为计算机视觉、多模态信息融合

2000年生. 硕士研究生. 主要研究方向为计算机视觉

1974年生. 博士,教授,博士生导师. 主要研究方向为智能多媒体、虚拟现实及交互

1983年生. 博士,副教授. 主要研究方向为计算机视觉、多媒体处理

1999年生. 硕士. 主要研究方向为智能电网

1994年生. 博士,讲师. 主要研究方向为计算机视觉、图像处理

Skeleton-Based Human Action Recognition with Fusion of Motion Domain Knowledge and Adaptive Spatio-Temporal Transformer
Chengwu Liang1, 2 , Songqi Jiang1, 2, Jie Yang1, 2, 3, Peiwang Zhu1, 2, Yun Tie5, Lei Gao6, Wei Hu1, 2, 4, Wenbo Guo1
Affiliations
  • 1School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan, Henan 467041
  • 2College of Electrical Engineering and New Energy, China Three Gorges University, Yichang, Hubei 443002
  • 3State Grid Hunan Electric Power Co., Ltd. Huaihua Power Supply Branch, Huaihua, Hunan 418006
  • 4State Grid Anhui Electric Power Co., Ltd. Tongling Power Supply Branch, Tongling, Anhui 244151
  • 5School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001
  • 6Department of Electrical, Computer and Biomedical Engineering, Metropolitan University of Toronto, Toronto, Canada M5B2K3
  • born in 1982. PhD, professor, master supervisor. Member of CCF. His main research interests include computer vision, machine learning, and multimedia processing

    born in 1999. Master candidate. His main research interests include target detection and human action recognition

    born in 1998. Master. His main research interests include computer vision and multi-modality information fusion

    born in 2000. Master candidate. His main research interest includes computer vision

    born in 1974. PhD, professor, PhD supervisor. His main research interests include intelligent multimedia, and virtual reality and interaction

    born in 1983. PhD, associate professor. His main research interests include computer vision and multimedia processing

    born in 1999. Master. His main research interest includes smart grid

    born in 1994. PhD, lecturer. His main research interests include computer vision and image processing

出版时间: 2025-12-01 doi: 10.7544/issn1000-1239.202440558
文章导航
收藏切换

现有人体骨架行为识别方法往往忽略运动学领域知识,造成模型内在人类可理解的逻辑决策解释性不足. 基于此,提出一种融合领域知识与自适应时空片段Transformer的骨架行为识别方法,以提高骨架行为识别模型的性能和可解释性. 首先,受短时运动领域知识启发,设计多时间分支结构用于学习和捕捉多时间尺度的短时子动作特征. 其次,提出一种动态信息融合模块,学习不同时间分支的权重向量进而动态融合多时间分支、多尺度短时运动特征. 最后,融合长时运动领域知识,提出多尺度时间卷积特征融合模块捕捉长时运动关联,用于学习不同子动作片段之间的关系并促进不同骨架关节点间的运动信息交互. 在4个数据集上进行评估与实验,包括人体日常行为数据集NTU RGB+D和NTU RGB+D 120、体育行为数据集FineGym,以及工业场景行为数据集InHARD. 结果表明所提方法的行为识别性能,优于包括基准Transformer方法在内的多个先进方法,可有效提升骨架序列短时运动特征学习和关节点之间信息交互的建模能力,并具有一定可解释性.

骨架行为识别  /  领域知识  /  时间片段  /  子动作  /  Transformer  /  自注意力机制

Existing methods for skeleton-based human action recognition always ignore motion domain knowledge, resulting in the lack of interpretability of logical decision-making that human can understand. In this paper, we propose a novel skeleton-based human action recognition method by fusion of domain knowledge and adaptive spatio-temporal Transformer, to improve recognition performance and interpretability. Firstly, inspired by the short-term motion knowledge, a temporal multi-branch structure is designed to learn and capture the characteristics of short-term sub-acitons. Secondly, a dynamic information fusion module is proposed to learn the weight vectors of different temporal branches, and then fuse multiscale short-term motion features. Finally, to learn the relationship between different sub-actions and facilitate the motion information interaction between skeleton joints, a multiscale temporal convolution feature fusion module is proposed to capture the long-term motion correlations, by integrating the domain knowledge of the long-term motion. Experimental evaluations are conducted on four large action datasets, including NTU RGB+D, NTU RGB+D 120, FineGym, and InHARD. The experimental results show that the recognition performance of the proposed method is superior to several data-driven methods, effectively improving the modelling ability of short-term motion feature learning and information interaction between skeleton joints, with the interpretability.

skeleton-based action recognition  /  domain knowledge  /  temporal segment  /  sub-aciton  /  Transformer  /  self-attention mechanism
梁成武, 蒋松琪, 杨杰, 朱培旺, 帖云, 高磊, 胡伟, 郭文博. 融合运动领域知识与自适应时空Transformer的人体骨架行为识别. 计算机研究与发展, 2025 , 62 (12) : 3028 -3041 . DOI: 10.7544/issn1000-1239.202440558
Chengwu Liang, Songqi Jiang, Jie Yang, Peiwang Zhu, Yun Tie, Lei Gao, Wei Hu, Wenbo Guo. Skeleton-Based Human Action Recognition with Fusion of Motion Domain Knowledge and Adaptive Spatio-Temporal Transformer[J]. Journal of Computer Research and Development, 2025 , 62 (12) : 3028 -3041 . DOI: 10.7544/issn1000-1239.202440558
人体行为识别作为计算机视觉领域的研究热点,在视频监控、自动驾驶以及人机交互等多个领域具有广泛应用[1-3]. 人体行为识别技术可以赋予具身智能系统理解和适应人类工作生活环境的能力,提高智能体对真实世界的理解能力. 鉴于骨架序列数据的紧凑性和对背景与光照条件变化的鲁棒性,基于骨架序列的人体行为识别任务逐渐引起研究者的广泛关注. 随着深度学习的迅速发展,目前基于骨架序列的人体行为识别方法主要分为基于图卷积神经网络(graph convolutional neural network,GCN)[4-8]和Transformer的方法[9-11]. Yan等人[4]利用骨架序列的人体内在拓扑结构和时间关系构建时空图,并通过空间图卷积和时间卷积逐步学习更高层次的时空运动特征. 针对GCN难以有效捕捉长时特征和全局依赖关系的局限性,石跃祥等人[11]提出协作卷积Transformer模型,通过自注意力机制提升模型的全局特征建模能力. 人体运动领域知识和骨架行为识别密切相关,主要包括人体运动的动态规律、行为习惯模式和时空上下文先验知识等. 尽管以上方法在人体骨架识别任务中取得一定进展,但往往忽略人体短时子动作、长时运动特征学习在模型设计中的重要性,缺乏人体运动领域知识的运用,导致骨架序列数据特征的挖掘学习和可解释性不足.
由短时运动领域知识可知,人体行为可以由多个具有运动学语义的基本短时子动作组成,因此对短时骨架子动作的显式建模有助于学习和分类完整的人体行为. 具体而言,骨架序列中短时间范围(相邻帧)中的不同关节点通常呈现高度相关性. 以“跳远”动作为例,前一帧的手臂与后一帧的腿部动作会协同运动并存在一定相关性. 此外,不同人体行为的子动作也存在持续时间、运动速率等变化规律的差异,对不同的人体行为需要进行针对性设计. 受上述人体运动领域知识启发,行为识别研究者们对骨架行为识别方法进行一系列改进[12-15]. 其中Wang等人[13]提出一种时间差分网络,采用先建模短时运动信息,后建模长期运动信息的策略,显著提升了人体行为动作的时空特征建模能力. 基于人体行为可由若干子动作组成这一领域知识,Qiu等人[15]提出一种固定长度的时间分段注意力机制对短时运动特征进行显式建模,但未能考虑到不同人体行为的特征差异性,难以适用于不同行为的短时子动作建模.
短时子动作特征的建模是学习和识别人体行为的基础,与之不同,长时运动特征的建模可以学习不同子动作间的关联以及不同骨架关节点间的长时相关性. Chen等人[7]采用多尺度时间卷积模块学习不同尺度的时间特征,忽略了不同骨架关节点之间的长时信息交互. 考虑到现有方法缺乏对关节点间的长时依赖建模,Duan等人[16]在时间特征建模的过程中通过引入全局骨架特征来增强不同关节点之间的信息交互,但缺少对不同局部子动作之间关联的建模.
基于上述分析,本文提出一种融合运动领域知识与自适应时间片段注意力的Transformer模型用于人体骨架行为识别. 主要贡献如下:
1)受短时运动领域知识启发,提出自适应时间分段注意力(adaptive temporal segment attention,ATSA)模块学习不同行为的短时子动作特征. ATSA模块通过多时间分支结构对不同时间尺度的局部人体子运动特征进行建模,并通过动态信息融合模块整合多时间分支、多尺度子动作特征.
2)融合长时运动领域知识,提出多尺度时间卷积特征融合模块MTCF(multiscale temporal convolution with skeleton feature fusion)学习长时行为片段间的运动特征和不同子动作间的关联. MTCF采用不同感受野的2D时间卷积进行长时特征建模,以学习人体行为子动作间的关联,同时引入骨架特征融合机制增加人体骨架不同关节点间的信息交互.
3)在4个数据集NTU RGB+D[17],NTU RGB+D 120[18],FineGym[19],InHARD[20]上进行性能评估,分别取得93.2%(Cross-Subject)、89.7%(Cross-Subject)、95.0%和88.3%的Top1识别准确率. 结果表明本文方法通过融合长短时运动领域知识,可有效增强模型的短时运动特征建模能力和长时信息交互能力,进而提高骨架行为识别性能和可解释性.
现有人体行为识别模型在捕捉长短时运动特征以及可解释性方面存在局限性,模型内在人类可理解的逻辑决策解释性不足. 受短时运动领域知识启发,设计ATSA用于学习和捕捉多时间尺度的短时子动作特征. 为学习骨架运动不同时间分支的权重向量及其动态融合,提出动态信息融合(dyamic information fusion,DIF)模块进行多时间尺度分支短时运动特征融合. 为捕捉长时骨架运动关联,提出MTCF学习不同子动作片段之间的关系.
如图1所示,首先,线性嵌入层将骨架序列中的关节点坐标信息映射为高维特征向量序列. 然后,输入特征序列被送入$L$层堆叠的编码器模块进行时空特征建模,其中每一层分别由ATSA和MTCF组成. 最后,得到的输出特征经全局平均池化层、线性层和Softmax层后输出预测行为类别.
具体地,图1右下侧ATSA模块中实线和虚线分别表示不同时间尺度分支所对应帧内和帧间的关节点信息交互. ATSA模块通过学习相邻帧中不同关节点的相关性来建模局部子动作特征. 图1右上侧MTCF模块中实线和虚线代表骨架特征融合过程和多尺度时间卷积. MTCF模块采用多尺度时间卷积来学习不同子动作之间的关联.
综上,本文提出一种融合运动领域知识与自适应时间片段注意力的Transformer方法用于人体骨架行为识别.
Transformer模型中嵌入层的目的是将离散化的输入数据转换为高维密集的连续向量. 这些向量进一步与位置编码相结合,共同构成了模型处理的输入序列. 本文方法中的线性嵌入层用于将输入骨架序列编码为输入特征向量序列.
不同于传统视觉Transformer[21]将图像裁剪为若干固定大小的图像块,然后进行线性映射后编码得到输入特征向量序列的方法,骨架序列仅由少量的人体关节点组成,因此可直接对每个关节点进行线性映射得到输入特征向量矩阵:
$ {{\tilde{\boldsymbol Z}}_0} = Conv{\text{2}}{D_{\left( {1 \times 1} \right)}}( {{{\boldsymbol{X}}_{\rm seq}}} ) \in {\mathbb{R}^{T \times V \times C}} , $
其中${{\tilde{\boldsymbol Z}}_0}$表示经过线性映射的特征向量矩阵. $T$,$V$,$C$分别表示帧数、关节点数和特征通道数. $ Conv{\text{2}}{D_{\left( {1 \times 1} \right)}} $表示卷积核大小为$1 \times 1$的2D卷积.
经线性映射得到的特征向量序列未包含人体骨架序列的空间位置、时间顺序信息以及固有属性信息,需要引入位置编码向量对该信息进行补充. 本文与Transformer[22]一致,采用不同频率的正弦和余弦函数来生成位置编码向量${\boldsymbol{PE}}$,其中
$ {\boldsymbol{PE}}\left( {p,2i} \right) = \sin \left( {p/{{10\;000}^{2i/C}}} \right) , $
$ {\boldsymbol{PE}}\left( {p,2i + 1} \right) = \cos \left( {p/{{10\;000}^{2i/C}}} \right) , $
其中$ p $表示每个特征向量在输入序列中所处的位置,$ i \in \{ {0,1, … ,{C / 2}} \} $表示位置编码向量的具体索引. 然后,将特征向量序列和位置编码向量相加得到输入特征矩阵$ {{\boldsymbol{Z}}_{\rm in}}\in {\mathbb{R}^{T \times V \times C}} $:
$ {{\boldsymbol{Z}}_{\rm in}} = {{\tilde{\boldsymbol Z}}_{\rm in}} + {\boldsymbol{PE}} . $
经线性嵌入层后,输入特征矩阵$ {{\boldsymbol{Z}}_{\rm in}} $送入$L$层堆叠的编码器模块进行特征建模.
基于人体运动领域知识可知,完整的人体行为可视为若干个子动作在时间维度上的语义组合,因此对人体子动作特性的学习是对人体行为分析理解的基础和关键. 为使模型能有效学习局部时间片段窗口内的子动作特征,段间自注意力机制(intra-segment self-attention,ISSA)对固定长度时间片段内的短时运动特征进行显式建模.
如图2所示,相较传统时空自注意力机制[23]对时空特征进行全局建模,ISSA通过将自注意力机制操作限制在局部时间片段内,以此达到对子动作特征显示建模的效果. 首先输入特征$ {{\boldsymbol{Z}}_{{\text{in}}}} $沿时间维度被分为$ {T_n} $个时间长度相等且不重叠的时间片段:
$ {{\boldsymbol{Z}}_{\rm in}} \in {\mathbb{R}^{T \times V \times C}} \to {\mathbb{R}^{{T_n} \times n \times V \times C}} \to {\mathbb{R}^{{T_n} \times {V_n} \times C}} , $
其中$ n $表示每个时间片段的长度,${V_n} = V \times n$表示每个时间片段内的关节点总数.
经过时间分段后,ISSA对每个时间片段分别进行自注意力机制操作. 自注意力机制通过查询(${\boldsymbol{Q}}$)与键-值对(${\boldsymbol{K}}{\text{-}}{\boldsymbol{V}}$)之间的交互实现对时间片段内不同关节点信息的动态聚合. ${\boldsymbol{Q}}$,${\boldsymbol{K}}$,$ {\boldsymbol{V}} $分别通过对输入特征$ {{\boldsymbol{Z}}_{{\text{in}}}} $进行线性映射得到:
$ \boldsymbol Q={{\boldsymbol{W}}_{\boldsymbol Q}}{{\boldsymbol{Z}}_{{\text{in}}}},\;\;\boldsymbol K={{\boldsymbol{W}}_{\boldsymbol K}}{{\boldsymbol{Z}}_{{\text{in}}}},\;\;\boldsymbol V={{\boldsymbol{W}}_{\boldsymbol V}}{{\boldsymbol{Z}}_{{\text{in}}}}, $
其中$ {{\boldsymbol{W}}_{\boldsymbol Q}} $,${{\boldsymbol{W}}_{\boldsymbol K}}$,${{\boldsymbol{W}}_{\boldsymbol V}}$分别对应${\boldsymbol{Q}}$,${\boldsymbol{K}}$,${\boldsymbol{V}}$的线性映射矩阵. 通过计算${\boldsymbol{Q}}$和${\boldsymbol{K}}$的相关性可得到注意力权重矩阵,然后将其与${\boldsymbol{V}}$相乘即可得到输出特征:
$ {\boldsymbol{D}} = \tanh ( {{\boldsymbol{Q}}{{\boldsymbol{K}}^{\rm T}}/\sqrt C } ) , $
$ {{\boldsymbol{Z}}_{\rm attn}} = {\boldsymbol{V}}\left( {\alpha \times {\boldsymbol{D}} + {\boldsymbol{P}}} \right) , $
其中$ {{\boldsymbol{Z}}_{\rm attn}} $表示自注意力机制操作的输出特征. $\tanh $表示双曲正切函数,$\sqrt C $用于提高训练过程中的梯度稳定性. ${\boldsymbol{D}}$表示通过计算${\boldsymbol{Q}}$和${\boldsymbol{K}}$之间的相似度得到的注意力权重矩阵. 为引入人体骨架内在拓扑关系,增加可学习的参数矩阵${\boldsymbol{P}}$用于表示人体固有铰链的拓扑连接关系,该矩阵在模型训练过程中迭代更新参数. $ \alpha $为可学习的矩阵权重参数,用于平衡${\boldsymbol{D}}$和${\boldsymbol{P}}$的关系.
为提高特征多样性和模型性能,ISSA同样采用多头注意力机制. 多头注意力机制的计算方法是将输入特征分为多组后,分别进行自注意力机制计算,最后通过拼接得到输出特征$ {{\boldsymbol{Z}}_{\rm attn}} $:
$ {{\boldsymbol{Z}}_{\rm attn}} = Concat\left( {{\boldsymbol{Z}}_{\rm attn}^1,{\boldsymbol{Z}}_{\rm attn}^2, … ,{\boldsymbol{Z}}_{\rm attn}^h} \right) , $
其中Concat表示沿特征通道维度进行矩阵拼接,${\boldsymbol{Z}}_{\rm attn}^h$表示第$h$组注意力输出特征. 与Transformer编码器模块类似,ISSA最后采用全连接层用于提升网络的拟合能力:
$ {{\boldsymbol{Z}}_{\rm ISSA}} = Conv{\text{2}}{D_{\left( {1 \times 1} \right)}}\left( {{{\boldsymbol{Z}}_{\rm attn}}} \right) , $
其中${{\boldsymbol{Z}}_{\rm ISSA}}$表示ISSA的输出特征. $ Conv{\text{2}}{D_{\left( {1 \times 1} \right)}} $表示$1 \times 1$卷积操作,用于实现全连接层的功能.
ISSA通过将自注意力机制限制在固定长度的时间片段内,从而实现对人体行为子动作特征的显式学习和捕捉. 然而,由人体运动领域知识可知,在不同人体行为类别之间和相同行为类别的不同时间维度,子动作的持续时间和动态规律均存在显著差异,这导致单一固定时间长度的ISSA难以适用于不同人体行为样本的子动作特征建模.
不同骨架行为样本之间,运动速率和运动阶段存在较大差异. 因此,需要针对每个特定骨架行为样本选择合适长度的时间片段进行短时运动特征建模. 受短时运动领域知识的进一步启发,提出ATSA,用于学习不同人体行为的短时子动作特征. ATSA采用多分支结构对不同长度的时间片段进行局部特征建模,并通过动态信息融合模块DIF整合不同时间尺度的短时运动特征.
如图3所示,首先输入特征$ {{\boldsymbol{Z}}_{{\text{in}}}} $沿通道维度被分为$K$组特征:
$ {\boldsymbol{Z}}_{\rm in}^1,{\boldsymbol{Z}}_{\rm in}^2, … ,{\boldsymbol{Z}}_{\rm in}^i, … ,{\boldsymbol{Z}}_{\rm in}^K = S plit\left( {{{\boldsymbol{Z}}_{\rm in}}} \right){\text{ }} , $
其中Split表示沿通道维度对输入特征进行分割操作. 每一组特征$ {\boldsymbol{Z}}_{\rm in}^i \in {\mathbb{R}^{T \times V \times C/K}} $被输入不同的分支以分别进行短时运动特征建模:
$ {\boldsymbol{Z}}_{\rm ISSA}^i = IS S {A_{n{{ = }}i}}\left( {{\boldsymbol{Z}}_{\rm in}^i} \right), $
其中${\boldsymbol{Z}}_{\rm ISSA}^i$表示第$i$个分支的输出特征. $IS S {A_{n{{ = }}i}}$表示采用时间片段长度为$i$的段间自注意力机制处理过程. 经过不同分支的ISSA建模后,模型能够学习到不同时间尺度的人体行为子动作特征. 此外值得注意的是,由于各个分支的特征通道维度仅为输入特征的${1 / K}$,ATSA模块的注意力机制计算量降低为单分支的${{1 / K}^2}$,从而有效减少了模型计算开销.
为有效适应不同人体行为的子动作差异性,受文献[24]启发提出动态信息融合(dynamic information fusion,DIF)模块用于对不同时间尺度的子动作特征进行融合. DIF由特征选择和特征融合2个阶段组成,特征选择阶段基于全局表征计算每组特征的权重向量,特征融合阶段根据每组特征的权重进行多尺度特征融合.
首先,在特征选择阶段,对$ K $个分支的不同尺度的输出特征进行相加,再送入全局平均池化层(global average pooling,GAP). 经由全局平均池化处理后的特征可以视为一种全局表征,具有数据紧凑的特点,并且可极大降低线性层的计算开销. 然后,全局表征经过多层感知器层(multi-layer perceptron,MLP)和Softmax层计算得到每组特征各自的权重向量:
$ {\boldsymbol{W}} = MLP\left( {GAP\left( {\sum\limits_{i = 1}^K {{\boldsymbol{Z}}_{\rm ISSA}^i} } \right)} \right) \in {\mathbb{R}^{KC \times 1 \times 1}} , $
$ {{\boldsymbol{W}}_1}{\text{,}}{{\boldsymbol{W}}_2}{\text{,}} … {\text{,}}{{\boldsymbol{W}}_K}{{ = }}S plit\left( {\boldsymbol{W}} \right) \in {\mathbb{R}^{C \times 1 \times 1}} , $
其中${{\boldsymbol{W}}_i}$表示第$i$个分支的权重向量.
然后,在特征融合阶段,将每一组ISSA模块的输出特征与各自的权重向量相乘,并进行求和得到最终的输出特征:
$ {{\boldsymbol{Z}}_{\rm DIF}} = \sum\limits_{i = 1}^K {{{\boldsymbol{W}}_i} \odot {\boldsymbol{Z}}_{\rm ISSA}^i} . $
根据输入特征的全局信息来动态计算每一组输入特征的权重向量,动态信息融合模块能够对不同时间尺度的特征进行动态融合,使得模型能更好地处理不同类别和不同变化速率的人体行为样本.
此外,为进一步提升模型训练过程中的稳定性,ATSA模块在最终阶段引入残差连接:
$ {{\boldsymbol{Z}}_{\rm ATSA}} = DIF\left( {{\boldsymbol{Z}}_{\rm ISSA}^1,{\boldsymbol{Z}}_{\rm ISSA}^2, … ,{\boldsymbol{Z}}_{\rm ISSA}^i, … ,{\boldsymbol{Z}}_{\rm ISSA}^K} \right) + {{\boldsymbol{Z}}_{\rm in}}, $
其中$ {{\boldsymbol{Z}}_{\rm ATSA}} $表示ATSA模块的输出特征.
ATSA通过多分支结构分别学习不同时间尺度的运动特征,然后对其进行动态融合,可有效学习和捕捉不同人体行为的子动作特征.
学习人体行为的子动作特征是行为识别的基础,同时学习不同子动作间的关联与不同骨架关节点间的长时相关性同样重要. 受长时运动领域知识指导,提出MTCF. 如图4所示,MTCF模块首先采用多尺度时间卷积进行时间特征建模,对ATSA模块的短时子动作建模进行补充,然后引入一种骨架特征融合机制来加强关节点特征和骨架特征之间的动态交互.
首先,给定输入特征${{\boldsymbol{X}}_{\rm in}} \in {\mathbb{R}^{C \times T \times V}}$,对${{\boldsymbol{X}}_{\rm in}}$在关节点维度进行平均池化操作可以得到包含全局信息的骨架特征:
$ {{\boldsymbol{X}}_{\rm S}} = \frac{1}{V}\sum\limits_{i = 1}^V {{{\boldsymbol{X}}_i}} , $
其中${{\boldsymbol{X}}_{\rm S}}$表示骨架特征,${{\boldsymbol{X}}_i} \in {\mathbb{R}^{C \times T}}$表示第$ i $个关节点特征,$V$表示关节点数量.
然后,骨架特征和输入特征分别经由$1 \times 1$卷积降低通道维度并分为6个分支,进行多尺度时间特征学习. 多尺度时间卷积(multiscale temporal convolution,MTC)由6个分支组成:$1 \times 1$卷积、$3 \times 1$最大池化以及4个膨胀系数不同的$3 \times 1$卷积. $1 \times 1$卷积可以通过调整权重来实现特征的重新组合和增强;$3 \times 1$最大池化用于捕捉时间序列中的显著帧或运动模式;4个$3 \times 1$卷积的感受野逐渐增大,可以捕获从短期到长期的不同时间尺度的信息. 以上分支共同作用,使模型能够学习到多尺度的时序依赖关系,提高了模型对时间信息的处理能力. MTC首先将输入特征沿通道维度均分为6组,然后分别由不同分支处理,最后将所有分支的输出特征沿通道维度进行拼接,并采用$1 \times 1$卷积进行特征融合得到MTC的输出特征. MTC计算过程为:
$ {{\boldsymbol{X}}_{{\text{MTC}}}} = Conv2{D_{\left( {1 \times 1} \right)}}\left( {Concat\left( {{{\boldsymbol{X}}_1},{{\boldsymbol{X}}_2}, … ,{{\boldsymbol{X}}_6}} \right)} \right), $
其中${{\boldsymbol{X}}_1},{{\boldsymbol{X}}_2}, … ,{{\boldsymbol{X}}_6}$为6个分支各自的输出特征,$Concat$表示沿通道维度进行拼接操作,$ Conv{\text{2}}{D_{\left( {1 \times 1} \right)}} $表示$1 \times 1$卷积操作.
考虑到多尺度时间卷积仅对每个骨架关节点各自的时间特征进行建模,缺少对骨架关节点之间的长时相关性进行学习. 因此经过多尺度时间特征建模后,关节特征通过一组可学习的动态权重向量与骨架特征进行融合得到输出特征:
$ {{\boldsymbol{X}}_{\rm out}} = MTC\left( {{{\boldsymbol{X}}_{\rm in}}} \right){\text{ + }}MTC\left( {{{\boldsymbol{X}}_{\rm S}}} \right){{\boldsymbol{W}}_j} , $
其中$ {{\boldsymbol{X}}_{\rm out}} $表示MTCF模块的输出特征. ${{\boldsymbol{W}}_j}$表示可学习的权重向量,用于动态融合骨架和关节特征.
通过多尺度时间建模和骨架特征融合,MTCF模块能够有效学习不同子动作特征之间的关联,同时加强不同关节点之间的信息交互.
综上所述,本文提出的融合运动领域知识与自适应时间片段注意力的Transformer方法,包括线性嵌入层、ATSA和MTCF,最后经全局平均池化层、线性层和Softmax层输出预测类别. 该方法可有效提升骨架序列短时运动特征学习和关节点之间信息交互的建模能力,用于人体骨架行为识别.
为验证本文方法的有效性和在不同环境的泛化性,在4个数据集上进行评估与实验,包括人体日常行为数据集NTU RGB+D[17]、数据集NTU RGB+D 120[18]、体育行为数据集FineGym[19]、工业场景行为数据集InHARD[20].
1)NTU RGB+D是室内场景拍摄下的人体日常生活行为数据集,如图5所示包含日常行为、健康相关以及双人交互3种场景的行为类别. 数据集包含60类行为,56 880个样本,每个样本提供RGB视频、深度图序列、3D骨架序列和红外视频4种数据模态. NTU RGB+D提供Xsub(Cross-Subject)和Xview(Cross-View)两种标准来划分训练集和测试集. NTU RGB+D 120数据集为NTU RGB+D的扩展版,共包含120个类别,114 480个样本. 与NTU RGB+D数据集相比,此数据集增加了跑步、胸痛以及搀扶等60类运动、医疗健康等不同场景的人类行为模式. NTU RGB+D 120提供Xsub和Xset(Cross-Setup)2种标准来划分训练集和测试集.
2)FineGym是一个基于细粒度动作理解的数据集,主要针对体育运动中体操领域行为识别分析. 如图6所示,数据集包含跳马、自由体操、平衡木以及高低杠4种体操项目中的99类行为,32 697个样本. FineGym中的样本在背景信息具有高度相似性,可避免模型利用行为无关信息进行识别.
3)InHARD为工业生产环境下的人体行为识别数据集,主要用于涉及人机协作工业环境中的人类行为识别与分析任务. 如图7所示,数据集包含14个工业行为类别,5 303个行为样本. 每个样本提供RGB视频和骨架模态,以及3种不同的视角. InHARD根据执行主体和视角将样本划分为训练集和测试集.
实验环境配置为Intel Xeon Platinum 8160Ts CPU,GPU采用4张显存为24 GB的NVIDIA GeForce RTX 3090Ti,深度学习框架采用Pytorch1.5. 所有消融和对比实验中,模型均采用随机梯度下降优化器进行训练,训练轮数为120,初始学习率设置为0,批次大小设置为32. 实验中的模型参数和计算量采用MMAction2[25]工具箱计算得到.
为验证提出方法中各个模块的有效性、编码器模块层数以及多模态数据融合等对模型性能提升的影响,首先需要对提出的模型进行消融实验[26-27]. 鉴于NTU RGB+D数据集涵盖多种场景下的人体行为种类,并且样本数量较为充足,能够较为全面地反映典型场景中的人体行为特征[7,28]. 因此,为便于性能评估和消融实验对比,与文献[7]一致,本文在NTU RGB+D数据集的Xsub和Xview设置下进行消融实验与分析.
ATSA通过融合短时运动领域知识,能根据不同的行为样本针对性学习和捕捉子动作特征. ATSA模块由多个不同时间分段长度的ISSA模块组成,首先对时间分段长度的影响进行实验验证. 表1所示为ISSA模块的消融实验,其中ISSA(n=1)表示时间片段长度为1的段间自注意力机制,即空间自注意力机制.
由表1可以发现,单独采用ISSA(n=1)模块的情况下,模型在NTU RGB+D数据集中2种设置下的识别准确率分别仅为89.0%和93.8%. 随时间分段长度n逐渐增加,模型的识别准确率逐渐提升,ISSA(n=3)模块相较于ISSA(n=1)模块分别提升1.3个百分点和1.2个百分点. 此外可以发现,当时间分段长度继续增加后(n>3),模型识别准确率逐渐下降,说明对长时间分段进行显式建模会影响模型的短时特征建模能力. 以上实验结果表明基于短时运动领域知识设计的ISSA,通过对相邻骨架帧进行显式建模,能有效对人体行为中的子动作特征进行学习,进而提升模型识别准确率.
表2中给出不同信息融合策略的识别性能对比,其中对平均融合中所有分支的输出特征进行相加后取平均得到融合特征,线性融合中所有分支的输出特征经过线性层得到融合特征. 由表2可以看出,在NTU RGB+D数据集中2种设置下,ATSA采用平均融合策略时的Top1识别准确率分别为90.8%和95.6%. 当采用DIF模块,模型的识别准确率相对于线性融合策略提升0.4个百分点和0.5个百分点,验证了DIF模块的有效性. 这充分说明对于不同的人体行为,每个分支输出特征的重要性不同,因此动态融合表征可以更好地学习不同尺度的短时运动特征.
消融实验结果表明,ATSA模块相较于ISSA对短时运动领域知识的融合更加充分,同时说明针对不同行为类别进行自适应时间分段建模的重要性.
受长时运动领域知识启发,MTCF模块采用多个不同感受野的2D卷积进行长时特征建模,并引入特征融合机制增强不同关节点间的信息交互. 表3为MTCF模块的消融实验,其中多尺度时间卷积(MTC)的具体介绍见2.4节,4分支表示$1 \times 1$卷积、$3 \times 1$最大池化以及2个膨胀系数从1~2的$3 \times 1$卷积;6分支表示$1 \times 1$卷积、$3 \times 1$最大池化以及4个$3 \times 1$卷积.
由表3可以发现,采用MTC(4分支)模块的情况下,模型在NTU RGB+D数据集中2种设置下的识别准确率分别仅为90.2%和94.2%. 将MTC模块的分支增加到6后,模型的识别准确率分别提升0.3个百分点和0.7个百分点,表明更丰富的多尺度时间特征学习有利于提升模型识别准确率.
相较于每个关节点单独进行长时特征建模的MTC模块,MTCF模块通过采用权重向量对骨架特征和节点特征进行融合,以增强模型的长时信息建模能力. 由表3可以发现,采用固定权重融合全局特征的识别准确率分别为90.7%和95.4%,采用动态权重融合全局特征的识别准确率分别为90.8%和95.4%,相对于MTC(4分支)模块分别提升0.6个百分点和1.2个百分点. 这说明MTCF模块,不管是固定权重还是动态权重策略融合方式,均能有效提升模型识别准确率,同时验证了增强关节点长时信息交互的重要性.
更进一步,将MTCF模块同样扩展为6分支后,识别准确率进一步提升至91.0%和95.8%. 这表明长时运动领域知识指导的MTCF模块可通过融合骨架全局特征来有效增强不同关节点之间的信息交互能力,从而提升模型识别准确率.
多流模态融合策略被广泛应用于现有骨架行为识别方法. 不同数据模态有各自的特点,通过融合多模态数据可显著提高识别性能. 与主流方法[7]一致,本文分别使用关节点、骨骼、关节点运动以及骨骼运动模态进行训练,然后按2∶2∶1∶1的比例对模型的推理输出进行加权求和得到最终结果. 骨骼模态数据基于相邻关节点的差值计算得到,关节点运动和骨骼运动模态数据基于相邻帧差值得到. 表4为本文方法采用4种单模态以及4种模态融合的情况下在NTU RGB+D数据集的消融实验结果.
由表4可以看出,任意单模态数据的识别准确率均低于多模态数据融合的结果. 在单一关节点模态下,本文方法在Xsub和Xview设置下的识别准确率分别为91.0%和95.8%. 采用4种模态数据融合的情况下,本文方法在Xsub和Xview设置下的识别准确率分别为93.2%和97.1%,相比于单一关节点模态分别提升2.2个百分点和1.3个百分点. 此外可以发现,尽管关节点运动和骨骼运动2种模态各自的识别效果不佳,但通过多模态数据融合后仍可带来一定识别准确率提升,验证了多模态信息融合的有效性.
为直观观察本文方法对不同行为的识别情况,如图8中给出NTU RGB+D 120数据集上的归一化混淆矩阵. 混淆矩阵的横坐标和纵坐标分别表示预测类别和真实类别,颜色越深表示识别准确率越高.
如图8所示,本文方法对NTU RGB+D 120中的前60类行为的识别准确率较高,第70~80类的部分行为识别准确率低于60%. 其中第73类“订书”的识别准确率最低仅为41%,易混淆为第76类“剪纸”,主要原因在于骨架序列数据缺乏关键的物体交互信息,导致难以准确识别人物交互行为. 第71类“做OK手势”和第72类“做胜利手势”准确识别率分别为57%和56%,互为易混淆类别. 主要原因为这2类行为的主要区别仅取决于手指姿态的细微变化,而NTU RGB+D 120数据集提供的手部关节点较少,导致模型无法获取足够的鉴别信息用于分类决策. 此外,对于需要人体多个关节协同运动的第80类“深蹲”的识别准确率达到98%,充分说明融合长时运动领域知识后,可提升模型对不同关节点之间长时依赖关系的捕捉能力.
为进一步评估基于自适应时间片段注意力的Transformer模型的输出特征对行为识别性能的影响,以及人体运动领域知识的重要性,图9对本文方法在NTU RGB+D数据集中最后一层的不同ISSA分支输出特征进行可视化分析. 图9中横轴表示人体骨架关节点个数,纵轴表示时间索引号,颜色越深表示输出特征的激活值越大.
首先对比2种不同的行为可以发现,跳跃行为的输出特征激活时间区域主要集中在中期,而咳嗽行为的输出特征激活时间区域主要集中在早期,这2种行为各自的时间动态规律一致. 该现象表明通过融合人体运动领域知识,本文方法能够聚焦与行为识别高度相关的骨架帧,减少无关信息的干扰,从而提升模型的鲁棒性. 此外可以发现,不同ISSA分支关注的时间区域不同,能够学习不同尺度的短时运动特征信息. 对于跳跃行为而言,ISSA(n=1)的输出特征时间区域主要分布在第25~50帧,对应行为过程中的整体运动阶段. ISSA(n=3)的输出特征激活时间区域则主要集中在第30~35帧,对应过程中的关键阶段. 这充分说明基于短时运动领域知识设计的ATSA,能够通过多分支结构显式学习不同行为的人体子动作特征,从而有效捕捉人体行为的关键运动信息.
为验证本文方法的有效性和泛化性,在NTU RGB+D,NTU RGB+D 120,FineGym,InHARD数据集上与其他先进代表方法进行对比. 本文在4个数据集上均采用多模态融合策略[7]获得最终的识别结果. 实验结果分别如表5、表6、表7所示.
由表5可以看出,在NTU RGB+D数据集的2种设置下,本文方法的Top1识别准确率分别达到93.2%和97.1%. 在NTU RGB+D 120数据集上的2种设置下,相较于现有的Transformer方法TranSkeleton[34],本文方法分别将Top1识别准确率分别提升0.3个百分点和0.9个百分点. 对比基于GCN的FR-Head[31],HD-GCN[32],基于Transformer的STSA-Net[15],TranSkeleton[34]和基于MLP的SiT-MLP[35]人体行为识别方法,本文方法的Top1识别准确率在NTU RGB+D和NTU RGB+D 120这2个数据集上均具有可比性,充分说明融合人体运动领域知识的重要性. 此外,对比最新基于GCN的DEGCN[33]方法,本文方法在识别准确率相近的情况下,模型参数降低55%,表明了本文方法的有效性.
相比于GCN,Transformer结构的引入往往会进一步增加算法复杂度. 对比GCN与Transformer两类架构模型可以发现,基于Transformer架构的方法和基于GCN的方法在模型参数量区别不明显,但在计算量上整体明显高于基于GCN的方法. 本文方法采用时间分段的方式进行自注意力机制计算,相较于全局注意力机制能够显著减少模型参数和计算量,同时可对短时子动作特征进行显示建模. 此外,本文方法的模型参数和模型计算量相较于其他Transformer类方法均有一定下降,在识别性能和计算开销之间达到较好的平衡.
由表6可以看出,本文方法在FineGym数据集的Top1和平均Top1识别准确率分别为95.0%和93.1%. 对比最新的MDR-GCN[36]方法,本文方法的Top1识别准确率提升0.5个百分点. 对比最新的3D Deformable Transformer[44]的方法,本文方法的平均Top1识别准确率提升2.8个百分点,充分说明了本文方法的有效性. 由于FineGym中的行为样本在外观和背景信息上较为类似,并且大多数行为类别具有相同的子动作,因此本文方法通过融合短时运动领域知识对人体子动作特征进行显式建模,能够有效学习不同行为与子动作之间的关联,从而提升模型识别准确率. 为进一步评估本文方法在不同场景的表现和泛化能力,在工业生产人体行为识别数据集InHARD上与其他先进代表方法进行对比. InHARD为工业生产人体行为识别数据集,样本数量较少,可评估模型在不同领域的泛化能力. 由表7可以看出,本文方法在InHARD数据集中Top1识别准确率为88.3%,优于TSM[41]和VideoSwin[46]. 此外,本文方法采用骨架模态进行人体行为识别,在取得更高识别性能的情况下保持较低的计算量和模型参数. 结果表明,本文方法能够很好地泛化至工业生产领域.
现有人体骨架行为识别方法对人体运动学领域知识利用不足,并缺少对短时子动作特征的显式建模,导致局部特征没有得到充分利用,造成内在人类可理解的逻辑决策解释性不足. 为此本文提出融合运动领域知识和自适应时间片段注意力的Transformer模型用于人体骨架行为识别. 首先,受短时运动领域知识启发,提出ATSA学习短时子动作特征. ATSA通过多分支结构对不同时间尺度的局部运动特征进行建模,并通过动态特征融合整合多个分支的短时特征信息. 然后,融合长时运动领域知识,提出MTCF用于学习子动作之间的关联和增强关节点的长时信息交互.
4个公开数据集的性能评估和实验结果表明,本文方法可有效提升对骨架序列局部动作特征的建模能力,增加不同关节点之间的信息交互能力,从而提高人体骨架行为识别的识别准确率.
作者贡献声明:梁成武提出了方法思路和实验方案; 蒋松琪、杨杰执行主要实验;朱培旺、帖云、胡伟负责论文修订、补充和实验分析;梁成武、杨杰撰写论文;高磊、郭文博提出指导意见并修改论文.

参考文献 引证文献
排序方式:
1
裴利沈, 刘少博, 赵雪专. 人体行为识别研究综述[J]. 计算机科学与探索, 2022, 16(2): 305−322
Pei Lishen, Liu Shaobo, Zhao Xuezhuan. Review of human behavior recognition research[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(2): 305−322 (in Chinese)
2
李洪均, 丁宇鹏, 李超波, 等. 基于特征融合时序分割网络的行为识别研究[J]. 计算机研究与发展, 2020, 57(1): 145−158
Li Hongjun, Ding Yupeng, Li Chaobo, et al. Action recognition of temporal segment network based on feature fusion[J]. Journal of Computer Research and Development, 2020, 57(1): 145−158 (in Chinese)
3
卢健, 李萱峰, 赵博, 等. 骨骼信息的人体行为识别综述[J]. 中国图象图形学报, 2023, 28(12): 3651−3669
Lu Jian, Li Xuanfeng, Zhao Bo, et al. A review of skeleton-based human action recognition[J]. Journal of Image and Graphics, 2023, 28(12): 3651−3669 (in Chinese)
4
Yan Sijie, Xiong Yuanjun, Lin Dahua. Spatial temporal graph convolutional networks for skeleton-based action recognition[C]//Proc of the 32nd AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2018: 7444−7452
5
Shi Lei, Zhang Yifan, Cheng Jian, et al. Two-stream adaptive graph convolutional networks for skeleton-based action recognition[C]//Proc of the 28th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2019: 12026−12035
6
Cheng Ke, Zhang Yifan, He Xiangyu, et al. Skeleton-based action recognition with shift graph convolutional network[C]//Proc of the 29th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 183−192
7
Chen Yuxin, Zhang Ziqi, Yuan Chunfeng, et al. Channel-wise topology refinement graph convolution for skeleton-based action recognition[C]//Proc of the 18th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2021: 13359−13368
8
曹毅, 吴伟官, 张小勇, 等. 基于自校准机制的时空采样图卷积行为识别模型[J]. 工程科学学报, 2024, 46(3): 480−490
Cao Yi, Wu Weiguan, Zhang Xiaoyong, et al. Action recognition model based on the spatiotemporal sampling graph convolutional network and self-calibration mechanism[J]. Chinese Journal of Engineering, 2024, 46(3): 480−490 (in Chinese)
9
Plizzari C, Cannici M, Matteucci M. Skeleton-based action recognition via spatial and temporal transformer networks[J]. Computer Vision and Image Understanding, 2021, 208/209: 103219−103228
10
Shi Lei, Zhang Yifan, Cheng Jian, et al. Decoupled spatial-temporal attention network for skeleton-based action-gesture recognition[C]//Proc of the 15th Asian Conf on Computer Vision. Berlin: Springer, 2020: 38−53
11
石跃祥, 朱茂清. 基于骨架动作识别的协作卷积Transformer网络[J]. 电子与信息学报, 2019, 45(4): 1485−1493
Shi Yuexiang, Zhu Maoqing. Collaborative convolutional Transformer network based on skeleton action recognition[J]. Journal of Electronics and Information Technology, 2019, 45(4): 1485−1493 (in Chinese)
12
Zhang Shiwen, Guo Sheng, Wang Liming, et al. Knowledge integration networks for action recognition[C]//Proc of the 34th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2020: 12862−12869
13
Wang Liming, Tong Zhan, Ji Bin, et al. TDN: Temporal difference networks for efficient action recognition[C]//Proc of the 30th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2021: 1895−1904
14
梁成武, 胡伟, 杨杰, 等. 融合时空领域知识与数据驱动的骨架行为识别[J]. 计算机工程与应用, 2025, 61(5): 165−176
Liang Chengwu, Hu Wei, Yang Jie, et al. Integration of spatiotemporal domain knowledge and data-driven skeleton behavior recognition[J]. Computer Engineering and Applications, 2025, 61(5): 165−176 (in Chinese)
15
Qiu Helei, Hou Biao, Ren Bo, et al. Spatio-temporal segments attention for skeleton-based action recognition[J]. Neurocomputing, 2023, 518: 30−38
16
Duan Haodong, Wang Jiaqi, Chen Kai, et al. DG-STGCN: Dynamic spatial-temporal modeling for skeleton-based action recognition[J]. arXiv preprint, arXiv: 2210.05895, 2022
17
Shahroudy A, Liu Jun, Ng T T, et al. NTU RGB+D: A large scale dataset for 3D human activity analysis[C]//Proc of the 25th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2016: 1010−1019
18
Liu Jun, Shahroudy A, Perez M, et al. NTU RGB+D 120: A large-scale benchmark for 3D human activity understanding[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 42(10): 2684−2701
19
Shao Dian, Zhao Yue, Dai Bo, et al. FineGym: A hierarchical video dataset for fine-grained action understanding[C]//Proc of the 29th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 2616−2625
20
Dallel M, Havard V, Baudry D, et al. Inhard-industrial human action recognition dataset in the context of industrial collaborative robotics[C]//Proc of the 1st IEEE Int Conf on Human-Machine Systems (ICHMS). Piscataway, NJ: IEEE, 2020: 1−6
21
Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[J] arXiv preprint, arXiv: 2010.11929, 2021
22
Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proc of the 31st Int Conf on Neural Information Processing Systems. Long Beach, CA: Curran Associates Inc, 2017: 6000−6010
23
Bertasius G, Wang Heng, Torresani L. Is space-time attention all you need for video understanding?[C/OL]//Proc of the 38th Int Conf on Machine Learning. 2021[2025-04-10]. https://proceedings.mlr.press/v139/bertasius21a.html
24
Li Xiang, Wang Wenhai, Hu Xiaolin, et al. Selective kernel networks[C]//Proc of the 28th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2019: 510−519
25
MMAction2 Contributors. Openmmlab’s next generation video understanding toolbox and benchmark[CP/OL]. 2020[2025-04-10]. https://github.com/open-mmlab/mmaction2
26
Li Maosen, Chen Siheng, Chen Xu, et al. Actional-structural graph convolutional networks for skeleton-based action recognition[C]//Proc of the 28th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2019: 3595−3603
27
Liu Ziyu, Zhang Hongwen, Chen Zhenghao, et al. Disentangling and unifying graph convolutions for skeleton-based action recognition[C]//Proc of the 29th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 143−152
28
Chi H, Ha M H, Chi S, et al. InfoGCN: Representation learning for human skeleton-based action recognition[C]//Proc of the 31st IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 20186−20196
29
曹毅, 夏宇, 高清源, 等. 基于超连接图卷积网络的骨架行为识别方法[J/OL]. 吉林大学学报: 工学版, 2025, [2025-04-12]. https://doi.org/10.13229/j.cnki.jdxbgxb.20230440
Cao Yi, Xia Yu, Gao Qingyuan, et al. Based on super connection diagram convolution network skeleton behavior recognition method[J/OL]. Journal of Jilin University: Engineering Science, 2025,[2025-04-12]. https://doi.org/10.13229/j.cnki.jdxbgxb.20230440 (in Chinese)
30
Jang S, Lee H, Kim W J, et al. Multi-scale structural graph convolutional network for skeleton-based action recognition[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2024, 34(8): 7244−7258
31
Zhou Huanyu, Liu Qingjie, Wang Yunhong. Learning discriminative representations for skeleton based action recognition[C]//Proc of the 32nd IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023: 10608−10617
32
Lee J, Lee M, Lee D, et al. Hierarchically decomposed graph convolutional networks for skeleton-based action recognition[C]//Proc of the 19th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2023: 10444−10453
33
Myung W, Su Nan, Xue Jinhao, et al. DEGCN: Deformable graph convolutional networks for skeleton-based action recognition[J]. IEEE Transactions on Image Processing, 2024, 33: 2477−2490
34
Liu Haowei, Liu Yongcheng, Chen Yuxin, et al. TranSkeleton: Hierarchical spatia-temporal Transformer for skeleton-based action recognition[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(8): 4137−4148
35
Zhang Shaojie, Yin Jianqi, Dang Yonghao, et al. SiT-MLP: A simple MLP with point-wise topology feature learning for skeleton-based action recognition[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2024, 34(9): 8122−8134
36
Liu Shenlan, Ding Yuning, Zhang Jingro, et al. Multidimensional refinement graph convolutional network with robust decouple loss for fine-grained skeleton-based action recognition[J]. IEEE Transactions on Neural Networks and Learning Systems, 2025, 36(4): 7615−7626
37
Girdhar R, Ramanan D, Gupta A, et al. ActionVLAD: Learning spatio-temporal aggregation for action classification[C]//Proc of the 26th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2017: 971−980
38
Carreira J, Zisserman A. Quo vadis, action recognition? A new model and the kinetics dataset[C]//Proc of the 26th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2017: 6299−6308
39
Wang Liming, Xiong Yuanjun, Wang Zhe, et al. Temporal segment networks: Towards good practices for deep action recognition[C]//Proc of the 11th European Conf on Computer Vision. Berlin: Springer, 2016: 20−36
40
Zhou B, Andonian A, Oliva A, et al. Temporal relational reasoning in videos[C]//Proc of the 13th European Conf on Computer Vision. Berlin: Springer, 2018: 803−818
41
Lin Ji, Gan Chuang, Han Song. TSM: Temporal shift module for efficient video understanding[C]//Proc of the 17th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2019: 7083−7093
42
Kim M, Kwon H, Wang Chunyu, et al. Relational self-attention: What’s missing in attention for video under-standing[C]//Proc of the 35th Int Conf on Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc, 2021: 8046−8059
43
Shi Jing, Zhang Yuanyuan, Wang Weihang, et al. A novel two-stream Transformer-based framework for multi-modality human action recognition[J]. Applied Sciences, 2023, 13(4): 2058
44
Kim S, Ahn D, Ko B C. Cross-modal learning with 3D deformable attention for action recognition[C]//Proc of the 19th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2023: 10265−10275
45
Wang Xiaolong, Girshick R, Gupta A, et al. Non-local neural networks[C]//Proc of the 27th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2018: 7794−7803
46
Liu Ze, Ning Jia, Cao Yue, et al. Video swin Transformer[C]//Proc of the 31st IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 3202−3211
2025年第62卷第12期
139
62
0
引用本文
BibTeX
文章信息
doi: 10.7544/issn1000-1239.202440558
  • 接收时间:2024-06-21
  • 首发时间:2026-11-10
  • 出版时间:2025-12-01
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2024-06-21
  • 修回日期:2025-04-22
  • 录用日期:2025-06-11
基金
作者信息
    1河南城建学院电气与控制工程学院 河南平顶山 467041
    2三峡大学电气与新能源学院 湖北宜昌 443002
    3国网湖南省电力有限公司怀化供电分公司 湖南怀化 418006
    4国网安徽省电力有限公司铜陵供电分公司 安徽铜陵 244151
    5郑州大学电气与信息工程学院 郑州 450001
    6多伦多都会大学电气计算机和生物医学工程系 加拿大多伦多 M5B2K3
参考文献
分享链接
https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202440558
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏