born in 1982. PhD, professor, master supervisor. Member of CCF. His main research interests include computer vision, machine learning, and multimedia processing
born in 1999. Master candidate. His main research interests include target detection and human action recognition
born in 1998. Master. His main research interests include computer vision and multi-modality information fusion
born in 2000. Master candidate. His main research interest includes computer vision
born in 1974. PhD, professor, PhD supervisor. His main research interests include intelligent multimedia, and virtual reality and interaction
born in 1983. PhD, associate professor. His main research interests include computer vision and multimedia processing
born in 1999. Master. His main research interest includes smart grid
born in 1994. PhD, lecturer. His main research interests include computer vision and image processing
Existing methods for skeleton-based human action recognition always ignore motion domain knowledge, resulting in the lack of interpretability of logical decision-making that human can understand. In this paper, we propose a novel skeleton-based human action recognition method by fusion of domain knowledge and adaptive spatio-temporal Transformer, to improve recognition performance and interpretability. Firstly, inspired by the short-term motion knowledge, a temporal multi-branch structure is designed to learn and capture the characteristics of short-term sub-acitons. Secondly, a dynamic information fusion module is proposed to learn the weight vectors of different temporal branches, and then fuse multiscale short-term motion features. Finally, to learn the relationship between different sub-actions and facilitate the motion information interaction between skeleton joints, a multiscale temporal convolution feature fusion module is proposed to capture the long-term motion correlations, by integrating the domain knowledge of the long-term motion. Experimental evaluations are conducted on four large action datasets, including NTU RGB+D, NTU RGB+D 120, FineGym, and InHARD. The experimental results show that the recognition performance of the proposed method is superior to several data-driven methods, effectively improving the modelling ability of short-term motion feature learning and information interaction between skeleton joints, with the interpretability.
| 科 Family | 属数 Number of genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) | 属 Genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) |
|---|---|---|---|---|---|---|
| 鹅膏菌科Amanitaceae | 2 | 11 | 5.26 | 鹅膏菌属 Amanita | 10 | 4.78 |
| 小菇科 Mycenaceae | 2 | 12 | 5.74 | 丝盖伞属 Inocybe | 5 | 2.39 |
| 多孔菌科 Polyporaceae | 8 | 14 | 6.70 | 蜡蘑属 Laccaria | 5 | 2.39 |
| 红菇科 Russulaceae | 3 | 23 | 11.00 | 小皮伞属 Marasmius | 6 | 2.87 |
| 小菇属 Mycena | 11 | 5.26 | ||||
| 光柄菇属 Pluteus | 5 | 2.39 | ||||
| 红菇属 Russula | 17 | 8.13 | ||||
| 栓菌属 Trametes | 5 | 2.39 |