Nondriving behavior identification is one of the important ways to improve the safety of driving. The current recognition method based on skeleton sequence and image fusion has the problems of large model calculation and the difficulty of feature fusion. To address the above problems, the skeletonimage based behavior recognition network (SIBBRNet) is proposed in this paper, which is based on the multiscale skeleton graph and the local visual context. SIBBRNet fully extracts motion and appearance features through a graph convolution network based on multiscale skeleton graphs and a convolutional neural network based on local vision and attention mechanisms, and better balances the relationship between model representation capabilities and model calculation. The feature bidirectional guided learning strategy based on hand motion, an adaptive feature fusion module and an auxiliary loss on the static feature space can guide mutual guidance and updating between motion and appearance features to achieve adaptive fusion. SIBBRNet is finally tested on the Drive & Act dataset, and the average accuracy is 61.78% for dynamic labels and 80.42% for static labels. The Floatingpoint Operations per Second (FLOPS) of SIBBRNet is 25.92G, which is 76.96% lower than that of the optimal method.
| 科 Family | 属数 Number of genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) | 属 Genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) |
|---|---|---|---|---|---|---|
| 鹅膏菌科Amanitaceae | 2 | 11 | 5.26 | 鹅膏菌属 Amanita | 10 | 4.78 |
| 小菇科 Mycenaceae | 2 | 12 | 5.74 | 丝盖伞属 Inocybe | 5 | 2.39 |
| 多孔菌科 Polyporaceae | 8 | 14 | 6.70 | 蜡蘑属 Laccaria | 5 | 2.39 |
| 红菇科 Russulaceae | 3 | 23 | 11.00 | 小皮伞属 Marasmius | 6 | 2.87 |
| 小菇属 Mycena | 11 | 5.26 | ||||
| 光柄菇属 Pluteus | 5 | 2.39 | ||||
| 红菇属 Russula | 17 | 8.13 | ||||
| 栓菌属 Trametes | 5 | 2.39 |