DTRSF-Net: Dual-Temporal RGB-Skeleton Fusion for Fitness Action Recognition on National Fitness Platforms

Abstract

The national fitness platform is moving toward an intelligent coaching stage. Action recognition therefore needs to remain stable and reliable for users of different ages, physical abilities, and exercise types. Existing action recognition methods in fitness scenarios tend to be affected by local occlusion, lighting changes, and complex backgrounds. In addition, methods relying only on a single video modality encounter difficulties in balancing high accuracy with real-time inference efficiency in platform applications. To address these problems, this study proposes a Dual-Temporal RGB-Skeleton Fusion Network (DTRSF-Net) for fitness action recognition, which combines a video dual temporal sampling branch with a skeleton branch. The front end first slices the video and forms a sparse sampling sequence and a dense sampling sequence. It also performs keypoint estimation to obtain a normalized skeleton sequence. The back end uses dual branch three-dimensional convolution encoding to extract spatiotemporal features. It also introduces a skeleton action encoding network to obtain posture structure information. A fusion head then produces multimodal decision outputs. Experimental results show that after 200 training epochs, the Top-1 accuracy of DTRSF-Net reaches 98.04%. The comparison and baseline models achieved 94.29% and 90.82%, respectively. The Top-5 accuracy reaches 99.70%. The comparison model and the baseline model achieve 98.40% and 96.79%. The results indicate that multimodal collaborative spatiotemporal feature learning achieves higher recognition accuracy in fitness action recognition tasks. The method provides intelligent action recognition capability for the national fitness platform across multiple scenarios. It also offers stable data support for functions such as online correction, action counting, and training records.