MS-SACNet: Multi-Scale Self-Attention Convolution network for vision-based anti-misoperation system

Abstract

Keypoint detection and pose estimation are pivotal in various industrial applications such as vision-based anti-misoperation systems. This system typically incorporates computer vision techniques, including keypoint detection models and object detection, where precision and robustness are critical to the effectiveness and reliability of the system. Advanced deep learning models like YOLO, HRNet and EfficientNet are commonly employed in these applications, but challenges remain due to dynamic scale variations and the requirement for high accuracy and reliability. To address this issue, this paper proposed a Multi-Scale Self-Attention Convolution based keypoint detection network(MS-SACNet), which enhances keypoint localization accuracy by effectively extracting fine-grained features from the input feature map, enabling more precise detection across varied scales. Additionally, we incorporate a generative probabilistic Gaussian Mixture Model (GMM) to enhance the model's detection capability by involving feature distributions during the training stage. The cross-domain data augmentation techniques further improve the model's performance by exposing it to diverse scenarios and noise, thereby improving its robustness in real-world conditions. Our proposed method has been integrated into the deployed system, and its effectiveness has been validated using the both public datasets such as VOC2007 and VOC2012, and our self-made industrial datasets. For example, by integrating MS-SACNet, our framework outperforms YOLOv5 by 5.2mAPpose and 2.5mAPbox, demonstrating superior effectiveness and reliability.