AMF-SparseInst: Attention-guided Multi-Scale Feature fusion network based on SparseInst

Abstract

SparseInst will generate redundant background features in multi-scale feature fusion, which will cause feature loss for small objects with low resolution and similar pixels to the background. To address the issue, we propose a real-time instance segmentation model named AMF-SparseInst (Attention-guided Multi-Scale Feature SparseInst), which can effectively highlight the most critical features of small objects from cluttered backgrounds. Firstly, we design a pyramid pooling module (called SimAM-ASPP), which consists of some depthwise separable convolutions with three different expansion rates and a 3D attention mechanism (called SimAM). It can capture contextual information from different receptive fields and focus on small object features. Secondly, we designed the Lite -BiFPN module to associate and integrate different levels of semantic information from top to bottom and from bottom to bottom. Finally, we propose a feature enhancement module FEM, which uses N3 and N5 respectively to reweight fusion features in spatial and channel dimensions to enhance the effective information of multi-scale fusion features. Experimental results demonstrate the superiority of AMF-SparseInst over the benchmark on COCO 2017 test-dev. Specifically, the AMF-SparseInst makes a 3.6% improvement in overall segmentation accuracy, while increasing speed by 2.5 FPS. Moreover, it achieves a favorable balance between accuracy and speed on the Cityscapes validation set.