Research on Pedestrian Detection Based on Multimodal Infor-mation Fusion

Abstract

The automatic driving system based on a single-mode sensor is susceptible to the external environment in pedestrian detection. This paper proposes a fusion of light and thermal infrared multimodal pedestrian detection methodology. Firstly, 1 × 1 convolution and dilated convolution square measure are introduced within the residual network, and also the ROIAlign methodology is employed to exchange the ROIPooling methodology to map the candidate box to the feature layer to optimize the Faster R-CNN. Secondly, the generalized intersection over union (GIoU) loss function is employed as the loss function of prediction box positioning regression. Finally, to explore the performance of multimodal image pedestrian detection methods in different fusion periods in the improved Faster R-CNN, four forms of multimodal neural network structures are designed to fuse visible and thermal infrared pictures. Experimental results show that the proposed algorithm performs better on the KAIST dataset than current mainstream detection algorithms. Compared to the conventional ACF + T + THOG pedestrian detector, the AP is 8.38 percentage points greater. The miss rate is 5.34 percentage points lower than the visible light pedestrian detector.