Submodular Pseudo-3D Network for Micro-Video Summarization

Abstract

With the explosion of micro-videos, it is essential to develop video summariza-tion algorithms that simultaneously capture the diverse video content and represent theoriginal video. Many methods employ deep neural networks (DNNs). However, DNNsummarization models do not directly consider summary diversity. On the other hand,submodular functions can be seen as a form of diversity. However, the shallow structureprevents the data representation at a more abstract level. This paper proposes a novelsubmodular pseudo-3D network (SP3D), which equips submodular functions with a multi-layered network for micro-video summarization. Unlike standard DNNs, the proposedSP3D network and the correspondingoptimization method consider the interlock depen-dency among the selected frames, thus improving the diversity. The experimental resultsindicate that the proposed model and optimization method are effective in micro-videosummarization.