Video Summarization by Deep Visual and Categorical Diversity

PEDRO ATENCIO; GERMAN SANCHEZ; JOHN BRANCH; CLAUDIO DELRIEUX

IET CONTROL THEORY AND APPLICATIONS

INST ENGINEERING TECHNOLOGY-IET

Año: 2019

1751-8644

We propose a video summarization method based on visual and categorical diversity by transfer learning. Our methodextracts visual and categorical features from a pre-trained deep convolutional network (DCN) and a pre-trained word embeddingmatrix. Using visual and categorical information we obtain video diversity, which it is used as an importance score to selectsegments from the input video that best describes it. Our method also allows to perform queries during the search process, in thisway personalizing the resulting video summaries according to the particular intended purposes. The performance of the methodis evaluated using different pre-trained DCN models in order to select the architecture with the best throughput. We then compareit with other state-of-art proposals in video summarization using a data-driven approach with the public dataset SumMe, whichcontains annotated videos with per-fragment importance. The results show that our method outperforms other proposals in mostof the examples. As an additional advantage our method requires a simple and direct implementation that does not require atraining stage.