Video Shot Analysis for Digital Curation and Preservation of Historical Films

Abstract

In automatic video analysis and film preservation, Shot Boundary Detection (SBD) and Shot Type Classification (STC) are fundamental pre-processing steps. While previous research focuses on detecting and classifying shots in different video genres such as sports movies, documentaries or news clips only few studies investigate on SBD and STC in historical footage. In order to promote research on automatic video analysis the project Visual History of the Holocaust (VHH) has been started in January 2019. The main aim of this paper is to present first results on the fundamental topics SBD and STC in the context of the project VHH. Therefore, a deep learning-based SBD approach is implemented to detect Abrupt Transitions (ATs). Furthermore, a CNN-based algorithm is analyzed and optimized in order to classify shots into the four categories: Extreme-Long-Shot (ELS), Long-Shot (LS), Medium-Shot (MS) and Close-Up (CU). Finally, both algorithms are evaluated on a self-generated historical dataset related to the National Socialism and the Holocaust. The outcome of this paper demonstrates a first quantitative evaluation of the SBD approach and displays a F1;Score of 0.866 without the need of any re-training or optimization. Moreover, the proposed STC algorithm reaches an accuracy of 0.71 on classifying shots. This paper contributes a significant base for future research on automatic shot analysis related to the project VHH.