Stylistic Patterns for Generating Cinematographic Sequences

Abstract

For film editors, the decision of how to compose and sequence camera framings is a question pertaining to a number of elements involving the semantics of shots, framings, story context, consistency of style, and artistic value. AI systems have brought a number of techniques to create procedural generative systems for game animation and narrative content. However, due to its computational complexity, current automated cinematography relies heavily on constraint and rule-based systems, or pre-calculated camera positions and movements that implement well-known idioms from traditional cinematography. Existing dynamic systems only have limited reaction to complex story content and cannot bring affective emotional depth to the scenario. Yet in actual filmmaking, directors often employ camera techniques, which are arrangements of shots and framings, to convey multiple levels of meanings in a sequence. In this paper we propose a language for defining high-level camera styles called Patterns, which can express the aesthetic properties of framing and shot sequencing, and of camera techniques used by real directors. Patterns can be seen as the semantics of camera transitions from one frame to another. The language takes an editors view of on-screen aesthetic properties: the size, orientation, relative position, and movement of actors and objects across a number of shots. We illustrate this language through a number of examples and demonstrations. Combined with camera placement algorithms, we demonstrate the language's capacity to create complex shot sequences in data-driven generative systems for 3D storytelling applications.