Interactive, Example-driven Synthesis and Manipulation of Visual Media

Abstract

This thesis proposes several novel techniques for interactive, example-driven synthesis and manipulation of visual media. The numerous display devices in our everyday lives make visual media, such as images, videos, or three-dimensional models, easily accessible to a large group of people. Consequently, there is a rising demand for efficient generation of syn- thetic visual content and its manipulation, especially by casual users operating on low-end, mobile devices. Off-the-shelf software supporting such tasks typically requires extensive training and in-depth understanding of the underlying concepts of content acquisition, on the one hand, and runs only on powerful desktop machines, on the other hand, limiting the possibility of artistic media generation to a small group of trained experts with appropriate hardware. Our proposed techniques aim to alleviate these requirements by allowing casual users to synthesize complex, high-quality content in real-time as well as to manipulate it by means of simple, example-driven interactions. First, this thesis discusses a manipulation technique that visualizes an additional level of information, such as importance, on images and three-dimensional surface models by local, non-uniform, and self-intersection-free size manipulations. Second, we propose a technique to automatically arrange and sort collections of images based on the images’ shape and a sparse set of exemplar images that builds on a novel distribution algorithm. Along this line, an extension for higher dimensions such as three-dimensional models is presented and the implications of distributions for lower-dimensional projections are discussed. Further, the spectral properties of the distributions are analyzed and the results are applied for efficient, high-quality image synthesis. Finally, we suggest an algorithm to extract deformable, three- dimensional content from a two-dimensional video leveraging a simple limb representation that the user sketches onto a sparse set of key frames. All methods build on the availability of massively parallel execution hardware, such as graphics processing units (GPUs), nowadays built also into cheap mobile devices. By mathematical abstraction, parallelization, and task distribution our algorithms achieve a high efficiency that allows running our methods in real-time on low-end devices.