Structure-preserving Style Transfer

Abstract

Transferring different artistic styles to images while preserving their content is a difficult image processing task. Since the seminal deep learning approach of Gatys et al. [GEB16], many recent works have proposed different approaches for performing this task. However, most of them share one major limitation: a trade-off between how much the target style is transferred, and how much the content of the original source image is preserved [GEB16, GEB*17, HB17, LPSB17]. In this work, we present a structure-preserving approach for style transfer that builds on top of the approach proposed by Gatys et al. Our approach allows to preserve regions of fine detail by lowering the intensity of the style transfer for such regions, while still conveying the desired style in the overall appearance of the image. We propose to use a quad-tree image subdivision, and then apply the style transfer operation differently for different subdivision levels. Effectively, this leads to a more intense style transfer in large flat regions, while the content is better preserved in areas with fine structure and details. Our approach can be easily applied to different style transfer approaches as a post-processing step.