Colorization by Example

Abstract

We present a new method for colorizing grayscale images by transferring color from a segmented example image. Rather than relying on a series of independent pixel-level decisions, we develop a new strategy that attempts to account for the higher-level context of each pixel. The colorizations generated by our approach exhibit a much higher degree of spatial consistency, compared to previous automatic color transfer methods [WAM02]. We also demonstrate that our method requires considerably less manual effort than previous user-assisted colorization methods [LLW04]. Given a grayscale image to colorize, we first determine for each pixel which example segment it should learn its color from. This is done automatically using a robust supervised classification scheme that analyzes the low-level feature space defined by small neighborhoods of pixels in the example image. Next, each pixel is assigned a color from the appropriate region using a neighborhood matching metric, combined with spatial filtering for improved spatial coherence. Each color assignment is associated with a confidence value, and pixels with a sufficiently high confidence level are provided as micro-scribbles to the optimization-based colorization algorithm of Levin et al. [LLW04], which produces the final complete colorization of the image.