Warp-based Near-Regular Texture Analysis for Image-based Texture Overlay

Abstract

Image-based texture overlay or retexturing is the process of augmenting a surface in an image or a video sequence with a new, synthetic texture. Some properties of the original texture such as texture distortion as well as lighting conditions should be preserved for a realistic appearance of the augmented result. One approach would be to estimate a 3-dimensional geometry of the surface. However, this is an ill-posed problem for complex deformed surfaces like cloth, especially if only one image is given. In an image-based approach, these properties are directly estimated from the image. The key challenge is to separate the shading information from the actual local texture and to retrieve the texture distortion from an image without any knowledge of the underlying scene. In this paper, we model an image of a deformed regular texture as a combination of its deformed surface albedo, a shading map and additional high frequency details. We present a method for determination of these intrinsic parts of a given texture image by first estimating the appearance of a small texture element and then synthesizing a reference image of the undeformed regular texture. In a subsequent image-based optimization method this reference image is iteratively warped spatially and photometrically onto the original image whilst estimating deformation and illumination parameters. The decomposition is used to create images of new textures with the same deformation and illumination properties as in the original image.