Interactive Video Completion with SiamMask

Abstract

In this project, we are developing a method to quickly and easily perform video completion. Under the proposed method, the user specifies a target object by drawing a bounding box around it in the first frame of the video; this bounding box is taken as input by SiamMask. SiamMask then tracks the target object and produces its mask in each frame. The resulting masks are then taken as input by Interactive Video Completion, which produces the final video completion result. SiamMask and Interactive Video Completion take several seconds to process an 80-frame video at a pixel resolution of 854x480, i.e., these methods are computationally efficient. The goal of this project is that, after drawing the bounding box, the user immediately obtains the video completion result. However, the mask produced by our current method is not always perfect. When imperfections arise, the user still has to manually modify the mask using an image editing software. We want to improve the quality of automatically produced masks and further reduce the burden of manual modification on the user in the near future.