Reconstructing Baseball Pitching Motions from Video

Abstract

Baseball is one of the most loved sports in the world. In baseball game, the pitcher's control ability is a key factor for determining the outcome of the game. There are a lot of video data shooting baseball games, and learning baseball pitching motions from video can be possible thanks to the pose estimation techniques. However, reconstructing pitching motions using pose estimators is challenging. When we watch a baseball game, motion blur occurs inevitably because the pitcher throws a ball into the strike zone as fast as possible. To tackle this problem, We propose a framework using physics simulation and deep reinforcement learning to reconstruct baseball pitching motions based on unsatisfactory poses estimated from video. We set the target point and design rewards to encourage the character to throw the ball to the target point. Consequently, we can reconstruct plausible pitching motion.