NeuroDog: Quadruped Embodiment using Neural Networks

Abstract

Virtual reality (VR) allows us to immerse ourselves in alternative worlds in which we can embody avatars to take on new identities. Usually, these avatars are humanoid or possess very strong anthropomorphic qualities. Allowing users of VR to embody non-humanoid virtual characters or animals presents additional challenges. Extreme morphological differences and the complexities of different characters’ motions can make the construction of a real-time mapping between input human motion and target character motion a difficult challenge. Previous animal embodiment work has focused on direct mapping of human motion to the target animal via inverse kinematics. This can lead to the target animal moving in a way which is inappropriate or unnatural for the animal type. We present a novel real-time method, incorporating two neural networks, for mapping human motion to realistic quadruped motion. Crucially, the output quadruped motions are realistic, while also being faithful to the input user motions.We incorporate our mapping into a VR embodiment system in which users can embody a virtual quadruped from a first person perspective. Further, we evaluate our system via a perceptual experiment in which we investigate the quality of the synthesised motion, the system’s response to user input and the sense of embodiment experienced by users. The main findings of the study are that the system responds as well as traditional embodiment systems to user input, produces higher quality motion and users experience a higher sense of body ownership when compared to a baseline method in which the human to quadruped motion mapping relies solely on inverse kinematics. Finally, our embodiment system relies solely on consumer-grade hardware, thus making it appropriate for use in applications such as VR gaming or VR social platforms.