Speech and Text-Based Motion Generation and Matching System

초록

Speech and text-based motion generation is an emerging research area in both academia and industry. However, there is a problem that related studies have defined the generated motion structure differently so far. To solve this problem, we propose a novel system for matching the generated motion to SMPL-X. In this study we focused on using joint information from a pre-trained model and matched this model to another model which has different joint structure. Since this system runs on Unity, it can be applied to various industry fields. Detailed motion videos and codes are available in https://github.com/kinsingo/Speech-and-Text-based-Motion-Generation-and-Matching-System © 2023 IEEE.