With Lily: Design and Implementation of Verbal and Nonverbal Interaction with a Virtual Humanoid in Mixed Reality for Affective Communication

초록

This demo presents a virtual humanoid that enables affective communication through verbal and nonverbal interaction in a mixed reality (MR) environment, showcased as an experiential exhibition titled With Lily. The aim of this work is to provide people - who desire social interaction yet feel burdened by concerns such as evaluation or misunderstanding - with a comfortable and enjoyable virtual conversation that is free from real-world social constraints. To this end, the demo implements a comprehensive system that integrates the technical design of verbal and nonverbal interaction for natural emotional and intentional exchange with an augmented virtual presence. The system further incorporates environment design, conversational narrative, and character personality design to foster users' communication motivation and emotional engagement. © 2026 IEEE.

키워드