Status-Indicator Design and Reference Architectures for LLM-Driven Conversational Avatars in VR
An empirical VR study and system design for conversational avatars driven by a locally deployed large language model integrated with automatic speech recognition, text-to-speech, and lip-syncing. A pilot study compares three avatar status indicators shown during the LLM's response-generation latency (per-state lights, a loading bar, and no feedback) and derives design considerations for perceived responsiveness and realism in LLM-driven conversational systems. The work also documents two integration architectures: an LLM-based state machine that controls avatar behavior, and a retrieval-augmented-generation pipeline that grounds responses in external context.
Takeaways from Applying LLM Capabilities to Multiple Conversational Avatars in a VR Pilot Study
We present a virtual reality (VR) environment featuring conversational avatars powered by a locally-deployed LLM, integrated with automatic speech recognition (ASR), text-to-speech (TTS), and lip-syn…