Natural Interaction in Virtual Heritage: Enhancing User Experience with Large Language Models

A novel virtual reality (VR) system enhances user experience at UNESCO World Heritage Site San Cristóbal de La Laguna by integrating Large Language Models (LLMs) to enable natural, unscripted dialogue with virtual humans. This approach, combining Speech-to-Text and Text-to-Speech synthesis, prioritizes usability, immersion, and dialogue quality while addressing user-centered design principles. Preliminary studies highlighted the importance of historically adapted language and feedback mechanisms for virtual guides, with a qualitative test revealing preferences for gender selection options. Although limitations such as latency in dialogue generation and reduced performance in non-English languages persist, the research offers a generalist methodology for unscripted virtual human interactions in VR applications, providing actionable recommendations for future implementations.

Sánchez-Berriel, I., Pérez-Nava, F., & Pérez-Rosario, L. (2025). Natural Interaction in Virtual Heritage: Enhancing User Experience with Large Language Models. Electronics, 14(12), 2478. https://doi.org/10.3390/electronics14122478