DIGITAL LIBRARY
HUMAN-ROBOT INTERACTION USING MIXED REALITY AND VISUALIZATION: OVERVIEW AND LESSONS LEARNT FROM TECHNOLOGY, HCI-DESIGN, AND ART
University of Applied Sciences Duesseldorf (GERMANY)
About this paper:
Appears in: INTED2026 Proceedings
Publication year: 2026
Article: 1727
ISBN: 978-84-09-82385-7
ISSN: 2340-1079
doi: 10.21125/inted.2026.1727
Conference name: 20th International Technology, Education and Development Conference
Dates: 2-4 March, 2026
Location: Valencia, Spain
Abstract:
Mobile robots are becoming increasingly versatile, powerful and affordable. Their potential applications range from cleaning and transport tasks to inspections, and they are increasingly being developed and used in public spaces. However, unlike factories with clearly defined environmental parameters and user profiles, public spaces are highly heterogeneous environments where people with different intentions, needs and expectations come together. Developing robotic systems that can be seamlessly integrated into such contexts is therefore complex and challenging, especially given the often opaque design, technical, social and legal framework conditions. We show how different projects can be combined to exploit synergies for the development of human-robot interaction.

The authors are part of a competence centre that deals with the safe and user-oriented deployment of mobile robots in public environments. Instead of developing new robot systems, the project focuses on adapting existing solutions to the requirements of public spaces. In various application-specific case studies, relevant stakeholders were involved in the development of design recommendations, safety guidelines and legal advice for new regulatory frameworks. Requirements for human-robot interaction in public spaces were derived from this project. At the same time, XR technologies are being developed in the field of engineering sciences in another large research project. Mixed reality technologies have been developed for a model factory that use interactive pre-visualisation to enable efficient human-robot interaction and make initial results tangible in the early stages of project implementation. Technologies for mixed reality and human-robot interaction have been adopted from this project.

Thirdly, the authors work at the interface of art and technology, creating artistic projects that integrate robot technologies into contemporary dance performances. It makes sense to identify synergies between different technical areas of robot applications on the one hand and artistic research on human-robot interaction on the other. We describe the interdisciplinary development process from the perspective of the various disciplines in our project Spheres that created an artistic human-robot interaction performance. Workflows for mixed reality and human-robot interaction were adopted from this project, and experience was gained in the implementation of complex projects at the interface of human-robot interaction, mixed reality and art in public spaces.

The results achieved in the three projects will then be adapted in a new project that deals with the education of schoolchildren through the development of an artistic performance with mixed reality and robotics. The BeyondNow++ project develops innovative interfaces between computer science, artificial intelligence & robotics, mixed reality (AI/XR) and digital arts. Building on the various digital projects of the interdisciplinary consortium mentioned above, BeyondNow++ expands the interdisciplinary approach towards computer science, robotics and programming for digital art projects in order to inspire secondary school students to take an interest in computer science and digital technologies in the field of presentation and design, and to anchor this concept sustainably in the everyday extracurricular lives of young people.
Keywords:
Human-robot-interaction, mixed reality, visualization, artistic research, STEM education.