Advanced Interfaces

From the line of research in Human Computer At ITI Interaction , we work on the design, development, and evaluation of immersive experiences (Virtual Reality, Mixed Reality, Augmented Reality) and multimodal interfaces (haptics, biofeedback , gestures, etc.) that enhance the sense of presence and realism in these scenarios. In the defense field, the following ITI capabilities would be applicable:

• Development of immersive environments: Using extended reality devices, it's possible to generate customizable virtual scenarios to train professionals for risky situations. These environments could replicate common situations that professionals may face, allowing them to become accustomed to these scenarios in controlled environments without compromising their safety.

• Multi-user, geographically distributed immersive environments: In modern defense situations, teamwork and coordination between different units is essential. Immersive environments allow for the creation of multiplayer simulations where teams can train together in high-stress situations, reinforcing cohesion and joint strategy. ( CORE-MHC Project: Remote Multi-User Environments )

• Biofeedback and continuous monitoring : Emotional responses, measured by biometric sensors (pulse, respiration, sweating, brain activity, etc.), allow the capture of relevant information to adapt the immersive environment in real time , as well as to monitor the user's state at all times. Using adaptive immersive environments, where the virtual experience reacts in real time to the user's behavior and emotions, we can simulate conflict de-escalation situations, decision-making in extreme conditions, and stress management, which takes into account the user's emotional state and facilitates learning and emotional control under pressure. ( ARCADIA - Avatar reacting to biofeedback, https://dl.acm.org/doi/10.1145/3613904.3642123 )

• Mixed/Augmented Reality: Mixed/augmented reality can overlay key information, such as maps, trajectories, and critical real-time data, onto the operator's view to assist with decision-making during tactical or training operations, increasing effectiveness. ( CELIA Project - Augmented Data Visualization )

• Haptics : Haptic interaction enriches virtual environments and enhances the sense of presence and realism. Simulating combat, rescue, or first aid scenarios, where users can experience tactile simulations of impact, temperature, or equipment resistance, will improve preparedness and response in critical situations.

• Robot and drone teleoperation : The teleoperation of robots and unmanned aerial vehicles (UAVs, UGVs, UCAVs) is crucial in defense scenarios where human safety is at stake. Using immersive technologies, we can create control interfaces that allow operators to interact more efficiently and naturally with robots. Furthermore, the use of haptics will allow operators to feel the physical interaction with objects or terrain, improving precision and control in search and rescue, explosive ordnance disposal, or patrol missions. (CELIA and MANTRA projects - AGV teleoperation through mixed reality)