The SUN Horizon Europe project aims at investigating and developing extended reality (XR) solutions that integrate the physical and the virtual world in a convincing way, from a human and social perspective. The virtual world will be a means to augment the physical world with new opportunities for social and human interaction.

Relevant limitations that will be addressed are:

• Lack of solutions to develop scalable and cost-effective new XR applications.
• Lack of convincing solutions for mixing of the virtual and physical environment.
• Lack of plausible and convincing human interaction interfaces in XR.
• Barriers due to resource limitations of end-user devices.

The SUN project develops solutions to surpass the aforementioned limitations, including:

• Scalable solutions to obtain plausible and convincing virtual copies of physical objects and environments.
• Seamless and convincing interaction between the physical and the virtual world.
• Wearable sensors and haptic interfaces for convincing and natural interaction with the virtual environment.
• Artificial intelligence-based solutions to address current computing, memory, and network limitations of wearable devices.

The solutions will be demonstrated in three real-life scenarios, focusing on social interaction and collaboration:

1. XR for rehabilitation after accidents or diseases.
2. XR in the industry to increase safety and improve social interaction among workers.
3. XR to remove interaction barriers for persons with disabilities.

MKLab develops visual analysis tools (e.g. for pose estimation) as well as multimodal fusion techniques that combine different types of data (visual, sensors from wearables, etc.).