This use case is expected to combine multiple heterogeneous smart wearables, 3D video projectors, advanced AR/VR/XR headsets and low cost devices such as smartphones and tablets.
Smart Media/ City & XR Use Cases & Living Lab
Use case Applications
- Round of Athens Race: During the race, media content is captured by many spectators along the running circuit using smartphones, a few professional and CCTV cameras. Incoming content is automatically processed, annotated and rendered (partially on the device using already trained AI/ML models and partially at the edge), and a selection is directly broadcasted (e.g. via social media) based on location info of the (top) runners and interesting events during the race (e.g. based on contributor annotation). The audience has the option to improve their contributions and can interact with contributors in case of specific race incidents.
- XR Time Machine: We aim to push the boundaries of immersive experience by optimising multi-sensorial stimuli via effects such as wind, heat, vibration, in addition to audiovisual (AV) and tactile. We aim to create an environment that will enable multiple users to interact with virtual or augmented/XR worlds ranging from a virtual trip to a house in Ancient Greece to augment dinosaurs in todays’ world. This comes with a great increase in terms of bandwidth (8K/12K resolutions for 360o video & geometry plus textures for 3D media, optionally accompanied by metadata like lighting, rendering shades, etc.) that also manifests in tighter latency requirements for streaming content, due to the higher refresh rates (90Hz for VR head mounted displays and 240Hz for AR headsets).
In both cases, we will investigate advanced networking and data processing NEMO features, such as:
- At AIoT level, through AV and image analysis using ML techniques, NEMO will be able to calculate the exact location of a user and multi-users’ relevant movement to personalize the view/sound
- At Edge cloud, for automated processing and rendering of virtual and augmented objects.
- At cloud level to enable true shared experience even with remotely located users.
The use cases are expected to implement a) a Smart Media/Smart City Living Lab implemented by NOVO, utilizing their leading smart city platform, already installed in the city of Athens, with more than 70.000 subscribed citizens and UPM specialized in user generated 3D smart media and b) a Smart Media/XR Living Lab implemented by FHW, one of the leading European organizations in that field of cultural interactive media, with millions of visitors every year and Maggioli Cultura, the MAG division, specialized in smart media and 3D/XR cultural heritage. The XR part will utilize the FHW “Tholos”, a dome-shaped Virtual Reality “theatre”, with state-of-the-art 3D audiovisual, sound and lighting systems, advanced AR/VR headsets (e.g. Oculus Quest 2, Microsoft HoloLens 2 and VR SDKs). The 5G connectivity will be provided by OTE, offering > 99.9% coverage in the Athens and CMC TSN bridge.
NEMO as Differentiator at Smart Media/Smart City and Smart Media/XR Living Lab
- Very fast/time sensitive services migration to the edge and extreme large media from thousands of users.
- Media processing and rendering at the IoT-to-Edge-to-Cloud continuum to support multiple users.
- Advanced FML analytics to calculate the accurate positions and orientation and create ML models and alarms.
- Validate NEMO user acceptance from a citizen viewpoint by utilizing the NOVO smart city platform in Athens