While media organisations increasingly support non-linear experiences for the consumer, those are still limited to single channels and media domains. Although several media organisations have recently succeeded in breaking data silos, data sharing is mostly limited to the organisation. Thus, there are challenges for producing content feeding multiple channels with different granularities and structures, mainly related to data discovery, management and (re-)use. XRECO will create a new data-driven ecosystem for the media industry, focusing on facilitating data sharing, search and discovery and supporting creation of news and entertainment content, in particular, the creation and (re-)use of location-related 2D and 3D assets and the creation of XR experiences. The ecosystem core, represented by a Neural Media Repository (NMR), will foster inter-organisation content sharing and provide increased access to content for media creators, considering novel data monetization and rights management policies. A set of AI-based media transformation services are built around the NMR to produce novel media- and XR experiences, including 3D neural reconstructions, neural based device localisation, image stitching, de-/re-lighting and holoportation. The developed technology will be validated in use case scenarios for (i) the news media for XR-based broadcasting and automatic and customized multi-target news publishing, and for (ii) location-based information and entertainment content, with applications in tourism and the automotive industry.

MKLAB will be responsible in the XRECO project for multi-modal content retrieval and indexing (e.g. visual, audio, text and point cloud modalities) in order to develop cross-modal features, such as hand gestures and body poses of character models.