What is the motivation behind MULTISENSOR?

During the past decade, the rapid development of digital technologies and the low cost of recording media have led to a great increase in the availability of multilingual and multimedia content worldwide. In the best case, this content is repetitive or complementary across political, cultural, or linguistic borders.However, the reality shows that it is also often contradictive and in some cases unreliable.

The consumption of such large amounts of content regardless of its reliability and cross-validation can have important consequences on the society. An indicative example is the current crisis of the financial markets in Europe, which has created an extremely unstable ground for economic transactions and caused insecurity in the population.

The consequence is an extreme uncertainty and nervousness of politics, and economy on the one side, which makes national and international investments (e.g. SME internationalisation) really risky, and on the other side, the inability of journalism and media monitoring to equally consider all the media resources leaves the population in each of these encapsulated areas in its own perspective – without the realistic opportunity to understand the perspective developed in the other areas in order to adjust the own.

To break this isolation, we need technologies that provide unified access to multilingual and multicultural economic, news story material across borders, that ensure its context-aware, spatiotemporal, sentiment-oriented and semantic interpretation, and that correlate and summarise the content into a coherent whole.

These technologies should be capable to capture, interpret and relate economic information and news from various subjective views as disseminated via TV, radio, newspapers, blogs and social media.

On top of this, semantic integration of heterogeneous media including computer-mediated interaction is required to gain a usable understanding based on social intelligence, while a correlation with relevant incidents with different spatiotemporal characteristics would allow for extracting hidden meanings. Although there are several research works targeting these areas independently, there is a gap in accessing these resources in a holistic manner.

What is the concept?

In order to achieve multidimensional integration of heterogeneous resources, MULTISENSOR proposes a content integration framework that builds upon multimedia mining, knowledge extraction, analysis of computer-mediated interaction, topic detection, semantic and multimodal representation as well as hybrid reasoning.

MULTISENSOR aims at bridging this gap by envisaging at an integrated view of heterogeneous resources sensing the world (i.e. sensors) such as international TV, newspapers, radio and social media.

The approach of MULTISENSOR will build upon the concept of multidimensional content integration by considering the following dimensions for mining, correlating, linking, understanding and summarising heterogeneous material:

• language
• multimedia
• semantics
• context
• emotion
• time and location.

The overall goal of MULTISENSOR is to research and develop a unified platform, which will allow for the multidimensional content integration from heterogeneous sensors, with a view to providing end-user services such as international media monitoring, and decision support for SME internationalisation.

Scientific Objectives:

• content distillation of heterogeneous multimedia and multilingual data;
• sentiment and context analysis of content and social interactions;
• semantic integration of heterogeneous multimedia and multilingual data;
• semantic reasoning and intelligent decision support;
• multilingual and multimodal summarization and presentation of the information to the user.

Deliverables:

There is an overall of 41 deliverables (most of them public) derived from the nine different workpackages.
Depending on the workpackage they are assigned to, they aim at fulfilling different goals ranging from management to research, development and exploitation.

Under Achievements > Deliverables you can take a look into the publicly available versions of these deliverables.

Workstructure:

The work is divided into different workpackages aligned with the different aspects of the project.

You can find more details about this in the the menu under Project > Workstructure.

Use Cases:

The developed technologies will be validated with the aid of 2 main use cases:

1. International mass media news monitoring
   and
2. SME international investments.

More Details about the different Use Cases and Scenarios can be found under Specifics > Use Cases.