aceMedia: Integrating knowledge, semantics and content for user centred intelligent media services", 6th FP IST Integrated project, 2004 - 2007. The R&D objectives of aceMedia include the following:

• Introduction of the Autonomous Content Entity (ACE) concept combining content, metadata, intelligence in a single entity
• Knowledge and context-assisted content analysis techniques based on a multimedia ontology infrastructure to support semantic entity detection and tracking of ACE content
• High-level semantic reasoning tools for automatic annotation and generation of the ACE metadata layer
• Query analysis tools and intelligent ACE search, retrieval, ranking and relevance feedback mechanisms
• Intelligent mechanisms to manage ACE communication, and methods to produce aesthetically appealing content and enhanced visualisation for navigation and rendering
• User interfaces and adaptivity support for aceMedia target application

BOEMIE: Bootstrapping Ontology Evolution with Multimedia Information Extraction, 6th FP IST-STREP, 2006-2008.
BOEMIE will pave the way towards automation of the process of knowledge acquisition from multimedia content, by introducing the notion of evolving multimedia ontologies which will be used for the extraction of information from multimedia content in networked sources, both public and proprietary. BOEMIE advocates a synergistic approach that combines multimedia extraction and ontology evolution in a bootstrapping process involving, on the one hand, the continuous extraction of semantic information from multimedia content in order to populate and enrich the ontologies and, on the other hand, the deployment of these ontologies to enhance the robustness of the extraction system. The ambitious scope of the BOEMIE project and the proven specialized competence of the carefully composed project consortium ensure that the project will achieve the significant advancement of the state of the art needed to successfully merge the component technologies. The main measurable objective of the project is to improve significantly the performance of existing single-modality approaches in terms of scalability and precision. Towards that goal, BOEMIE will deliver a new methodology for extraction and evolution, using a rich multimedia semantic model, and realized as an open architecture. The architecture will be coupled with the appropriate set of tools, implementing the advanced methods that will be developed in BOEMIE. Furthermore, BOEMIE aims to initiate a new research activity on the automation of knowledge acquisition from multimedia content, through ontology evolution. The resulting technology has a wide range of applications in commerce, tourism, e-science, etc. During the project, the technology will be evaluated through the development of an automatic content collection and annotation service for public events in a number of major European cities. The extracted semantic information will enrich a digital map, which will provide a friendly interface to the end user.

CHORUS: Coordinated approacH to the EurOpean effoRt on aUdio-visual Search engines

CHORUS is a Coordination Action which aims at creating the conditions of mutual information and cross fertilisation between the projects that will run under Strategic objective 2.6.3 (Advanced search technologies for digital audio-visual content) and beyond the IST initiative.
CHORUS workplan includes the set up of a coordination and information exchange platform (cluster meetings, newsletter, website) to enhance the interactions and to invoke discussions between the key players of the IST 'Cluster' and thus to stimulate the creation of the European Research Area (ERA).
An important task of the project consists of supporting the preparation of an analysis and a roadmap for the realisation of Audio-visual search engines in EU. That activity will need a multi-disciplinary (societal, economic, regulatory and technological) and strategic approach and will imply the participation of major Stakeholders of the domain through the creation of a Think-Tank. The Think Tank, to which several representative organizations have already agreed to participate, will contribute to give an industrial relevance to the coordination action and its visions. The areas a priori investigated by the project are described in WP2 activities.
Interaction and dissemination will be further ensured through the set-up of a conference and dedicated workshops addressing a large community which complement the coordination instruments proposed by CHORUS.

CHORUS+: Coordinated approacH to the EurOpean effoRt on aUdio-visual Search engines +

CHORUS+ is a Coordination Action, which aims at creating the conditions of mutual information and cross fertilisation between the FP7 projects, National initiatives, Iniatives external to Europe with a particular emphasis on setting concrete R&D and industrial objectives for multimedia search in Europe. More specifically, CHORUS+ objectives are to support the integration and strengthening of the European Research Area by stimulating interaction and co-ordination on an EU level, in the area of Audiovisual search engines and extend cooperation to Asian countries. Additionally, CHORUS+ will support the creation of a ‘holistic’, multi-disciplinary approach for the implementation and realisation of Audiovisual search engines in EU and the dissemination of results and strategies developed within the ICT Programme. The starting date of CHORUS+ is the 1/1/2010. 

COST292: Semantic Multimodal Analysis of Digital Media, European Cooperation in the Field of Scientific and Technical Research, 2004 - 2008. Objective of this action is to push forward the frontiers of current research on semantic analysis, inference and conceptualisation for high-level annotation and retrieval of digital audiovisual content.

ELU: Enhanced Learning Unlimited. The objective of Enhanced Learning Unlimited (ELU) is to look on how interactive digital TV can be used to increase learning opportunities in the home, office and school. The television is a familiar and reliable consumer device with around 95-99% penetration in European households. It is also perceived to be a source of learning. However, in its more traditional role it has tended to be used in a passive viewing mode.

GLOCAL: Event-based Retrieval of Networked Media", FP7 IST IP, 2009-2012.
The key idea underlying GLOCAL is to use events as the primary means to organize and index media, e.g., photos, videos, journal articles. Instead of starting from media and seeing, a posteriori, how we can meaningfully understand their contents (e.g., by tagging them), we organize a priori our data and knowledge in terms of events and use media to populate them, thus providing their experiential dimension. Events provide the common framework inside which the local experience-driven contextual information can be not only coded, but also shared and reduced to a common denominator. (Scientific Responsible: Vasileios Mezaris).

JUMAS: Judicial Management by Digital Libraries Semantics, IST-STREP

An advanced audio and video knowledge management system JUMAS addresses the need to build an infrastructure able to optimise the information workflow in order to facilitate later analysis. New models and techniques for representing and automatically extracting the embedded semantics derived from multiple data sources will be developed. The most important goal of the JUMAS system is to collect, enrich and share multimedia documents annotated with embedded semantic minimising manual transcription activity. JUMAS is tailored at managing situations in which multiple cameras and audio sources are used to record assemblies in which people debates and event sequences need to be semantically reconstructed for future consultations. The prototype of JUMAS will be tested interworking with legacy systems, but the system can be viewed as able to support business processes and problem-solving in a variety of domains.

K-Space: Knowledge Space of Semantic Inference for Automatic Annotation and Retrieval of Multimedia Content, 6th FP IST NoE, 2006-2008.
K-Space will create a sustainable network of world-leading research teams from academia and industry to conduct integrative research and dissemination activities in semantic inference for automatic and semi-automatic annotation and retrieval of multimedia content, aiming at closing the gap between the low-level content descriptions that can be computed automatically by a machine and the richness and subjectivity of semantics in high-level human interpretations of audiovisual media. Specifically, K-Space integrative research will focus on three key areas: Content-based multimedia analysis; Knowledge extraction; and Semantic multimedia. To ensure that the output of the targeted research becomes the primary feeding ground for industrial innovation, an Industrial Advisory Board with a core of four key industrial players has been set up and has been consulted from the very outset of proposal building. Its mission is to assist the network in defining research directions and to advise on potential exploitation of research results as they emerge. Specific dissemination objectives:

• Dissemination of the technical developments of the network across the broad research community
• Delivering the means to bring the results beyond the research community in a way that appeals to the EU citizen
• Enabling the non specialist multimedia user to fehttp://www.mesh-ip.eu/edback into the K-Space technological framework
• Influencing and contributing to related knowledge-based multimedia standardisation activities

Knowledge Web: Realizing the Semantic Web", 6th FP IST NoE, 2004-2007. The mission of Knowledge Web is to strengthen the European industry and service providers in one of the most important areas of current computer technology: Semantic web enabled e-work and e-commerce. It will concentrate the efforts around the outreach of this technology to industry. Naturally, this includes education and research efforts to ensure the durability of impact and support of industry. Therefore, the main Objectives of Knowledge Web are:

• Outreach to Industry
• Outreach to Education
• Coordination of Research

MEDUSA: Multi Sensor Data Fusion Grid for Urban Situational Awareness, EDA 

The MEDUSA project will study a combination of different types of sensors supporting operations within urban environments. The essential improvement that the MEDUSA project will be implementing is the provision of a means to address major existing capability shortfalls: (i) in the fusion of data from multiple diverse types of sensors and (ii) in the data representation, by means of a consolidated and integrated view, including overlaying across 3D GIS (Geographical Information System) models to facilitate decision-making by commanders of control and support operations.

The MEDUSA project will analyze a diverse cross-section of existing imaging sensor technologies including those in the field of video, with specific reference to images on visible and infrared bands. The fusion of the sensor data in particular with Digital Terrain Elevation information provided by the GIS is currently an acknowledged key technology/capability shortfall and thus a salient benefit of the MEDUSA project outcome – both for increasing overall knowledge in the domain and as a means of accelerating downstream exploitation of enhanced approaches. Another group of sensors that will be considered for data fusion in generating an Operational Ground Picture, are sensors in very different categories, inclusive of acoustic sensors, motion detectors, chemical sniffers, through the wall radars and the like.

MEDUSA is funded by the European Defense Agency (EDA).

The project's starting date is September 1st, 2009.

MESH: Multimedia Semantic Syndication for Enhanced News Services, 6th FP IST Integrated Project, 2006-2008.
Multimedia Semantic Syndication for Enhanced News Services (MESH) will apply multimedia analysis and reasoning tools, network agents and content management techniques to extract, compare and combine meaning from multiple multimedia sources, and produce advanced personalized multimedia summaries, deeply linked among them and to the original sources to provide end users with an easy-to-use 'multimedia mesh' concept, with enhanced navigation aids. A step further will empower users with the means to reuse available content by offering media enrichment and semantic mixing of both personal and network content, as well as automatic creation from semantic descriptions. Encompassing all the system, dynamic usage management will be included to facilitate agreement between content chain players (content providers, service providers and users). In a sentence, the project will create multimedia content brokers acting on behalf of users to acquire, process, create and present multimedia information personalized (to user) and adapted (to usage environment). These functions will be fully exhibited in the application area of news, by creation of a platform that will unify news organizations through the online retrieval, editing, authoring and publishing of news items.

MKLab collaborates with Motorola Research in a series of projects:

Recommender System Development via Automatic Semantic Induction

The project's scope is to develop a distributed semantic framework for personalised recommendation using advanced textual analysis and user profiling techniques,  knowledge-based reasoning and collaborative filtering to enable targeted recommendation of textual content, for any domain for which there is textual data even when little or no expert knowledge is available. 

Motorola US Inc, (2009-2010)

Personalized Recommendation of Multimedia Content and Advertising by Means of Distributed Knowledge-based Reasoning

The aim of the project was to develop semantic textual analysis, knowledge-based reasoning and content filtering technologies to enable targeted advertising based on analysis of textual descriptions associated to advertisements, usage tracking, and implicit user profiling by means of machine learning techniques. A patent ("Method and System for Recommendation of Content Items") was filed.

Motorola UK Ltd, (2007 - 2008)

Contextual Search based on Semantic Text Analysis

The project aimed at the development of a framework for the automatic formulation of search queries based on the current context (e.g. web page being read) of the user. The framework comprises a series of modules for semantic analysis, query formulation, query validation and query expansion. A patent ("Selection of Associated Content for Content Items") was filed and the following paper was presented at the MUM 2008 conference in Umea, Sweden.

Motorola UK Ltd, (2007)

Distributed Knowledge Management for Personalized Content Delivery

The main objective was to build a distributed reasoning architecture, aiming at delivering personalized news content to mobile users. The personalization process builds upon a text analysis framework for news content and the development of algorithms for adaptive user profiles. A patent ("A content recommendation system") was filed and a journal article was written after completion of the project.

Motorola UK Ltd, (2005 - 2006) 

 

PATExpert: Patents belong to the few types of public information that have a big impact on the European economy, and whose proper monitoring, retrieval, representation, interpretation, and assessment so clearly depend on the access to its content, and, thus, on advances in semantics-based techniques. However, research and development in the area of patent processing still focuses on selected traditional tasks such as text retrieval, classification, and shallow linguistic analysis. PATExpert's overall scientific objective is to change the paradigm currently followed for patent processing from textual (viewing patents as text blocks enriched by picture material that are interpreted as 'canned' material, sequences of morpho-syntactic tokens, or collections of syntactic structures) to semantic (viewing patents as multimedia knowledge objects) processing.

MKLab is closely collaborating with the Department of Electrical and Computer Engineering of the Faculty of Engineering of the Aristotle University of Thessaloniki - AUTH and in particular with the Information Processing Laboratory (IPL) of AUTH, in the research activities that are carried out in the PATexpert project.

PESCaDO: Personalized Environmental Service Configuration and Delivery Orchestration

The overall goal of PESCaDO is to research and develop strategies for user-centred environmental service orchestration and multilingual information delivery in the context of active decision support for environment conditioned tasks. These strategies, realized as an operational workbench, will support the user during the entire process of environmental information-based decision-making from the formulation of the problem, via the search for adequate environmental service nodes and their connection (including data, knowledge and language synchronization), up to multilingual information, advice, hints, etc. delivery. PESCaDO will adapt its strategies and language preferences to the profile and the needs of the user. Furthermore, it will allow for intervention and feedback at any stage of the process, incorporate the received feedback and facilitate several loops of the process until the full satisfaction of the user. The backbone of the involved strategies will be semantics. In addition, PESCaDO will develop visual analytics strategies that will be specific to incremental environmental conditions oriented decision support and problem solving. The starting date of the project is the 1/1/2010.  

SCHEMA: Network of Excellence in Content-Based Semantic Scene Analysis and Information Retrieval, IST-2001-32795, Principal Contractor: CERTH/ITI. SCHEMA is a Network of Excellence aiming to promote research and innovative applications in the field of Content Based Semantic Scene Analysis. SCHEMA aims to (i) bring together a critical mass of partners; (ii) exchange information among the partners in order to combine their research work and skills; (iii) disseminate the results of the partnership. The SCHEMA Network of Excellence creates a synergy and a multiplier effect enriching the capabilities of the group of partners by bringing together complementary skills and know-how.

VIDI-Video: Interactive semantic video search with a large thesaurus of machine-learned audio-visual concepts. VIDI-Video project takes on the challenge of creating a substantially enhanced semantic access to video, implemented in a search engine. The engine will boost the performance of video search by forming a 1000 element thesaurus detecting instances of audio, visual or mixed-media content. This project's approach is to let the system learn many, possibly weaker, detectors instead of modelling a few of them carefully.
Concrete outputs will be a fully implemented audio-visual search engine, consisting of two main parts, viz. a learning system and a runtime system, where the former will feed its results into the latter after each round of training-and-thesaurus-update. The learning system will consist of software to be developed for overall video processing; visual analysis; audio analysis; integrated feature detector; and multimedia query and user interface.
The key objectives of this project are:

• to build a large scale thesaurus well-spread over the semantic clues
• to design, adapt and evaluate methods to learn large thesauri of detectors
• to define and evaluate powerful sets of visual, audio, and cross-modal invariant features
• to deliver effective interaction with the user
• to evaluate the approach in relevant application areas

WeKnowIt: The main objective of WeKnowIt is to develop novel techniques for exploiting multiple layers of intelligence from user-contributed content, which together constitute Collective Intelligence, a form of intelligence that emerges from the collaboration and competition among many individuals, and that seemingly has a mind of its own.

To this end, input from various sources is analysed and combined: from digital content items and contextual information (Media Intelligence), massive user feedback (Mass Intelligence), and users social interaction (Social Intelligence) so as to benefit end-users (Personal Intelligence) and organisations (Organisational Intelligence).

The automatic generation of Collective Intelligence constitutes a departure from traditional methods for information sharing, since for example, semantic analysis has to fuse information from both the content itself and the social context, while at the same time the social dynamics have to be taken into account. Such intelligence provides added-value to the available content and renders existing procedures and workflows more efficient.

The starting date of the project is April 1st, 2008. 

 

X-Media: Knowledge Sharing and Reuse Across Media, 6th FP IST Integrated project, 2006 - 2009.
X-Media addresses the issue of knowledge management in complex distributed environments. It will study, develop and implement large scale methodologies and techniques for knowledge management able to support sharing and reuse of knowledge that is distributed in different media (images, documents and data) and repositories (data bases, knowledge bases, document repositories, etc.) or that is inaccessible for current systems, which cannot capture the knowledge implicit across media. X-Media will study, design and develop:

• Effective and efficient new paradigms for knowledge retrieval, sharing and reuse which enable users to define and parameterize views on the available knowledge according to their needs. Users will be supported by an anticipatory system able to trace automatically the user profile and help focusing in the large search space
• Novel and cutting-edge knowledge fusion methods to support knowledge workers in making decisions when confronted with - possibly contradicting - knowledge derived from different resources
• Techniques able to represent and manage (i) uncertainty, (ii) trust and provenance as well as (iii) dynamic aspects of knowledge, in order to help users balancing their judgment
• Robust and scalable knowledge acquisition and data analysis tools operating across media boundaries (text, images and data) to automatically cross-relate and annotate text and images with metadata for easier sharing and retrieval as well as fostering integration with other resources
• A methodology and a technical infrastructure able to deliver knowledge from across media to the knowledge workers, taking into account the complexity of managing different media with different size of data
• A generic and flexible architecture allowing end users to easily customize it and integrate it with their KM practices or needs as well as a mainly open source reference implementation and libraries which technology providing companies can reuse

DELTIO: Analysis of Multimedia Content using Evolutionary Ontologies and Application to Television News Bulletins, General Secretariat for Research and Technology, Infosociety, 2006-2007. Objective of the project is the development of innovative techniques for the representation, analysis and extraction of semantic information for the manipulation of multimedia content, with emphasis on their application to television news bulletins. Research and development will concentrate on techniques that enable the automatic analysis of multimedia content and the extraction of knowledge, resulting in the (semi-) automatic creation of metadata, as well as provide support for smart, semantic search services. The final output of the project will be a system for the analysis of television new bulletins, which will provide intelligent search functionalities in archives of digital television material.

CIRCE: Content-Based and Semantic Multimedia Indexing and Retrieval in Collaborative Environments, General Secretariat for Research and Technology, Hungary - Greece, Joint Research and Technology Programmes, 2004 - 2006. Objective of the collaboration will be the definition, specification and implementation of an architecture for the content-based and semantic indexing and retrieval of visual information, and the sharing of the knowledge acquired during retrieval by different users.

Knowledge-assisted analysis of medical image and video for doctor decision support applications, General Secretariat for Research and Technology, Slovenia - Greece, Joint Research and Technology Programmes, 2004 - 2006. Objective of the collaboration will be the definition, specification and implementation of the algorithms and architecture for the knowledge-assisted analysis of medical image and video data, suitable for use in applications such as doctor decision support.

GSA: General State Archives
Procurement and Development of Archival Data Management Integrated Software and Translation of Archival Descriptions
MKLab is participating in this project in collaboration with the General State Archives of Greece. The project starts on June 15th 2008, has duration 6 months and it is funded by the Ministry of National Education and Religious Affairs. It aims to develop applications for Archival Data Management and Translation of Archival Descriptions.

The key objectives in the context of this project are the following:

1. Implementation of integrated software for archival data management with modular architecture, that offers:

• Acquisition (selection and accession) of archival content
• Archive arrangement and multilevel description
• Management of the preservation and storage conditions of the holdings
• Distribution of archival content (access, dissemination, restoration, reproduction)
• Availability of finding aids and digital content in the internet (e.g. search with keywords, authority files or thesauri)
• Extended access services to remote users (e.g. digital copies, scalable access rights, e-mail management)
• Various reports (e.g. statistics, progress reports, etc)
• Collaboration and communication with portals, digital libraries and archival databases

2. Development of technological solutions for the exploitation, linking and embedding of digital copies generated during the Subproject 1, namely “Digitization, scientific management and documentation of G.S.A. archival data”
3. Migration of documentation metadata produced (in digital form) during the Subproject 1 “Digitization, scientific management and documentation of G.S.A. archival data” of the project “G.S.A. digitization of cultural collections” and of metadata acquired in digital form according to archival standards by the Central Service and the Regional Services of G.S.A., in agreement with the article 4.2.11, Part B of the Proclamation.
4. Support and maintenance services of the application software.
5. Translation of archival descriptions into English and, also, into the native language of the holdings.

 

MULTI-MINE: Research and Technology Training Network in Multimedia Knowledge Discovery and Management, GSRT-HUMAN RESOURCES. Principal Contractor: CERTH/ITI. The network, consisting of six Greek academic institutions and two SMEs, will organize a series of training activities in order to introduce knowledge technologies for multimedia content in Greek research and commercial institutions.

ONTOMEDIA: Semantic Analysis of Multimedia Content Using Knowledge Technologies. The main technological objective of this project is to discover and exploit knowledge inherent to the content. Key scientific and technological research project objectives include:

• Research and development of novel knowledge structures integrating semantic concepts and multimedia descriptors
• Research and development of knowledge-assisted content analysis techniques based on a multimedia ontology infrastructure to support semantic entity detection and tracking
• Research and development of high-level semantic reasoning and personalization tools for automatic annotation and generation of metadata

Semantic Processing of 3D Intravascular Ultrasound, General Secretariat for Research and Technology Operational Programme "Competitiveness" 2000-2006. The current project focuses on the research, development of specific techniquesalgorithms and implementation of an integrated system/tool for the 3D reconstruction of the vessel's model from 2D IVUS images. The specified tool will be used for the blood flow simulation through the examined vessel and will also support the semi-automatic semantic image processing and image annotation.

REACH New Forms of Distributed Organization and Access to Cultural Heritage, General Secretariat for Research and Technology, Research and Technological Development Consortia in Sectors of National Priority - CULTURE. The overall aim of REACH is to provide a web portal enabling access to different databases of cultural content with emphasis on multimedia material. The objective is to use ontologies to provide enhanced unified access to heterogeneous distributed cultural heritage digital databases. In order to achieve this, the project employs ontology-based representation of cultural content, multimedia content-based search algorithms, hybrid ontology and content-based access and user-friendly search interfaces. This fusion approach provides users with resources for building queries of multimedia repositories sequentially using multiple individual search tools, and performs higher quality searching. The system will be tested using real content such as inscriptions and coins from the Greco-Roman time period.

SERVISIO: Providing Services in Intelligent Content-Based Management and Search of Visual Data, GSRT-PRAXE. Set-up of a spin-off company.

Visual Asset: Multimedia document analysis, information extraction and management system, GSRT-EPAN, 2004-2006. Objective of this collaboration is the development of novel techniques for the analysis, extraction of semantic information and management of multimedia content and in particular multimedia documents.