: open source smart assistant dedicated to companies and profesional usages 

Main issues and objectives

Context

As part of its strategy to develop innovative open source tools for collaboration, LINAGORA wants to design a smart conversational assistant for companies. "LinTo" is a physical device providing a set of employee support services driven by speech. These services will help staff members in their various tasks: personal ones like consulting one's agenda, reading one's emails, managing reminders, searching for document or data in the company information system or collective ones involving several meeting participants or team members: planing a meeting by crossing individual agendas, taking notes, recommending relevant information or documents, generating reports, etc. LinTo is designed as an extension of the OpenPaaS virtual office developed by LINAGORA. This cloud platform offers a virtualized environment for managing emails, appointments, contacts and documents as well as a video conferencing tool within the "Hubl.in" web browser. The latter is already able to deduce in real time the topics of the meeting through a voice flow analysis service. In connection with this technology, LinTo offers a device to assist employees through the use of artificial intelligence techniques (recognition of speech (RAP), natural language processing (NLP) and speaker recognition, voice and communicative intentions) and the respect of personal information (GDPR). As a smart assistant, LinTo allows each user to quickly access the information he needs, through natural language, which enable him/her to interact more effectively with his/her colleagues or partners.

SAMoVA's contribution

Our team is involved in the process of analyzing conversational and spontaneous speech in meeting and characterizing communication situations and conversational interaction contexts. Three main situations are considered in this project:

  • LinTo as a personal assistant in a "face to face" interaction with a privileged user; 
  • LinTo as a conversational assistant, in a multi-user context , interacting with several speakers; 
  • LinTo as a collaborative assistant, helping participants during meeting continuity (by providing relevant information) or later (by providing a synthesis or a summary of the meeting topics or keypoints)

LinTo being able to capture audio and visual data from its embedded sensors (microphone array and camera), it will have to process, analyze and merge data from different modalities (sound, images). Such perceptive functionnalities will be the core of LinTo's abilities to distinguish meeting participants from each other as well as relevant participant actions or behaviors related to the meeting processing. Approaches associating audio-visual signatures to participants will be investigated and will constitute the first step to make LinTo able to get a representation of the meeting processing and act as a smart assistant. SAMoVA team will be mainly involved in the analysis of the audio modality (speech, voice, prosody, word saliency, ...). The analysis of the visual modality will be done by LAAS-CNRS researchers. LAAS and IRIT will collaborate to propose fusion strategies of both modalities in order to build audiovisual signatures. Artificial Intelligence and Machine Learning methods will be used in this three steps. The aim of this work will be to provide information to other partners (IRIT team MELODI, ...) involved in interaction management, information that could be used (i) online for immediate processing during Person/System interaction and Person/Person interaction during different meeting  phases and (ii) offline for producing an enriched representation serving as a basis for semantic analysis and summary production of meetings.

Partners

People involved in the SAMOVA team

Funding

  • Programme d’Investissements d’Avenir - GRANDS DEFIS DU NUMERIQUE- 2018
  • Funded by BPI France

Schedule

  • Start time: 1st April 2018
  • End time: 31st March 2021