Team LST

A bilingual or polyglot speaker has the ability to communicate coherently in several languages, adapting to different contexts. Transferring this skill to machines could contribute to the preservation of cultural heritage by maintaining less privileged languages, facilitating interaction between people of different cultures and languages, and reinforcing safety measures. ► Read more

We propose to create a Computer-assisted learning (CAL) system to train children from grades 4-5 to produce qualitative oral summaries, by automatically analyzing their production and providing differentiated feedback. ► Read more

Our project aims at integrating the production of speech recognition and translation into an online communication tool, which will create a true barrier-free communication platform. The project will focus on the production of French and English translations from an Arabic and English speech source.

Recent approaches to embedding learning have focused on results, often at the expense of interpretability and algorithmic complexity. Yet, interpretability is a necessary pre-requisite for the deployment of such technologies when they are used in sensitive domains such as law or medicine. Moreover, ecological imperatives create an urgent need to think about efficient and computationally efficient systems. With DIGING, we propose a new powerful and computationally efficient approach for the construction of interpretable lexical plots based on the theory of complex networks. With this original approach, the goal is to build vectors integrating polysemy natively by plunging words into a space with interpretable dimensions.

ESPERANTO is a collaborative research program based on a partnership between 15 academic and 4 non-academic partners, having complementary expertise and resources. The Esperanto project involves academic institutions leading the field of speech processing on 4 continents for decades and covering a large range of speech processing applications. ESPERANTO will contribute to the development of the next generation of AI; AI interacting with their users for better learning but also explainable systems: systems whose decisions are understandable by humans. In order to make these technologies accessible to the largest number of people, ESPERANTO will address applications with limited resources in the field of rare languages, robotics or education.

When we think of Franco-German relations, it is clear that they are not described in the same terms on either side of the Rhine. While the figure of the "couple" dominates the perception of Franco-German cooperation in France, in Germany it is seen in terms of the "tandem". By studying the notions, metaphors and patterns of perception used to think about Franco-German relations, this project aims to identify the similarities and differences in the way the French and Germans view relations between their countries. Ultimately, the aim is to find out whether there is a shared Franco-German imaginary. ► Read more

The DIETS proposes to focus on the problematic of diagnosis/evaluation of end-to-end automatic speech recognition (ASR) systems by integrating human reception of transcription errors from a cognitive point-of-view. The challenge is here twofold: 1) to analyze finely ASR errors from a human reception, and 2) to understand and detect how these errors manifest themselves in an end-to-end ASR framework, whose work is inspired by how the human brain works.

SpeechBrain is an open-source and all-in-one speech toolkit relying on PyTorch. The goal is to create a single, flexible, and user-friendly toolkit that can be used to easily develop state-of-the-art speech technologies, including systems for speech recognition (both end-to-end and HMM-DNN), speaker recognition, speech separation, multi-microphone signal processing (e.g, beamforming), self-supervised and unsupervised learning, speech contamination / augmentation, and many others. ► Read more

Media and society have an intricate relationship. Descriptive, the media are also prescriptive and shape our perception of the world. The GEM project aims to describe the differences in representation and treatment between women and men in the media, based on the automatic analysis of large volumes of data in French contained in the INA and Deezer collections: TV, radio, print media and music collections. ► Read more

The Capdif project aims at developing a differential captation system for speech signal. This system will be designed to precisely locate each speaker in a meeting room, in order to facilitate and thus improve the performance of voice technologies. ► Read more

Effective modeling of variabilities, exhibited by human’s speech at phoneme, word, and sentence levels, is still an open research problem for SLU systems. Even more, extracting relevant keywords, themes or concept mentions from either a sentence or an entire spoken document is a difficult task even for the most advanced end-to-end systems. AISSPER aims to develop new paradigms that jointly model acoustic and semantic information for the semantic analysis of oral documents. ► Read more

ExTENSoR proposes fundamental research that aims to explore the potential of using end-to-end and automatically learned / evolving articial neural networks in order to overcome the limitations of hand-crafted features and network topologies that characterise the current state of the art in many fields of speech processing. ExTENSoR also aims to bring new insights to what information in speech signals is being used in order to arrive at the scores or decisions produced by the network. ExTENSoR will pursue its objectives within the context of automatic speaker recognition and anti-spoofing, two fields of speech processing research showing burgeoning interest in end-to-end, evolutive learning. ► Read more

The MMA group (Mutuelles du Mans Assurances), the leading player in property and casualty insurance in France, is highly involved in the analysis of customer paths. In a customer relationship strategy, MMA wants to determine the factors of satisfaction, commitment and recommendation. Its marketing department conducts satisfaction surveys to collect numerous textual feedback from customers. The volume collected is such that it is impossible to manually process all of this data. That is why the MMA group has called on LIUM's knowledge to automatically analyze opinion information. ► Read more

The lexical resources of SNCF's technical documentation testify to the richness and specificities of the business vocabulary used within companies such as SNCF. This vocabulary is sometimes uncommon in corpuses, but according to experts, it is of major importance for the characterization of documents. ► Read more

The SNCF group is currently undergoing a digital transformation and is increasingly turning to technologies that could use Natural Language Processing (NLP). Business documentation is currently undergoing a major transformation, with businesses that are digitising themselves, becoming more mobile and with new ways of consuming information. Various internal projects have helped to initiate a transition to digital, to find the right information at the right time. Beyond the digitisation of documents, the question arises of new intelligent systems for access to content, interpretation and data entry. The aim of the project is to identify, design and evaluate solutions that can enrich current digital documentation initiatives. The fields of application are writing assistance, information retrieval and navigation in text content. ► Read more

The ON-TRAC project proposes to radically change the architectures currently used in speech translation. It is based on end-to-end neural models for machine translation and is particularly aimed at lightweight and portable speech translation applications. ► Read more

DEEP-PRIVACY proposes a new paradigm based on a distributed, personalized, and privacy-preserving approach for speech processing, with a focus on machine learning algorithms for speech recognition. To this end, we propose to rely on a hybrid approach: the device of each user does not share its raw speech data and runs some private computations locally, while some cross-user computations are done by communicating through a server (or a peer-to-peer network).

The SIMPÆX project aims at the automatic segmentation and identification of expressive styles and speakers in a speech corpus. Indeed the extraction of features concerning the speaker, his emotional state and the social context, offers very relevant clues for various applications such as audio indexing, automatic speech recognition, speech synthesis or human-machine interactions. ► Read more

Neo is an interdisciplinary digital humanities research project aiming at the semi-automatic detection and analysis of contemporary neology. It results from the convergence of new language practices of the modern web (creation of new words or emergence of new senses) and recent advances in natural language processing, notably via lexicographic methods. This project combines the text mining skills of computer scientists with the experience of studying the neologisms of researchers in applied linguistics. ► Read more

The NEWS-BRIDGE project will build a commercially exploitable integrated tool with a complete workflow using language technologies to facilitate and enhance multilingual news production. This tool provides for a full broadcast news translation system, rendering existing audio, video and text news content in any of the languages supported by the various external tools for transcription, translation, voice synthesis and summarization in a variety of forms and automation applications. It combines different technologies, including speech-to-text, machine translation, text-to-speech and automated summarization. It allows for customization towards the clients, offering the user the option to get the audio of online video content into their language of choice, either as a voice-over or as subtitles by a simple mouseclick. ► Read more

The goal of the ALLIES project is to encourage and demonstrate the development of autonomous systems, able to sustain performance across time according to a given learning scenario. A learning scenario defines the importance given to the performance on “past” and “present” data in the optimization (or evaluation) process. By setting the learning scenario, a human supervisor (HS) allows or forbids the system to forget. In ALLIES, an autonomous system is fully unsupervised, incrementally adapting its models but also their structure to learn or forget events according to the given learning scenario and data that it automatically collects across time. ► Read more

The RAPACE project aims to bring to the field of automatic processing of oral and written language the latest advances resulting from the field of automatic learning and concerning deep neural networks. ► Read more

The general objective of the ANTRACT project is the analysis of the images and sounds produced weekly in the framework of an independent company created in 1945, les Actualités françaises (French News), over twenty five years. This major cinematographic vector, already partially worked, has never been the subject of a systematic analysis. ► Read more

Le projet MAGMAT vise à définir et mettre en œuvre une méthodologie agile et incrémentale de développement en temps contraint d’un système de traduction le la parole vers la parole. Sur le plan du développement de langues, deux objectifs principaux sont à considérer. Le premier objectif est de mutualiser drastiquement les développements de la synthèse, de la transcription et de la traduction avec une mise commun les ressources linguistiques. Le second objectif est de définir une méthodologie rendant le processus de développement de langue adapté au contexte visé. ► Read more

PASTEL is a research project that aims to explore the potential of real time and automatic transcriptions for the instrumentation of mixed educational situations where the modalities of the interactions can be face-to-face or online, synchronous or asynchronous. The speech recognition technology approaches a maturity level that allows new opportunities of instrumentation in pedagogical practices and their new uses. ► Read more

The project aims to develop a software solution to valorize phone exchange content between consumers and brands by analyzing this content. ► Read more

Communication is one of the necessary condition to develop intelligence in living beings. Humans use several modalities to exchange information: speech, written text, both in many languages, gestures, images, and many more. There is evidence that human learning is more effective when several modalities are used. There is a large body of research to make computers process these modalities, and ultimately, understand human language. These modalities have been, however, generally addressed independently or at most in pairs. However, merging information from multiple modalities is best done at the highest levels of abstraction, which deep learning models are trained to capture. ► Read more

This project will build and develop JOKER, a generic intelligent user interface providing a multimodal dialogue system with social communication skills including humor, empathy, compassion, charm, and other informal socially-oriented behavior. ► Read more

L’objectif principal de EUMSSI est de développer des technologies d’identification et d’agrégation d’informations non structurées provenant de sources de nature très différente (vidéo, image, audio, texte) et de différentes langues (anglais, allemand, espagnol et français). Je suis responsable de l’analyse en locuteur et nous développons conjointement avec l’IDAP un système d’identification multimodale des personnes. ► Read more

The VERA project aims at developing tools for diagnostic, localization, and measurements of automatic transcription errors. This project is based on a consortium of first-rate academic actors in this field. The objective is to study the errors in detail (at the perceptive, acoustico-phonetics, lexical, and syntactic levels) in order to yield a precise diagnosis of possible lacks of the current classical models on certain classes of linguistic phenomena.