Mobile application for automatic intelligibility measurement

Rvaysse/ juin 9, 2023/ Applications

Context

Intelligibility can be defined as an analytical, acoustic-phonetic decoding notion – i.e. addressing “low-level” linguistic units, referring to the quality of pronunciation at the segmental (phoneme and syllable) levels.

Speech intelligibility is usually assessed by speech therapists using perceptual tasks. While these are informative, they rely on subjective ratings, biased, among other factors, by the familiarity of the rater with the subject’s speech. They are also usually time-consuming because several assessors are needed to obtain a more objective measurement.

Overview

Our work involves developing a mobile application that can be used by therapists to automatically assess intelligibility. The core algorithm of this application is based on deep learning algorithms, so as to obtain a reproducible score that can be used to evaluate the evolution of the intelligibility score for the same patient.

The principle of the application is to have a patient read a short text (less than a minute) on a mobile device such as an Android or iOS tablet. The audio recording is then processed by our machine learning algorithm, which returns a score between 0 (bad intelligibility) and 10 (perfect intelligibility) corresponding to an estimate of speech intelligibility. This score can then be used by doctors to get an idea of the impact of the disease and treatments (medical or surgical) on the quality of speech of their patients.

Demonstration of the application for reading the text of “la chèvre de Monsieur Seguin” (Mr Seguin’s goat)


The application is currently being used at Toulouse University Hospital to assess the intelligibility of patients suffering from head and neck cancer.

Contributors

IRIT – SAMoVA

  • Arcin Gautier
  • Farinas Jérôme
  • Pinquier Julien
  • Quintas Sebastião
  • Roger Vincent
  • Vaysse Robin

Toulouse University Hospital

  • Woisard Virginie

Toulouse-Tech-Transfer

  • Garzone Guillaume

Share this Post