Mobile application for automatic intelligibility measurement
Context
Intelligibility can be defined as an analytical, acoustic-phonetic decoding notion – i.e. addressing “low-level” linguistic units, referring to the quality of pronunciation at the segmental (phoneme and syllable) levels.
Speech intelligibility is usually assessed by speech therapists using perceptual tasks. While these are informative, they rely on subjective ratings, biased, among other factors, by the familiarity of the rater with the subject’s speech. They are also usually time-consuming because several assessors are needed to obtain a more objective measurement.
Overview
Our work involves developing a mobile application that can be used by therapists to automatically assess intelligibility. The core algorithm of this application is based on deep learning algorithms, so as to obtain a reproducible score that can be used to evaluate the evolution of the intelligibility score for the same patient.
The principle of the application is to have a patient read a short text (less than a minute) on a mobile device such as an Android or iOS tablet. The audio recording is then processed by our machine learning algorithm, which returns a score between 0 (bad intelligibility) and 10 (perfect intelligibility) corresponding to an estimate of speech intelligibility. This score can then be used by doctors to get an idea of the impact of the disease and treatments (medical or surgical) on the quality of speech of their patients.
The application is currently being used at Toulouse University Hospital to assess the intelligibility of patients suffering from head and neck cancer.
Contributors
IRIT – SAMoVA
- Arcin Gautier
- Farinas Jérôme
- Pinquier Julien
- Quintas Sebastião
- Roger Vincent
- Vaysse Robin
- Woisard Virginie
- Garzone Guillaume