An app powered by artificial intelligence (AI) that can read speakers lips even if they do not enunciate words is available for those willing to collaborate in a study aimed at facilitating smooth conversation for people with speech difficulties.
The lip-reading AI technology, developed by Takeshi Saito, associate professor of intelligence information studies at the Kyushu Institute of Technology, can recognize numbers from zero to nine as well as 15 words or phrases such as “a-ri-ga-to-u” (thank you) and “ha-ji-me-ma-shi-te” (nice to meet you) in the study.
As it only analyzes lip movements, the app can determine particular words even when no sounds are produced. Using data collected from 48 students and fed into the AI, the study team achieved a correct interpretation rate of 71 percent.
photo credit to: https://www.theguardian.com
People who have lost their voices owing to larynx cancer or other reasons can feel frustrated or lonely, as they are unable to speak while still being able to move their lips, according to Saito.
In the field of the voice recognition, researchers have developed AI technologies that can determine not only words, but also sentences with a high correct interpretation rate.
Smartphones and car navigation systems employ voice-recognition technology equipped with AI to analyze a speaker’s voice, follow instructions and even have a conversation.
When those who have lost their voices will move their lips in front of the camera lens of a smartphone, Saito’s dream technology will allow AI to swiftly read the movements and turn them into sounds to be played over a speaker.
The tech can be applied to other settings and systems such as the car navigation devices otherwise hindered from recognizing a driver’s instructions owing to noise such as music and conversation among passengers.
With the assistance of a hospital in Okayama Prefecture, the team used the app to analyze movements of the lips of elderly people who had lost their voices. However, the correct interpretation rate in analyzing samples of those elderly persons was considerably lower than that for the students.
Reference: AI-powered app gives voice to people with speech deficits
