We use cookies on our websites. Information about cookies and how you can object to the use of cookies at any time or end their use can be found in our privacy policy.
Google opens new communication channels for people with disabilities
AI Google Google IO 2 min read 1 Comment

Google opens new communication channels for people with disabilities

Speech recognition has evolved rapidly in recent years. However, it remains difficult for people with speech impairments to communicate. Google wants to change this with Project Euphonia.

Who of you already uses language assistants like the Google Assistant? Whether you use them regularly or only now and then, you will have certainly been annoyed more than once about the fact that it simply did not understand what one wanted from it. Now imagine you were trying to communicate to a bot with a speaking impairment.

For people who cannot communicate like many others due to stroke, multiple sclerosis, ALS, Parkinson's disease or any other neurological or motor disorder, Google can help. After all, communication, especially verbal communication, is a significant part of our lives. To address this problem from a tech angle, Google has created Project Euphonia as part of its Social AI team.

Project Euphonia

The aim of Project Euphonia is to use AI to improve speech recognition and the recognition of facial expressions and gestures. For example, they have teamed up with the ALS Therapy Development Institute (ALS TDI) and the ALS Residence Initiative (ALSRI) to record numerous people with ALS and use them as a basis for improving speech recognition.

Euphonia Waveform example sentences 01
Some of the sentences read by ALS sufferers for AI training. / © Google

During recognition, Google's software converts the recorded speech samples into a spectrogram or a visual representation of the sounds. The whole thing is then converted into the usual spectrogram to "train" the system, which is then supposed to recognize this less common type of speech better.

In the following video, you can see Dimitri Kanevsky, a linguist who learned English from this base after becoming deaf as a child in Russia. He already uses a live translation program that was trained with the help of Project Euphonia and helps him to communicate.

As seen in the video, it is also important to be able to recognize facial expressions, gestures and even sounds such as humming or clicking with the tongue in order, for example, to continue to provide an opportunity for communication in advanced MS people. Steve Saling, who suffers from ALS, for example, uses noise detection to control various smart home devices.

Source: Google

1 Comment

Write new comment:
All changes will be saved. No drafts are saved when editing

  • I love the idea but it needs to actually be available and affordable. Last week there was all the news covering a supposedly new thought-to-speech system that is only in labs and not available for people to try. Back in 2016 a company named Prose found out how to use an Emotive thought-reading EPOC headset to achieve the same thing. You can buy the headsets but three years later the software is still in beta and not taking new testers.

    Will this ever actually be available for the general public who need it or will it remain a laboratory curiosity? The best accessibility software has broad reach. Think of audio books: they are not just for the blind but also for anyone who wants to read faster, or while they are driving, or whatever. Everyone benefits not just people with disabilities. Now, think about this technology and people who are on vacation in a country where they kind of know the language but have a really thick accent and the locals can barely understand them. Or even don't know any words and are just trying to order off a menu. There is a great potential here. I hope it becomes a reality.