The human being learns to speak and read correctly not only by observing the letters but also by repeating the sounds presented – an obstacle in the literacy of deaf children. An application that uses artificial intelligence and augmented reality can help in this process: StorySign, launched in Brazil by Huawei.
The app has existed since 2018, but only now has it been adapted to Portuguese, thanks to the work of the translator and interpreter of pounds Edílson Andrade and the proofreader Erik Honorato, who translated two national children’s books into sign language: Gildo, by Silvana Rando (Brinque-Book), and A Festa Encrencada, by Sônia Junqueira (Ática). Edílson gave life to the avatar that tells the stories in the app.
StorySign exists in 15 languages, and is present in the United Kingdom, France, Germany, Italy, Spain, Portugal, Australia and Brazil, and has a library in Libras of 71 books. There is no prediction of when the application, which is free and has support for Android and iOS, will have more titles here.
How StorySign works
1. With the StorySign-enabled reading book in front of you, the child opens the app and holds the smartphone over the page.
2. The app’s avatar (called Star) tells the story in pounds while the app highlights each gestured word.
3. The child learns the spelling of the word and its translation into pounds.
According to a statement from Huawei, StorySing was developed in partnership with the European Deaf Union, the British Deaf Association, the editorial group Penguin Books and Aardman Animations, among other institutions.
“The application captures movement and animation, ensuring that the facial expressions and hands of the characters convey the perfect representation of Libras, creating a fun virtual character,” said the company. In Brazil, the Chinese giant had the support of the National Federation of Education and Integration of the Deaf (Feneis).