Artificial intelligence narrates the world around the visually impaired

Sensor-Tech Lab with the support of the Megafon national digital mobile operator has developed a free app for visually impaired people. The app, Object Detector, can recognize objects, read road signs and detect doors and steps. The app is one of a kind: it uses Russian, does not require internet connection and is available for free for iOS and Android users.

The app has three modes. In the Objects mode, it will describe the scene around the user. The neural network can recognize up to 80 objects and will help the user find everyday items at home (such as computer, dishes, hygienic means, food, etc.) and suggest on how to walk around obstacles on the street such as parked cars. The Road Signs mode reads street signs, helps find a bus stop and detect the end of the residential area. In the Door and Steps mode, the app narrates the scene around the visually impaired helping them find doors and entrances.

“Object Detector operates on neural networks that were trained using machine learning. Building the software was possible due to another development by Sensor-Tech Lab, a smart assistant named Robin – an AI device that recognizes objects and faces, determines the distance to an obstacle and then sends this information to the visually impaired user. Object Detector is aimed to improve the quality of life for people with vision impairments and to allow them to live more independently,” Sensor-Tech Lab general director Denis Kuleshov said.

“The app is Sensor-Tech Lab’s second development, implemented with the assistance from the MegaFon mobile phone operator. The Banknote Detector app, which was launched in 2018, detects banknotes and tells their value to the user with perfect accuracy; the app has been downloaded over 10K times and received numerous positive reviews from visually impaired people,” Valentina Vatrak, Director for Corporate Development and Human Resources at MegaFon, said.  

With its special tactile option, the Object Detector app can be used by deafblind people as well. Those can choose between various vibration alerts to notify them of objects or signs; as soon as the app detects them, the phone will vibrate.  

Previous ArticleNext Article