Sensor-Tech Lab, a resident of the Skolkovo Foundation, has been developing technologies to help people with hearing or visual impairments socialize since 2016. Two years ago, the company released an AI-based app to read the face value of Russian banknotes, which was downloaded 10K times and has good reviews. It also recently released an object identifier app for iOS and Android platforms, with the support of the mobile operator Megafon – a free app now available on the App Store and Google Play. In one of its three modes, it identifies and sounds out household items.
Another function reads pedestrian traffic signs. The third one warns about doors and stairs. The app can use AI offline. It cannot estimate the distance to objects because this function is impossible to implement on mobile devices. Yet, the app can be the first step for users who want to explore modern technologies and AI, something to start with.
Over the past few years, Sensor-Tech engineers have been working on the Lab’s main project, Robin the smart assistant. Although it is similar to the object identifier app, Robin’s functionality is much wider. The bot recognizes objects and people’s faces, helps the user avoid obstacles and reports how far they are.
In an interview with Invest Foresight, Sensor-Tech Lab CEO Denis Kuleshov eagerly spoke about the original Robin project for people with impaired vision: “It is a smart assistant that combines a smart video camera and artificial intelligence in one. With the help of its cameras and sensors, Robin can ‘look around’ and recognize various objects, obstacles, and can tell how far they are to help its blind user navigate their environment. At the same time, Robin is not a replacement for a white cane, but a companion who can tell where things are – a car, a pedestrian crossing, an armchair, and other household items.”
According to Kuleshov, “Robin was originally meant to work offline without using cloud storage or servers – the device does all the computing using its processor running on original software.
Recognition takes only about one-fourth of a second – that is, items can be identified instantly. Announcing the result takes another half a second. If you point Robin at items around you and Robin sees a car, in about half a second it will say “a car.” If people walk by, in another half a second it will say “a person,” “two persons” or “three persons.” Robin recognizes everything so fast a user would hardly notice the delay.”
Robin can also recognize faces – although, unlike security systems, Robin does not have a database of faces. But if you upload a person’s photo and assign a label to it, Robin will name the person if it sees them.
Wireless Robin is powered by a battery. Like mobile phones, it can switch between different modes and operate continuously or occasionally. When used continuously for recognizing many items one after another, Robin’s battery charge can last up to two hours. If used on and off, Robin can work for up to five hours. “Its run time could be increased but that the device would require a better battery which would affect its weight and price. The device is most commonly used for trips to and from work and for walking so we came up with adequate capacity parameters for these purposes. During the day, users can charge the device at their workplace,” Denis Kuleshov explains. “Robin has a useful life of five years like any other computer equipment. But hypothetically, computers and laptops can operate for ten years and even longer. Of course, new models and functionality are released from time to time and users want to replace their old devices. In this respect Robin is no different from any other computer. Essentially, it is a mini computer equipped with AI.”
Denis Kuleshov adds that “research institutes and various startups come up with similar devices once or twice a year. However, only a few companies have worked their way up to a serial product that can be actually offered to consumers. In addition to white canes with embedded smart sensors, there is only one device in the world that is similar to ours: an Israeli company manufactures a device mounted on glasses that narrates texts for the visually impaired. It can also recognize faces and barcodes in shops. This device costs twice as much as Robin; it was developed four years ago and is now sold all over the world. Robin is only sold in Russia through distributors – major Russian companies engaged in delivering such devices to people with disabilities. These are Istok-Audio Group and Business Bureau. They sell them through catalogues and via their retail networks. Fifteen more devices were distributed in Russia via social charity events for the blind with donations and competitions, and people wanted to receive Robin.”
“Speaking of the conditions in Russia, it is important that our devices are included in the government list of rehabilitation devices because 95% of them are purchased by the Social Insurance Fund through the public procurement program and provided free of charge to the visually and hearing impaired and people with locomotor system disorders. The retail cost of high-tech devices for people with disabilities is very high: an ordinary family can’t afford to buy Robin for RUR 150K ($2.1K). Robin is not listed there yet, so Sensor-Tech Lab is negotiating with all relevant agencies. This will help both Sensor-Tech Lab and many other companies. There is no relevant algorithm for innovative products yet, and the list has not been updated for years,” Kuleshov said.
Sensor-Tech Lab has filed a PCT application and two patent applications in Russia. The company plans to launch additional models and will continue to receive patents for new devices.
“Along with Russia, Sensor-Tech Lab also operates through its distributors in CIS countries, where sales are due to begin soon,” Denis Kuleshov said. “The company was planning to enter export markets between February and June but the efforts were hampered by the COVID-19 pandemic and the global crisis; currently, Sensor-Tech Lab is intending to start operating in the United States, where an agreement has been signed with a local partner and sales will begin at major online marketplaces such as Amazon and eBay. Then the company plans to enter markets in the European countries where the device has already been presented; last year’s presentation in Germany sparked much interest,” Kuleshov said. In European and Asian countries, Sensor-Tech Lab will work only under agreements with local partners that will help it develop a marketing strategy and will have contacts with local distributors as well as communities of people with disabilities.
Robin’s localization takes about two weeks; the process includes modifying the interface and object sound files as well as translating and printing the work manual. In two weeks, Sensor-Tech Lab will be ready to export Robin to any country. “Currently, the device supports English and German; in the US market, it will support English and Spanish,” Denis Kuleshov noted. “In the future, certain difficulties may arise in the countries where some interior objects and road signs look different from those in Russia. Robin’s AI system has to be retrained to recognize new objects; while entering a country that has partners and an attractive market, we have to conduct analysis to detect objects that could pose difficulties, promptly build a database of such objects, collect their images made from different angles, and retrain the AI system to provide efficient work of the device in a new environment. This process is semi-automated for the countries where no retraining is required and everything can be done within three weeks, while it will take up to two months to adapt the device to a new environment in the countries with drastically different traditions and customs,” Kuleshov said.