Microsoft recently released a smartphone app that uses computer vision to describe the world for the visually impaired.
The Seeing AI allows users to point their phone’s camera at a person and it’ll say who they are and how they’re feeling. They can also point it at a product and it’ll tell them what it is. All of this is done using artificial intelligence that runs locally on their phone.
“Just hold up your phone and hear information about the world around you,” Microsoft promises.
It seems almost magical when you first hear about it. But the even better news is that Microsoft is far from alone in tackling this problem. Apple, Google, Facebook and Amazon are working on similar projects.
The app works in a number of scenarios. As well as recognizing people it’s seen before and guessing strangers’ age and emotion, it can identify household products by scanning barcodes. It also reads and scan documents, and recognizes US currency. This last function is a good example of how useful it can be. As all dollar bills are the same size and color regardless of value, spotting the difference can be difficult or even impossible for the visually impaired. An app like Seeing AI helps them find that information.
This is also the latest project from Microsoft built using its AI technology. The company has also released tools for video game developers to build games you can talk to, for instance. Microsoft also built a Twitter program called Tay that it hoped could simulate the sort of conversations a teenage girl might have.