This startup created an app that allows the visually impaired to ‘see’ the world, through artificial intelligence

Technology has now become the eyes and ears of the urban population. We ‘see’ through our smartphones, which present a layered image of our reality, and we ‘hear’ what our devices tell us, from something as diverse as the correct pronunciation of ‘dengue’ to music recommendations by YouTube.

While many have argued about the sustainability of this almost-complete dependency on our smartphones, or technology on a larger scale, there can be no denying the fact that technology has helped make our everyday lives not only more interesting but significantly easier as well.

Image : shutterstock

Today, technology is why we are thinking of taking our names to the moon, why people across oceans can see and talk to each other every other day, and why the disabled no longer feel handicapped.

Keeping to the idea of progress, and in certain cases empowerment through technology, Australian-based startup Aipoly devised an app of the same name to help the visually-impaired by using machine-learning to identify objects, all without the use of the internet.

Aipoly the app is touted as the ‘artificial intelligence for the blind’. In their promotional video, the company asked several visually-impaired employees, most over the age of 50, to use the app on an object and see the results. While some used it on everyday objects like headphones, others pointed it towards the sky to understand if it was under or above the clouds. In all these situations, the app was paramount in providing them with the answers, making them ‘see’ the world again.

Available only on iOS, the app is simple to use. All the user has to do is to open it, point the rear end of the device towards the object and click, and Aipoly will be able to identify it for them.

Since the app runs directly on the user’s iPhone, it doesn’t require the internet, giving it an edge over its counterparts like Be My Eyes and TapTapSee. It also can identify objects one after the other as the phone is being moved around, sparing the user the tedious process of having to click a separate picture for each object they wish to identify.

Aipoly uses a technique called ‘deep learning’, a kind of machine-learning technique inspired by studies of the brain. It uses a technology that breaks down the image into its most basic forms – straight lines, curves, circles, and other various patterns – and then proceeds to liken these images to a specific object. This is the same kind of technology used by companies like Facebook and Google to recognise faces and search images, respectively.

But like most apps in their initial phases, Aipoly contains a few flaws. For instance, it can only recognise up to approximately 1000 images as of now. Other than the fact that it is limited exclusively to iOS, it does not recognise objects a 100 percent right. However, Jeff Bingham, a professor in human-computer interaction at Carnegie Mellon University, speaks about the app’s effectiveness in differentiating between similar objects.

“What’s a lot more useful is telling things apart that are otherwise indistinguishable, like cans or boxes of food that feel about the same,” he says, as stated by MIT Technology Review.

However, as a counter to this problem, Simon Edwardsson the Co-founder of Aipoly speaks about an added feature to the app that will allow users to enter a correct word or phrase to identify an object that it is getting wrong, and that data will then be uploaded to Aipoly’s servers. As a result, the startup will add descriptions to Aipoly every few weeks or so, as it releases the subsequent newer versions of the app. He also aims for the app to recognise up to 5000 images with time, as compared to its current 1000.

As for the users, they couldn’t be more delighted to ‘see’ and ‘hear’ of the colour of their friend’s shirt or the right kind of celery at the grocery store. “I would love to take it to the wild: Take it to the gym and see what equipment is there, take it hiking, go on a walk with my kids and see what we’re all seeing and explain things to them,” says Lisa Maria Carvalho, the Vice President of the Lighthouse Center for the Blind, who swears by the app.