Facebook and Twitter go different direction in image description tech, new apps from Microsoft and Google

Big news from big tech giants this week.

Photos play great role in social networks growth and engagement. No wonder facebook and twitter are trying to improve things in this end for millions of blind and visually impaired people.

Both Facebook and twitter announced tech that will describe photos to blind users. Though they took completely different approach.

Facebook uses artificial intelligence to understand whats in a photo and describe it using words. Using AI will cover most of content. Automated process doesn’t require a lot of friction from the real users, so Facebook is not suggesting that people need to describe their photos for blind people as Twitter suggests.

. In case of user generated description – the quality and relevance of the text will be better then AI for now, but with huge amount of photo content Facebook can feed to AI, situation can change massively.

Microsoft also uses AI to help visually impaired and blind people, but with more trivial problems. People can use their new app as dog eye, to get the idea of whats going on around them, read the menu in restaurant, describe people near them and so on.

If you are android developer, and eager to help visually impaired people, if you want your app to be easy to use for everyone, including people with visual or other limitations – Google made a step forward to help you.
Google guys build free app that can go through every screen of your application and detect parts like text or buttons that can be adjusted the way they can be easier to interact with.

Small reminder that android accessibility guidelines can be found here – http://developer.android.com/intl/ru/guide/topics/ui/accessibility/index.html

Looks like weekly digest is turning to monthly digest

Hey folks, been a while)
Was busy as crazy pushing tons of new things at my current work at . Also this weekend was participating in NHS hackaton in London. Was great experience, also nice to see that such conservative and slow moving machine as NHS is trying to get closer to patients and solve their pain points.

Now back to our topic)

Tactile maps

Using tactile maps for navigation is huge problem for visually impaired people. Though technology and enthusiasts are trying to solve this problem.


Using the university’s 3D printers and modeling software undergraduate Jason Kim and his professor Howton Lee from Rutgers University School of Engineering created floor navigation for The Joseph Kohn Training Center in New Brunswick, New Jersey. The Center provides blind and visually impaired students with free, 20-week training courses that give them the skills they’ll need to attend college, secure jobs or become independent homemakers.

The map outlines all the rooms, and there’s a legend with different shapes representing bathrooms (a circle for the men’s room and a triangle for the women’s), elevators (squares), and stairways (multiple lines).

Maps That You Can Hear and Touch

Guys from LightHouse went slightly different approach. They have created tactical maps printed on embossing paper and are using livescrap-pen, so users can tap on icons on the map and get detailed audio instructions. Map is used to help bling people to navigate on the BART transit stations in San Francisco.

Toyota Blaid

Unexpectedly, one of the biggest car manufactures is trying to help visually impaired people too with its project Blaid

Toyota is doing wearable that will

  • Detect surrounding objects, including signage, and identify restrooms, escalators, stairs, elevators, doors, exit signs, and familiar storefront logos
  • Enable users to better explore indoor spaces, including airports, office buildings or shopping malls
  • In the future, integrate mapping, object identification, and facial recognition technologies

Omaha company invests in high-tech glasses for blind employees

Outlook Nebraska Inc. in Omaha, the state’s largest employer of blind people, bought 6 eSight glasses for their employees.

ESight technology itself looks interesting. They moto it as “Electronic glasses that let blind people actually see.” Which sounds promising but maybe limited only by several health conditions. Though blind people can try this tech for free, so worth checking.

Digest bonus

By the way, if u are into web tech, finishing university, interested in accessibility and want to work in dream company – facebook is hiring web tech intern, more details here https://www.facebook.com/careers/jobs/a0I1200000IABmE/

Thats all for now, have good weekend

Weekly digest

Happy holidays everyone
I have tons of materials i would like to share with readers on tech helping blind people, but have limited time, so i decided to write weekly digest to cover most interesting articles.

After quite holiday time lets start with some fun.

Blind gamer Terry Garrett is completing 3d game – legends of Zelda. Really amazing and nice to see that people like Garrett can enjoy things we think are inaccessible for visually impaired people. He finishe Oddworld: Abe’s Oddysee before Zelda, which was insane complex game for me, and i was not able to get to the games end. So Terry looks like real hardcore gamer.
He is using stereo-speakers that help him to orient in the game.

Make sure to check more videos on his youtube channel

Now back to tech. Nowadays almost all IT giants have their own solution on picture recognition. Some of them are even developing these solutions not only for commercial purposes, but actually to help blind people to “see” – i wrote a little bout this initiatives in previous blog post Teaching machine to see and recognize.

Fortunately there are some number of mobile solutions on the market made by enthusiast and researchers.

Joseph Paul Cohen, a Ph.D Candidate at the University of Massachusetts Boston build free android app called BlindTool. App can identify an object after you point your phone on it within seconds

More info on how it work can be foundhere

Another cool app called Bemyeyes is connecting volunteers with blind people, establishing video connection between them in case blind need some urgent help with “seeing” something. For visually impaired people, small domestic tasks can be challenging, and app can be in handy in this case.

Latest mobile OS have strong accessibility features, more and more apps coming to market to help blind people, but what of visually impaired person doesnt know how to use smartphones?

Some interesting news from Switzerland, which have now approximately 325,000 blind or visually impaired people. There is a school teaching bout 250 visually impaired people on how to use iphones to make their life easier.


Hacking kinect

IF you are really smart in tech, creative and want to solve real problem for people, u can find out that building product or tech will not cost you a fortune.

In 2011 several different groups of people were prototyping navigation assistant for visually impaired people using hacked xbox kinect.

Kinect is using infrared sensors to detect motion and distance to objects.

You can read more on how it works http://www.jameco.com/jameco/workshop/howitworks/xboxkinect.html.

And this tech appeared to be real handy to detect obstacles for blind people, all u need some computer to track the information and signaling system to warn the user.

Audio channel is not the best way to inform blind people during the navigation. In this case the whole sense is used for just one task, and people are loosing the way to get more info on surroundings, which really is creating more problems then solving.

So one talented team of enthusiasts went even further with kinect concept and have created Belly-mounted tactile matrix to worn user about approaching obstacles.

“Each tactile pixel is disassembled electromagnetic relay much like this with a curved paper-clip attached. Voltage is applied periodically so that pixel prods person several times a second. The more frequency is, the closer is the object under that pixel. Depth perception made that way is not very accurate, but even non-trained person could easily distinguish ‘very close’ from ‘about one meter’ from ‘about three metres’.”

Project won second place at Russian Imagine Cup at 2011.

More info on these projects:

Teaching machines to see and recognize

Tech giants Facebook and Google are working hard on improving machine learning capabilities. They want machine to understand pictures like human do it.

They have different purpose of their research – google is using this technology to power their Photos app, Facebook is improving accessibility of their platform.

Both tech can be in handy for visually impaired people.

Here is the latest example from facebook

Both companies are using neuron networks. And whats interesting there, that with evolving of technology, blind people will be able to get real time audio description of surroundings using onhead cam for example. Of course pushing all the info on whats near you through audio doesnt make sense, though using assistant and asking question can do the work.

Imaging this concept but using camera on your head instead of steady image from your facebook feed.

Using Dash cam and image analyze your smartphone will be able to predict the obstacles while you move, where there is a puddle or pole ahead and you need to turn to avoid it.

You can read more on what is Facebook doing on image recognition here:

and Google

Hello Folks

My name is Sergiy Voronov. I am UX/UI guy, working for UK startup www.clinpal.com. In this blog i will be writing bout the topic that excites me the most now – user experience for blind people.

Nowadays 285 million people are estimated to be visually impaired worldwide: 39 million are blind and 246 have low vision.

Due to tech bum (smartphones with gps and audio, bluetooth beacons, Braille smartwatches ,  intelligent personal assistants and other cool gadgets) life of blind people can be little easier.