Seeing AI is a Microsoft research project that uses Artificial Intelligence features to deliver an intelligent app, designed to assist visually impaired people with performing everyday tasks such as reading text, recognising people’s faces, identifying products, and identifying your surroundings. This app is fully accessible with VoiceOver and magnification features on iPhones.
How it works:
The app has nine different ‘channels’, which you can use for performing a specific task. Below is a brief description of each of the channels.
Short text:
Simply point the camera at text and it will be read aloud. Very useful for reading signs, text on noticeboards or even on food tins etc.
Document:
Hold the phone camera over a document such as a magazine, newspaper or letter and the app will automatically take a picture of the document and read it aloud to you. It will also try to format the document for you as well, so you have an idea of how the page is laid out.
Product:
Hold the camera over the barcode of a product and the app will scan it and then tell you the name of the product. You can also find additional information about the product such as cooking instructions or ingredients. Barcodes are sometimes difficult to locate on certain products, but a really cool feature of the app is that audible beeps are given to help with the locating and scanning of barcodes.
Person:
This channel allows you to take a picture of a person’s face and the app will then attempt to work out the age of the person and tell you what they look like. Note that you can also train the app to recognise people’s faces as well, so if you point the camera towards them, the app will actually tell you who they are.
Currency preview:
Point the camera at different types of currency such as Euro, Dollar and Pound to hear their values.
Scene preview:
Simply point the camera in front of you and take a picture. The app will attempt to describe everything in the view of the camera. Note that the Scene channel is still being developed, so is not always accurate in describing certain things. Hopefully it will improve over the next few months though.
Colour preview:
Just point the camera of your phone at any object and its colour will be announced. Note: lighting conditions are a factor.
Handwriting preview:
This experimental channel allows you to take a picture of handwriting and the app will attempt to recognise it and read it to you. Note: the text has to be the right way up for this channel to work.
Light detector:
The camera on the phone will detect the amount of light around you. This works by using pitch sounds. The higher the pitch, the more light there is.
First impressions
I have used this app for around two months and am really enjoying most of its features. The barcode and text reading facilities along with the facial recognition features are extremely accurate most of the time. The colour, handwriting and light detecting features also have huge potential, as does the scene preview mode. To be honest though, I am a little unsure about the currency feature, as now that plastic notes are being printed, all £10 notes have Braille in the top left corner. On the whole though, I really feel that this app has made a big difference in terms of increasing my independence on a daily basis. Now I don’t need to depend on friends or family to read my mail or find products in my freezer for me. The fact that all these features are available in one easy to use app is fantastic!
Price:
Seeing AI is free to download and use.
Note: This app is currently only available for Apple products and requires a constant internet connection.
Article meta data
Clicking on any of the links in this section will take you to other articles that have been tagged in the same category.
- Featured in the Karten Autumn 2019 Newsletter
- This article is listed in the following subject areas: Other, Technology