Microsoft’s  Seeing AI is an app that lets blind and limited-vision folks convert visual data into audio feedback, and it just got a useful new feature. Users can now use touch to explore the objects and people in photos.
It’s powered by machine learning, of course, specifically object and scene recognition. All you need to do is take a photo or open one up in the viewer and tap anywhere on it.
“This new feature enables users to tap their finger to an image on a touch-screen to hear a description of objects within an image and the spatial relationship between them,” wrote Seeing AI lead Saqib Shaikh in a blog post. “The app can even describe the physical appearance of people and predict their mood.”
Because there’s facial recognition built in as well, you could very well take a picture of your friends and hear who’s doing what and where, and whether there’s a dog in the picture (important) and so on. This was possible on an image-wide scale already, as you can see in this image:
But the app now lets users tap around to find where objects are — obviously important to understanding the picture or recognizing it from before. Other details that may not have made it into the overall description may also appear on closer inspection, such as flowers in the foreground or a movie poster in the background.
In addition to this, the app now natively supports the iPad, which is certainly going to be nice for the many people who use Apple’s tablets as their primary interface for media and interactions. Lastly, there are a few improvements to the interface so users can order things in the app to their preference.
Seeing AI is free — you can download it for iOS devices here.

The updated iOS app is available in 35 countries.Article Link: https://www.engadget.com/2017/12/14/microsoft-seeing-ai-app-update/
Artificial intelligence took center stage at Microsoft’s AI Summit in San Francisco on Wednesday. Aside from announcing AI smarts for a range of software — from Bing to Office 365 — the tech titan is also ramping up its Seeing AI app for iOS, which uses computer vision to audibly help blind and visually impaired people to see the world around them. According to Microsoft, it’s nabbed 100,000 downloads since its launch in the US earlier this year, which convinced the tech titan to bring it to 35 countries in total, including the EU.

It’s also getting a bunch of new features. The app now boasts more currency recognition, adding British pounds, US dollars, Canadian dollars, and Euros to its tally. Going beyond the color in a scene, it can also spot the color of specific objects, like clothes. Plus, it’s no longer restricted to just short printed text, with handwriting recognition now part of its skill set. You can also customize the voice that it uses to speak its observations out loud, and set how fast it talks.
Finally, a musical light detector alerts you to the light in an environment with an audible tone — Microsoft claims the tool will save users from having to touch a hot bulb or LED battery to check if it’s on. Despite the big update, there’s still no word on an Android launch.

by Emily Mullin

The prosthesis could help more people who have lost their vision than a device already on the market.

The maker of the world’s first commercial artificial retina, which provides partial sight to people with a certain form of blindness, is launching a clinical trial for a brain implant designed to restore vision to more patients.

The company, Second Sight, is testing whether an array of electrodes placed on the surface of the brain can return limited vision to people who have gone partially or completely blind. For decades, scientists have been trying to develop brain implants to give sight back to the blind but have had limited success. If the Second Sight device works, it could help millions of blind patients worldwide, including those who have lost one or both eyes.

The device, called the Orion, is a modified version of the company’s current Argus II bionic eye, which involves a pair of glasses outfitted with a camera and an external processor. The U.S. Food and Drug Administration has granted the company a conditional approval for a small study involving five patients at two sites, Baylor College of Medicine and the University of California, Los Angeles. Second Sight still needs to conduct further testing of the device and answer certain questions before starting the trial but hopes to begin enrolling patients in October and do its first implant by the end of the year.

Second Sight first won approval in Europe in 2011 for the Argus II, followed by an FDA approval in 2013 (see “Bionic Eye Implant Approved for U.S. Patients”). In Europe, two more retinal prosthetics have since been approved—one is marketed by French company Pixium Vision and another by German firm Retina Implant.

Also known as a bionic eye, all three devices are intended to bring back some vision in patients with a genetic eye disorder called retinitis pigmentosa. The disease causes gradual vision loss when light-sensing cells called photoreceptors break down in the retina—the tissue membrane that coats the back of the eye. An estimated 1.5 million people worldwide, including about 100,000 people in the U.S., have retinitis pigmentosa. That is a small percentage of the 39 million people worldwide who are blind, according to the World Health Organization.But Robert Greenberg, Second Sight’s board chair, says the company has only sold about 250 of the Argus II devices, a number lower than he expected. The device costs about $150,000 and restores minimal vision. Only 15 centers in the U.S. offer the technology, and with competition abroad, Second Sight is hoping its new brain implant could be used by far more people.

Second Sight’s Argus II uses a camera mounted on a pair of glasses to capture images. The images are sent to a small, patient-worn processor, which uses special software to convert the images into a set of instructions that are sent to the implanted chip near the retina. Those instructions are then transmitted as a series of electrical pulses to an array of electrodes, also implanted around the eye.

People with retinitis pigmentosa are able to benefit from the device because the disease destroys only specialized photoreceptors while leaving the retina’s remaining cells intact. These retinal cells are able to transmit the visual information along the optic nerve to the brain, producing patterns of light in a patient’s field of view.

The new device, the Orion, borrows about 90 percent of its technology from the Argus II but bypasses the eye. Instead, an array of electrodes is placed on the surface of the visual cortex, the part of the brain that processes visual information. Delivering electrical pulses here should tell the brain to perceive patterns of light.

“In some types of blindness, the optic nerve is damaged so you have to go downstream. With the Orion, we’re essentially replacing the eye and the optic nerve completely,” Greenberg says. With this approach, “anyone who had vision but has lost it from almost any cause could potentially be helped by the Orion technology.”Second Sight estimates that about 400,000 retinitis pigmentosa patients globally are eligible for its current device, but about 6 million people who are blind due to other causes, like cancer, diabetic retinopathy, glaucoma, or trauma could hypothetically use the Orion.

Greenberg hopes the approach will restore the same degree of vision as the Argus II, possibly a little more. Still, people with bionic eyes have limited sight. They can distinguish light from dark and recognize the outlines of objects in their view, but they can’t see color. Patient experiences also vary. Some can read small letters but others can’t.

A major downside is the device requires a more invasive surgery than the Argus II. A small section of the skull needs to be removed to expose the area of the brain where the array of electrodes is placed. Because electrical brain implants carry risks like infection or seizures, the first clinical trial will be small, and the company will start off by testing the implant in patients who are completely blind.

Last year, Second Sight tried out this approach, implanting an off-the-shelf neurostimulator device for epilepsy in the brain of a 30-year-old patient who had been nearly blind for eight years. The patient was able to see spots of light with no significant adverse side effects.

Researchers at Illinois Institute of Technology in Chicago and Monash University in Australia are working on similar artificial retinas that would connect directly to the brain.

Martha Flanders, director of the Central Visual Processing Program at the National Eye Institute, says a brain implant will be more difficult to get right than a retinal implant because the brain’s visual cortex is so much more complicated than the eye. Flanders says scientists are still in the early days of understanding how the brain processes images to produce vision and how neurons extract information from the visual cortex.

“If we could figure out how to process and filter visual information to correctly stimulate the electrodes we could eventually improve the type of image that person will be able to perceive,” she says.

Because of how complex the brain is, Mark Humayun, a professor of ophthalmology and biomedical engineering at the University of Southern California who spent 25 years developing the Argus II, says Second Sight will need to develop new software and algorithms to convert the visual information the camera picks up and turns into electrical pulses.

In order to get its new device approved by regulators, Humayun says Second Sight will need to show that the changes it made are worth the additional safety risks that come with a more invasive device.