Vision Through Sound for the Visually Impaired

Amedi Amir, HUJI, School of Medicine - IMRIC, Medical Neurobiology
Hanassy Shlomo , HUJI, School of Medicine - IMRIC


Visual cortex recruited to see using alternative senses, "soundscapes" and touch


Artificial vision, Medical applications

Development Stage

Prototype has been used to train blind individuals to “see” using sound and touch after several training sessions


About 2.6% of the total population of the world are visually impaired of whom about 0.6%, around 38 million people, are blind



  • Artificial vision system for sight restoration using sensory substitution devices (SSDs), which utilize small cameras that convert visual information to auditory (or tactile) input.
  • Blind people use the visual cortex, which is normally used in processing sight, to process sound and touch and enhance their memory and language capabilities

Our Innovation

System involving a tiny video camera worn by a blind person to capture the surrounding visual field and convert it into a soundscape (or tactile matrix) using a set of novel algorithms. After training, blind individuals have been able to recognize the letters of the alphabet, “see” pictures of animals, and even find an object—such as a pair of shoes or a person—in a complex visual landscape.

Key Features

  • Depending on a person's motivation and ability, the basis for the seeing-through-sound language may be learned in 10 to 20 hours.
  • Low-cost, non-invasive system works for sight-impaired people of any age regardless of the cause or severity of their disability
  • Bestows independence on the sight impaired without need for surgery or additional aids such as guide dogs

Development Milestones

  • The next step is to introduce the system more widely to children and others and develop improved algorithms and friendlier set-up design
  • Seeking cooperation with teachers and organisations for the blind to recruit candidates to learn the system and try it

The Opportunity

Eventually, SSDs may be integrated into medical devices employing other inputs, such as touch and echolocation (similar to the use of sounds echoes in bats and dolphins), to comprise a multisensory substitution for vision.

Patent Status

Granted US 9,579,236

Contact for more information:

Aviv Shoher
Contact ME: