There are currently 285 million people who are blind or have low vision and are affected by the reduction in their perception and orientation capabilities.
Sound of Vision is a European research project currently funded within the Horizon 2020 framework which has an extremely ambitious objective – to build a wearable device that allows blind and low-vision users to perceive their surroundings based on sound and haptic feedback.
The three-year project started in 2015 with the participation of 9 partners from 5 European countries – 2 from Iceland, 3 from Poland, one partner from Hungary and Italy, and 3 partners from Romania. Project management is undertaken by the University of Iceland, while the University Politehnica of Bucharest is responsible for technical coordination. The project consortium is structured to include 5 universities, 2 organizations for the blind, 1 research institute and a private company. Partners share a 4 million Euro budget provided by the European Union.
As we near the end of the project, the current device is in a TRL-8 prototype status and integrates hardware, software and training practices into a unitary whole. The device consists of a headgear which includes 3D and depth-sensing cameras that allow perceiving both outdoor as well as indoor environments, an inertial measurement unit, and a haptic belt that is worn around the waist and which provides sensory feedback in the form of directional vibration.
The accompanying software receives real-time information from the cameras and identifies the important objects in the scene – stairs, doors, potholes or obstacles. Each detected object is assigned an audio and haptic representation that is relayed to the user in real-time, ensuring that changes to the environment are perceived immediately. Several representations are available both for the audio and the haptics, and users can change to the representation they are most comfortable with.
Available training practices help users master device functions, get accustomed with the audio and haptic feedback and gain proficiency in understanding and navigating the environment. Training is undertaken using a series of increasingly complex virtual environments, followed by training in a controlled environment under the supervision of mobility instructors.
The system was intensively tested using a large number of users and underwent several stages of iterative improvements based on user feedback. Results are extremely promising, with trained users being able to quickly grasp the particularities of the environment and navigate it dynamically. The upcoming steps imply perfecting the device and adapting it from the prototype phase to that of a commercial product with an acceptable entry-cost.
The public presentation event took place on November 27th within the Senate Hall of the University Politehnica of Bucharest, with the participation of Romanian officials, project researchers and the public at large.
