First Glimpse at the Sound of Vision Prototype Device

The Sound of Vision device is designed to help low vision or blind users navigate both indoors as well as outdoors. It will achieve this via sensory-substitution – a complex set of video cameras, connected to a powerful, but wearable computer will ‘see’ in stead of the user and will provide orientation cues in the form of auditory information, via speakers that will also supply directional information. The device consists of two main parts: one is the headgear, which consists on a curved surface that is tightened on the user´s head, where the stereo cameras and inertial measurement units are. The other one is the carrying box, where the batteries, the USB hub, the sound processor and the main circuit board are contained.

The Headgear

As the Sound of Vision device must “translate” visual information to audio, it is very important that end-users also perceive the direction in which obstacles or other objects are located. For this, Head Related Transfer Functions (HRTF) are used. By characterizing how the human ear receives sound information, it allows devices to generate binaural sound that seems to come from a certain direction. In order to accurately measure HRTFs, the University of Iceland, as well as other institutions employ a so called Dummy-Head – a device which resembles the human head, torso and pinnae (the outside, visible part of the human ear), with electrostatic microphones inside the ear-cavities. This system may be used for the measurement of special filter coefficients, which provides information about the sound wave transfer mechanism between the sound source and the user’s ear canal.

In order for the device to accurately record the user’s movement, the cameras that record visual information must be placed on the user’s head. Also, the speakers that will provide audio cues must also be placed near the ears, in a way that they do not block sounds from the environment, but so that they allow synthesizing accurate binaural sound. To achieve this, the current prototype headgear is based on the comfortable inner structure structures for the headgear at a helmet store, located in Reykjavík, Iceland. The idea was to mount the headgear, and the multi-speaker system on a rigid, but comfortable surface, which fits onto the user´s head.

Next, taking the inner pieces of the structure of conventional construction helmets, the soft inner surface which often enables to ensure comfortability and strength, is attached to the headgear, which actually has the function of supporting the video-cameras and the multi-speaker system. The multi-speaker system has to deliver the user orientation and navigation clues for the environment that surrounds them, but which they cannot see. This multi-speaker system consists of four micro-speakers per pinnae. Two of them are placed on the upper and lower part of the sagittal plane, in order to give elevation cues for the listener, one is placed on the back of the pinna, and an auxiliary fourth one can be modular placed, for testing purposes. So, in this way, the HRTFs filters support sound localization in the horizontal plane, and for the elevation cues, constant power panning laws are applied to the up/down channels in order to enforce as well elevation perception. The mini-enclosures of the speakers were designed and 3D printed at the University of Iceland’s facilities.

The Carrying Box

The carrying box is where the “brain” of the system is located – the main system processor, the sound processor, various required connectors as well as the battery system that must ensure the portability of the device. For the first prototype, the focus was on ensuring the system is feasible for use and that is provides the end-user accurate, actionable information for indoor as well as outdoor orientation and mobility. Subsequent prototypes are expected to significantly reduce both the size of the headwear, as well as the dimensions and weight of the carrying box.