Sounds are probably the most important information for blind users when they navigate. Numerous research papers have shown in the past that blind and visually impaired people become exceptionally good at identifying the location of the source of a sound around them. So why would someone in their right mind play sounds in a blind person’s ears?

Well, this article will try to summarize our approach, and why we believe that sounds are the ultimate feedback that can guide blind people.

The types of sounds we play

First off, let’s start with a description of the types of sounds that biped plays. We focused on playing extremely short sounds, of less than 100 milliseconds, that are as intuitive and as easy to remember as possible.

biped uses 4 types of sounds:

• a “bip” sound for very close obstacles with a risk of collision
• a smoother “bip” sound for further obstacles that user should be aware of
• a “bell” sound for all moving elements
• a sound for crosswalks

The bell sound for moving elements is actually very easy to remember. Bells have reverberation. They can easily convey a notion of movement.

• If we play a high-pitch bell sound, it means that there is a pedestrian, a bicycle, or an electric scooter coming your way.
• If we play a medium-pitch bell sound, it means that there is a car or a motorcycle coming your way.
• If we play a low-pitch bell sound, it means that there is a truck or a bus coming your way.

Users can then easily differentiate between a pedestrian walking and a person on a bicycle based on how they hear the sounds moving.

Non-intrusive sounds

Won’t using headphones block the rest of the sounds around you? This is maybe one of the most common questions we get. But bone conduction headphones have evolved a lot recently! These headphones are worn around the ears, and do not block any other sound around the user. This is the first thing we test with our beta-users: can people handle a conversation with us, biped sound feedback and the busy street environment. This is something that we have validated quite early on when building biped, and results were surprisingly good!

Speed of information

biped very rarely uses speech, except for GPS feedback. The main reason behind that is that speech is slow to convey information. To give a rough order of magnitude, this is how fast we manage to convey information to the human brain with various methods:

• Vision has the highest band with, with over 100 Megabytes per second theoretically
• Ears only allow the processing of 100 bytes per second, 1 million times less.
• Speech, as a sub-part of audio feedback, only convey 39 bytes per seconds on average. Again, 3 times less.

It remains hard to identify similar metrics for haptic feedback (vibrations and tactile information), but understanding a complex pattern provided by vibrating motors remains fairly slow.

If we had to inform a user about the position of a bicycle on the left,

“bicycle, 6 meters, 10 o'clock”

Even if we double the normal speaking rate of a person, this still takes 750 milliseconds to play. If there are 3 important objects around you, this will take above 2 second to give you feedback, and it will be impossible for you to maintain a conversation at the same time.

Hence, 3D audio appeared as the perfect solution. You play a very short music note, and users can understand:

• the type of objects, depending on the sound you play
• the angle, with the 3D effect
• the distance, with the volume

This represents a 7X speed improvements over speech, while making the overall experience more enjoyable.

Filtering information level

You might at this point be concerned that biped starts playing sounds continuously. However, by default, biped will not produce any sound. The only cases sounds will be played is when there is something that requires your attention. If you’re walking in the middle of a large street, you won’t hear anything. If you walk on a narrow sidewalk, a light sound of the left will warn you aout the position of the wall, so that you can keep walking straight without having to touch the element with your white cane.

Finally, when you are in a crowded area, this is where biped reveals itself as the most useful in our opinion. You will only get warnings if a person has a risk of collision with you. The second they shift trajectory to avoid you, the sound stops.

Additionally, you can enable and disable specific sounds based on your preferences. If you enter a hallway of a train station and don’t want to hear sounds for pedestrians as you assume that they will avoid you, it will take a single tap on your smartphone. If a persons have a direct risk of collision, would will still get warnings as an obstacle.

Cities become quieter

Electric scooter, electric bikes, electric cars… All these vehicles help make cities a quieter place. However, there are to date, no regulations regarding the sounds an electric vehicle should produce for safety reasons. And this becomes increasingly dangerous for blind and visually impaired people. You can think of biped as a device giving sounds back to these vehicles.

Different from everything

We designed the sounds of biped with a gaming studio that worked on some of the biggest PlayStation games ever. We converged to a solution where the sounds biped plays are very different from anything you could hear in your daily environment. That way, there is not risk of confusion with any other sound of your environment.

Richness of the information

Audio feedback is rich. When you play a game, a whole environment is built around you in an immersive way. Instead of conveying a single information, i.e. is there something in front of you as vibration feedback would do, biped can tell if what this object is, where it is located, how fast it moves, and if it is important or not. This really open an entire new space!

Training time

Training time and potential adoption where crucial elements when building biped. We wanted users to get started with biped in as little time as possible. We reached a stage where the first introduction only takes 15 minutes. Then, 3 additional modules of 15 minutes each can be completed. 3D sounds are extremely intuitive to listen to, and most end-users finish the modules in lesser time than planned.

Universal language

The music notes that biped plays are a universal language. An end-user in the United Stated will hear the exact same sounds as a French user would. This is important to build communities and to leverage crowd feedback in order to make biped a better product over time.

Limitations

For all the aforementioned reasons, we believe that 3D sound feedback is the way to go. However, a deaf person with a visual impairment could not use biped in its current version. We can however play the sounds on any hearing aid that is Bluetooth-enabled. Future versions of biped will offer vibration feedback for deaf and visually impaired users.

‍