|
|
IC2Hear: Designing Ambient Displays of Sound
Can the deaf see what we hear?
There are many subtle ways in which people use sound to gain awareness of the state of the world around them. For example, in an office, the sound of officemates working provides awareness of whether you are alone in the room or if you have company. At home, you might hear the television at night and guess that your roommate is up late. Similarly, many everyday devices use sound to communicate events. Examples are doorbells, ovens or telephones.
In 1995 there were 350,000 non-hearing people in the US who could not use regular auditory sensing to gain awareness of these events. Instead they employ a variety of alternative mechanisms to gain this information through other channels. Some examples we have discovered through interviews include the use of vibrations to detect footsteps, or periodic visual inspection of a window to see if someone is at the door yet. However, even with these mechanisms, there remains a gap between the awareness gained by hearing people and that experienced by the deaf. We have designed a peripheral display to provide the deaf with awareness of sound in an office environment to help close the gap.
Evaluating an ambient visual representation of sound
We designed a ripples visualization from paper prototypes, surveys and other design exercises with users (see Figure 1). We also implemented a spectrograph visualization, which is typically used in speech therapy applications (see Figure 2).
Figure 1: A screenshot of a cellular phone ring displayed by our spectrograph visualization. It is recognizable by its distinctive frequency and volume patern, which one participant described as a "caterpillar shape". In this mapping, the height of the sound corresponds to the sound frequency, the color to volume and the x-axis to time.
Figure 2: A screenshot of a cellular phone ring displayed by our positional ripples protoype. It is identifiable by its position in the room; The phone is on the desk in the room. In this visualization, an overhead map of the room is drawn on white. The sounds are represented by the rings with emanate out from the position of the source. The size and shape of the rings map to the amplitude of the sound.
Publications
- F. Wai-ling Ho-Ching, Jennifer Mankoff, James A. Landay, (2003). From Data to
Display: the Design and Evaluation of a Peripheral Sound Display for the
Deaf. In Proceedings of CHI 2003. 8 pages.
Paper (638 K PDF)| Slides (4.2 MB PowerPoint) | Video (30 MB AVI)
- F. Wai-ling Ho-Ching, Jennifer Mankoff, James A. Landay. (2003) Using
peripheral displays to provide the deaf with awareness of environmental
audio. Workshop paper accepted to the Elegant Peripheral Awareness Workshop at
CHI 2003. 4 pages. PDF
-
Can you see what I hear? The Design and Evaluation of a Peripheral Sound
Display for the Deaf. (2002) F. Wai-ling Ho-Ching. Masters Thesis. University
of California at Berkeley. PDF
Contact Information
|
|
