5 Design Strategies
Both works took a similar approach to design, utilizing creative interpretation, previous research on acoustic cues, and in-depth interviews. At the core, they were also motivated by the desire to engage blind and visually impaired members of the general public, while also attracting and engaging sighted users.
Some differences arose from the contexts of use. In the first example, the objective was to create an educational show that would premier at a planetarium and utilize a built-in sound spatialization system. In the second, the desire was for the sonification to complement or replace the experience of viewing a celestial phenomenon in a mobile, outdoor setting. Hence, the second example did not seek to “educate” its audience but rather to create a memorable aesthetic experience in which data-driven sound augmented the visual phenomenon.
5.1 Conceptual & Perceptual Research
Many of our mapping decisions were based upon previous work regarding perception by researchers in the field. Sonification designs of the mass of planets and moons drew upon previous polarity mapping research (Walker 2002). Tempo was used to create a comparable pattern for day length (Flowers 2005). By using easily referenced mappings that people can easily internalize into a mental model for comparison, we supported ease of initial understanding for these sonic representations. Physical movement, such as the revolution of planets around a sun, were scaled and mapped through vector-based amplitude panning (Pulkki 1997) to persuasively simulate movement of the planets around the audience.
Leveraging of conceptual metaphors, such as pitch representing temperature range, supported concrete comparisons between familiar and unfamiliar information (e.g., temperature on Earth vs. Neptune) (Dubus and Bresin 2013). Other uses of conceptual metaphors leveraged knowledge about physical phenomena: how a ball bounces under the influence of Earth's gravity, for example. Using this information as a reference for comparisons of gravitational strength on other planets (e.g., Jupiter or Mercury) can make it easier for someone to internalize and understand the mappings. Playing concurrent auditory streams can scaffold these comparisons, especially when representations are changing in a single dimension (e.g., pitch only) (Schuett, Winton, Batterman and Walker 2014).
5.2 Interviews
For the solar system sonification, we conducted interviews with 5 astronomy teachers, including a planetarium instructor, and analyzed data from a misconception identifier survey answered by 69 college students. In the second example, we interviewed two blind people as a basis for approaching the range of expectations and desires of the visually-impaired population. The interviews and questionnaires identified what key areas needed to be conveyed in sound, such as details for the size and scale of the solar system as well as specific characteristics of each planet.
A shift in the approach of design for the eclipse was noted after the interviews: the first interviewee had lost his vision as an adult and had seen an eclipse before, and the second was congenitally blind and had never experienced an eclipse. They both expressed interest in hearing a sonification of the eclipse but offered different perspectives on what was important to convey. The first interviewee had expectations based upon what he remembered and could imagine happening during an eclipse. For the congenitally blind individual, aspects relating to the amount of light or the colors were not as important as the tangible properties. Their understanding of the surroundings was strongly based on touch, surroundings, and sounds. They could describe the mass, temperature, and dimensions of the sun and the moon but could only imagine them as physical representations, and thinking of them as “illuminated” objects in the sky was not very meaningful. They felt that the relative brightness would be important to convey as well as the animal sounds of the environment (i.e., crickets, frogs, and birds), which often follow their usual circadian rhythms in close succession due to the “false-sunset” and “false-dawn.” More details on the planetarium interviews can be found in Tomlinson et al. (2017).
5.3 Creative Interpretation
To effectively use insights from previous research and interviews with end-users, both systems involved some degree of creative interpretation. In the solar system project, previous research and interviews had revealed particular concepts to focus on, also indicating how they might be mapped, but several factors were left to be determined. For example, although the mass, day length, and year length were mapped to pitch, tempo, and speed of rotation around the audience, decisions needed to be made about the timbre. We selected bandpass filtered pink noise to abstract the sounds from recognizable or overly-synthetic timbres, which we felt could distract the listeners’ attention or bring to mind unnecessary associations. Again, from a planetary science perspective, no research could be found on how one should convey the force of gravity, so we decided to use the sound of a ball bouncing on each planet for conceptual scaffolding.[2]
In the eclipse project we faced the task of creating a sonification piece that could capture the listener's imagination for the length of the eclipse. Our sound design therefore had two broad goals: 1) to convey the physical process of the eclipse to the listener as it progressed and 2) to induce awe and elicit an emotional reaction that one might feel when witnessing such an event. This second goal was creative in nature, requiring our sonic design to merge musical and data-driven composition techniques. The eclipse was represented as a process and abstract experience whose structure was based upon movements and events. Soundscape-inspired composition techniques were employed to introduce sounds of physical entities, and the sounds of birds and crickets marked a simulated experiential and environmental “false dusk and dawn” before and after the eclipse.