6. Discussion
We will first discuss the design in terms of perceivability, representational aesthetics (how well it evokes the subject of air pollution), and its role in generating fruitful dialogue about climate issues and city infrastructures. Second, we will touch on what this evaluation process reveals about the requirements for sonifying socially relevant data, not only in terms of perceptual considerations but also with regard to symbolic mappings and aesthetics.
6.1. Macro Patterns in Perceivability and Representational Aesthetics
The shared listening activity revealed some common perceptual impressions when comparing Vancouver and Toronto in terms of sonified representations of air pollution. All participants in both groups correctly identified Toronto as the more polluted city, with the most salient perceptual characteristic that emerged in the follow-up discussion being density. Over half of the participants referred to Toronto’s air pollution sonification as “intense,” “more intense,” having a “higher state of urgency” and producing an “ominous” feeling, in contrast with the sonification of Vancouver’s air pollution, which was described by a couple of participants as more “mellow” and “subdued” with a “calmer” sound. One participant described Vancouver as having a “foggy” sound only occasionally punctuated by urgency (referring to the dynamic increase of noise bands and particulate matter clicks). Not surprisingly, the clicks representing particulate matter were one of the most distinct elements that all participants from both focus groups picked out and identified. Because of the somewhat cyclical patterns of pollution itself, the sonifications displayed rhythmic characteristics, as a result of which participants tuned in specifically to the differences in tempo of the different sonic elements. Potentially this is another reason the Geiger counter-style clicks stood out the most and were very strongly symbolically associated as referring to a sense of “urgency” in the sonification. Interestingly, the “higher intensity” of sound that Toronto was characterized with elicited recognition of more “distinct” elements, while being affectively interpreted as “disheartening,” producing a sense of tension, fear, and discomfort: Figure 3 presents a word cloud of descriptive words used by participants during group discussion.
Figure 3: Word cloud of participant comments describing the Vancouver and Toronto air pollution sonifications.
Aside from the PM2.5 clicks, the other most memorable and recognized element from both sonifications was the instance of clipping noise bands in the Toronto sonification (we decided to let pollution values drive noise band dynamics into a distortion range, towards a more emotive representation): those were referred to by several participants as “thunder” or “thunderbolts” and were experienced as quite dramatic. In general – and not surprisingly – all participants perceptually identified frequency of clicks, higher pitched harmonic identities, and dynamics (in that order) as the most perceptually salient components of the sonification. Everyone considered the overall soundscapes to be fairly pleasant to listen to and could imagine listening to them for long periods of time. Both focus groups identified clarity of elements, such as distinct harmonic streams, and accents, such as clicks and “thunderbolts,” as important elements that worked to balance against a pleasant overall listening experience (see Figure 4 for details).
Figure 4: Tabulated view of sonification parameter mappings against feedback words that participants used in describing the Vancouver and Toronto air pollution sonifications.
The individual listening activity (see Figure 2) was an attempt to probe the limits of possible data comprehension for everyday listeners so as to assess the likelihood of non-experts grasping any important and persistent patterns or relationships in the data. While most participants expressed that the more exposure they had to the data, the more it “made sense,” some common challenges emerged from all three tasks. First, the temporal resolution – that is, the ability to correlate heard and graphed data – was too difficult for all participants both in following along and charting tasks. This points to one of the central problems with perceivability in sonification as a tool for scientific analysis: the audible resolution of hearing variation is quite different for an average listener from the visual resolution of seeing graphed changes. While listeners dutifully compared incoming sound with previously heard sound, the presence of many small changes confused them and made it difficult to find these events on a graph. As one participant put it, he was only able to distinguish change between 3-4 stages of intensity, and any more were too difficult to follow or recreate. This is of course a specific problem involving integrating visual and auditory feedback simultaneously, but it still suggests important considerations for the design of data listening situations (with realistic expectations). This finding was confirmed in the graphing activity as well, where most participants tended to draw what they were hearing with far greater resolution than the visual timeline allowed, resulting in running out of space while the audio was only half-way through. In elaborating on this, most participants explained that this was directly connected to scaling issues: determining the relationship between increases in amplitude, pitch, or intensity/density and the visual curves on a graph. Most participants found it easy enough to distinguish between plateau (flat) moments in the data, and peaks (dynamics) yet experienced difficulty identifying one peak from another beyond three or four stages of change, despite the relative shortness of the example (1 min) and reduction of streams (two rather than five at a time) for the individual tasks.
As mentioned earlier, the data itself is relatively cyclical (e.g. a flare up of O3at sunrise, and a flare up of CO at dusk – rush hour, etc.), which makes the resultant sonification rhythmic particularly in terms of the time-based interplay between different elements. While participants attended to these temporal patterns, it is important to note that this came as a secondary observation in the individual listening tasks, given the need to focus on pitch and dynamic shifts in order to follow and graph change. In response to the question of what could have helped facilitate both hearability and intelligibility in this listening situation, several participants mentioned temporal pauses in between sonified time periods of data in order to help with temporal resolution but also reset the ear for change. Additional suggestions included an interactive display that would allow the user to scrub the audio: scroll back and forth through the sound file at a user-controlled speed in order to compare different sonic moments of the graphed information and comprehend relationships between elements more fully.
6.2. Proposed Guidelines For Sonification Design For Public Engagement
Both focus group discussions, which centered on “what should pollution sound like,” also produced some fruitful observations towards a model of sonification for public engagement. In terms of what we would call representational aesthetics, most participants found the symbolic mapping of more pollution to more noise bands/higher amplitude/faster clicks to be an intuitive one, conveying a continuum from noisiness or fuzziness toward clarity or precision that effectively represented the relationship to more or less pollution in the air. Some participants suggested that, conversely, silence could be emotionally impactful in a different way, as a “lack” of livability. One participant noted the use of “…constructed modulated sound, because it gives you that feeling that pollution is made by humans.” A slightly different design idea that also emerged out of this discussion proposed perceptual mappings, rather than symbolic parameter mappings of the soundscapes, which would produce a physical, experiential effect: in this case to mimic the health effects of air pollution. Several participants expressed a strong affinity for the idea that this type of sonification be aimed at producing a sense of fear, discomfort, and “stuffiness” – physical responses to poor air quality:
I don’t know what pollution should sound like but I know the feeling it should invoke, kind of like being in a small place, hiding, having a hard time breathing. (focus group 2)
[Sonification of pollution should be] something that should disturb you, scare you, put you in a heightened state. (focus group 1)
Overall, the results from the two focus groups reveal both expected guidelines in terms of well-supported perceptual mappings as well as important considerations concerning the politics of translation of data to sound that have to do with the relationship between the subject matter and the emotive context of the sonification. While higher pitched harmonic identities, frequency-based clicks, and amplitude dynamics emerged as perceptually salient, it was the “noisy” quality of harmonic noise bands that communicated the “drama” of pollution to participants most strongly. Aside from discernable change in each sonified dataset, each city (e.g. Vancouver and Toronto) conveyed its own “feel” – an overall quality of fuzziness, fogginess or noisiness versus clarity or distinctiveness of the data soundscape – enabling participants to perceive one as more polluted than the other. Further, given the proliferation of audio-visual information, our study points to the importance of exploring sonification in relation to infographics, line graphs as well as visual displays. And while participants certainly had difficulty discerning meaningful changes or patterns with accurate temporal and value resolution, the mere fact that average listeners were able to graph air pollution change within relative parameters already speaks positively to the potential and usefulness of sonification to inform a civically engaged public.
