Conclusion
The above mapping strategy is intended to give listeners an overall sense of the health of the Pervasive Nation network at any given time. As such it is designed to help the listener to easily determine normal (read “healthy”) patterns of activity from abnormal patterns of activity.
AudioObject 20: Bad Network
As the prevalence of noise increases, SNR ratios decrease for more and more nodes. As reverb increases, nodes are transmitting their payloads through an increased number of gateways, indicating connectivity problems. High SF factors, indicated by long message lengths, suggest the same problem. As pitch tends towards randomness, nodes are displaying increasingly erratic patterns of RF switching. The increased “glitching” of messages indicates bad MIC codes, suggesting possible security issues. Continual reboot sequences indicate problems with gateways, and continual rejection messages indicate problems with devices attempting to connect to the network. The mapping strategy aims to render each of these problematic patterns of activity understandable to a listener by using a relevant conceptual metaphor – machines talking to other machines – to frame the auditory display in terms of vocal communication. The auditory display is thus less arbitrary for the average listener skilled in the interpretation of patterns in speech vocal communication.
Conceptual blending and conceptual metaphor, as applied to design in HCI by Imaz and Benyon (2007), provide overarching guidance for the IoT mapping strategy presented above. This approach helps to address the mapping problem by advancing design frameworks which result in less arbitrary and more conceptually relevant mapping strategies. We believe that this approach provides a meaningful grounding that prevents sonification and auditory display solutions from becoming so arbitrary that they are difficult to interpret. The example presented here is currently under active development, and future empirical testing will be used to guide further refinements. It is presented here as a demonstration of how principles from the field of embodied cognition (in particular, conceptual metaphor, image schemata, and conceptual blending theories) might be applied to sonic information design.
To summarize, the application of an embodied sonic information design process, such as the present one, begins with a consideration of the metaphors used to discuss and reason about the domain under study. The chosen metaphor should result in the use of a recognizable sonic referent that is compatible with and effectively represents the data set. The conceptual blend is then considered as an aid in thinking about how features in the data space might be mapped to features in the metaphor space. Consideration is finally given to the technical implementation (i.e. parametric mappings) so as to flesh out the general approach to the mapping strategy.
One shortcoming of this approach is that there may not always be an obvious or dominant metaphor to draw upon in representing the data source. For example, different aspects of a single phenomenon might be conceptualized using a number of different metaphors, and this in turn would require a more complex data to sound mapping strategy. Furthermore, some metaphors will have no obvious sonic associations and metaphors can change between languages and cultures, making this approach culture-specific. Another limiting factor at play here is the level of difficulty in synthesizing and controlling the sounds suggested by the guiding metaphors. Whilst our example was designed to exploit established techniques for synthesizing human vocal sounds, such straightforward approaches to sound synthesis are not feasible in every case. Therefore, the approach presented here cannot act as a universal solution for designing meaningful data to sound mapping strategies. We suggest instead that designers consider data–to–sound mapping strategies informed by embodied cognition principles. Image schemata, conceptual metaphors and conceptual blends open up radical new ways for thinking about sound and can help designers to create better mapping strategies.
In the context of sonic information design, considering the representation of data with sound through the lens of conceptual metaphors, conceptual blends, and image schemata can open up possibilities for the use of new sonification parameters beyond pitch, duration, and timbre. In the examples explored here, prosodic features, environmental sounds, perceived audio fidelity, and formant profiles become important parameters for representing data. While these might not exemplify groundbreaking new parameter sets in and of themselves, as important as the question of what parameters should be used is the question of how they should be applied. This is addressed in the example of the IoT network where the framing metaphor of M2M communication acts as a guide not only for what parameters should be used (speech–like parameters), but also how they should be used, namely as organized around logical rules of conversation. This is also reflected in the structure of “The Good Ship Hibernia and the Hole in the Bottom of the World,” where the mapping strategy is largely defined by a conceptual metaphor. The conceptual metaphor offers a mechanism for reducing the perceived arbitrariness in a data to sound mapping strategy by grounding the design in a familiar and interpretable domain of experience.
If the mapping problem can be addressed, as we in fact believe possible, by reducing the arbitrariness in a data–to–sound mapping strategy by considering it using the framework of conceptual metaphors, then how is this to be achieved when the data represented has no clear sonic referent and no implicit link to any particular sound source or process? We argue that, as demonstrated in the previous examples, designing a mapping strategy using sound processes that can be related to the data (or data source) through shared image schemata, conceptual metaphors, and conceptual blends can help to reduce arbitrariness in the sonification’s framing, allowing the listener to consider the data in terms of other (similarly–structured) domains with which they are familiar. In this context, even relatively generic embodied metaphorical associations (the tension/relaxation axis of the vowel sounds and the verticality schema of the “reboot” sound gesture in the Pervasive Nation IoT mapping) may provide a sufficient element of “grounding,” based on familiar embodied experiences, when encountering an unfamiliar or relatively abstract data domain.