Editorial: On Sonic Information Design


David Worrall and Stephen Barrass


 

Sonic Information Design refers to the design of sounds to provide useful information in applications that have impact in our daily lives. The articles in this special issue of the Journal of Sound Studies on Sonic Information Design had their origins as responses to the theme of the 22nd International Conference for Auditory Display, held in Canberra, Australia in 2016.

 

Sonic Interaction Design recognizes design research as a valid method for contributions to knowledge in auditory display, drawing together related fields of data sonification, sound design and Sonic Interaction Design. In his introduction to the Third ICAD conference in 1996, the founder Gregory Kramer observed that while ICAD '92 and '94 saw more science-oriented efforts and theory-building, emanating predominantly from researchers in psychometrics and computer science, ICAD '96 was much more multidisciplinary and provided a stronger showing on the design front. He saw this as the maturation of a field with a distinct research agenda, which he modelled in a diagram with four quadrants: Science, Engineering/Technology, Art/Design, and Applications. He commented that support for ICAD from Aureal and Microsoft was, perhaps, a sign that some in the industry believed that the scientific and design research pointed towards viable commercial applications. Since then fragments of design research have appeared in ICAD conferences that include design patterns and participatory design methods. However, twenty years on, the lack of viable commercial applications points to the urgent need to make design research a high priority.

 

The articles in this special issue describe applications and present design research approaches that can help address this need. The articles fall into three broad categories: design theory, application, and evaluation. In the first category, Stephen Barrass’ contribution traces some of the epistemological foundations of auditory display and data sonification in the latter half of the twentieth century. These foundations have been challenged by the increasingly multidisciplinary input and contributions from practicing designers, including musicians coming from a variety of backgrounds and those with other areas of sonic expertise, most of whom do not use design methods other than empirical scientific ones to validate their work. Stephen Roddy and Brian Bridges turn their attention to discussing an issue so persistent in parameter-mapping sonification that it has its own name: the mapping problem. They outline an argument for addressing the problem through the use of embodied cognition and conceptual metaphors. They discuss examples of data-driven musical pieces that are informed by such models as well as a proposal for a sonic information design mapping strategy for a large-scale network-enabled physical objects: an internet of things network throughout Ireland. They contend that a mapping strategy that uses sound formation processes related to the data through listener-understood schemata and metaphors can help reduce confusions caused by the application of more arbitrary data mapping strategies that do not follow or include an embodied approach.

 

In the first article of the applications category, Milena Droumeva and Marc St Pierre discuss the design of a sonification for public engagement with socially relevant data, namely, urban air pollution levels of four Canadian cities. Their model details an approach to data parameter mapping which specifically considers public engagement issues as core to the design. They discuss usability and design implications for sonifying socially relevant information based on focus group evaluations. Mike Winters, Avrosh Kumar, Brianna Tomlinson and Bruce Walker introduce a planetarium – or “planetorium” as they call it – theme by discussing the strategies and designs behind two public experiences. Based on a mixture of previous research, in-person interviews, and creative interpretation, the first was an educational show in which the audience learned about the solar system by taking an auditory journey through space, and the second featured an informal, mobile, and real-time web-based musical sonification of the 2017 Great American Eclipse. These application examples demonstrate iterative user evaluation methodology as a method for understanding and developing a design.

 

The final two contributions demonstrate empirical evaluations for testing design hypotheses. The first, by William Martens, focuses on the sonification of proteomic data distributions. A Proteome is an entire set of proteins that is, or can be, expressed by a genome, cell, tissue, or organism at a particular time. The sonic information design for this study was guided by multiple criteria emphasizing practical use as well as aesthetics. Usefulness was measured in terms of whether or not listeners were able to hear differences in proteomic data associated with the different types of cells once the data had been transferred to sound. Techniques for aurally identifying each cell type include the mapping of spatiotemporal patterns in the data to the spatial distribution of sonic components, variations in timbre of non-spatialized components to create distinguishable sound, and the use of combinations of both timbral and spatial features. A number of experiments are presented, followed by a critical discussion of sonification methodologies as criteria for systematic evaluation. The final contribution to the issue by Keith Nesbitt and colleagues reports on the design and evaluation of the use of a simple informative sound that was added to a visual computer game. Player performance in various conditions, with and without sound, revealed that the informative sound display significantly reduced timeouts when compared with the visual only and constant sound versions of the task. Importantly, this reduction in timeouts did not impair the players’ performance in terms of their success rate or response time.

 

As a practice, the mapping of data into sound now ranges from the use of expressive musical forms within pragmatic applications in internet-connected devices to a growing use within empirical scientific research. Deep understanding of the role that design principles and practices can play in the effective use of sound in each situation, and why it should be implemented, is in its infancy. However, as the articles in this issue of JSS demonstrate, being aware of the power of a “designerly,” user-centered approach and taking on the challenges of sonic information designing in all its complexity is contributing to its establishment as a more mature design practice.



Table of Contents


 

 

Biographies


David Worrall (avatar.com.au) is a composer and polymedia researcher, known internationally for his music compositions for computers, traditional instruments, and data sonifications. His creative practice encompasses sound poetry, sound and polymedia installations, the development of software frameworks for composition, and the sonification of information in large or high-frequency multivariate datasets. He has published numerous journal articles and conference papers and several book chapters. David has been actively involved in the establishment of a several organizations, including the Electronic Music Foundation (USA), the Australia Council and the Music Council of Australia (which made him an Honorary Life Member in 2009), the Australasian Computer Music Association (including a 3 year stint as President) and the International Community for Auditory Display (ICAD) of which he is currently President. He is an editor for the journal Organised Sound (Cambridge University Press), a Professor in the Audio Arts and Acoustics Department of Columbia College Chicago, and occasionally performs with the electro band Australysis. His new book is about to be released by Springer-Verlag: Sonification Design: From data to intelligible soundfields.


Stephen Barrass is the CEO of sonification.com which is a startup dedicated to creating sonic information designs and products that make the world a better place through sound. He is developing methods for agile participatory design with clients and through collaborative workshops. His research lead to development of the Mozzi sound synthesizer for Arduino, and the MozziByte PCB for rapid prototyping of sonic ideas on the Arduino microcontroller. He holds a PhD in Information Technology (ANU 1998), a Bachelor of Electrical Engineering (UNSW 1986), and a Graduate Certificate in Higher Education (UC 2010).