Figure 1: "The Good Ship Hibernia"
Embodied and Aesthetic Mapping Strategies in Data–driven Music Composition
Sonic information design, as noted earlier, is the application of design approaches to choices of sonification mappings, including a focus on relationships between “form, function and content” and user experience aspects, including accessibility and aesthetic and affective aspects (Barrass and Worrall 2016). As embodied cognition is, in part, a response to the symbol grounding problem, it could provide an important framework for sonic information design, addressing relationships between a sonification’s form, its function, and its content. In addition, as embodied cognition has also been applied to music theory, it may also provide insights into aspects of aesthetics. It is in this context that Roddy and Bridges (2016) present and discuss the mapping strategy for “The Human Cost,” a piece of data-driven music in which the mapping problem is addressed in the adoption of a design approach informed by embodied cognition principles.
AudioObject 2: “The Human Cost”
Data–driven music composition is a subset of sonic information design which foregrounds aesthetic factors in a sonification with the aim of sustaining a listener’s engagement. The concern for aesthetic factors in sound mappings may also encourage a listening strategy which prioritizes a search for correlation and causality within and between the different elements of the sonification. “The Human Cost” uses socio-economic data sets from the period preceding and following Ireland’s post–2008 economic crash and recession. The piece is informed by Michael White’s (2003) observation that we use the notion of a living organism as a conceptual metaphor for the economy. GNP is mapped to control a parameterized heartbeat sound, providing a rhythmic grounding for the piece. Deprivation, unemployment, and emigration are mapped to control parameters of three synthesized vocal gestures, created in CSound using FOF synthesis, a hybrid of granular and formant synthesis techniques (Clarke 1992). The data is mapped so that the leading voice (representing emigration data) takes the foreground, while the other two voices (deprivation and unemployment) provide a sonic backdrop. Data is mapped to control the pitch vowel shape in each vocal gesture so that, as the economy worsens, the open vowel sounds shift to closed vowel sounds, and pitch increases to communicate a sense of tension, providing an embodied association through the corresponding increased tension in the throat and facial musculature with closed/higher vowel sounds such as “e” and “i” (Durand 2005).
From a culturally–specific perspective, this is informed by a similar approach to the structure of pitch and prosody found in old Irish laments, a type of song sung at a wake, a social gathering which was often held to honor either a deceased relative or a relative who was emigrating with no prospect of return. The backing vocal sounds move around the stereo stage. As the economic data worsens, the speed of this movement increases. The piece adopts the concept of the lament as a conceptual metaphor for the data represented, utilizing synthesis parameters to evoke vocal emphasis and stress patterns which are similar to the pitch and vowel contours found in laments. Furthermore, for narrative purposes, the piece is broken into two sections. The first section is marked by dynamic filtering of the spectral content to alter the timbre so that it sounds more diffuse and “dream-like.” This filtering is no longer present by the halfway point, as the 2009 data is heard. This aesthetic and narrative effect is intended to reference the painful realization, in the years directly after the financial crash, that the dream-like optimism and unrestrained growth of the “Celtic Tiger Era” was now a thing of the past.
AudioObject 3: “The Good Ship Hibernia and the Hole in the Bottom of the World”
Roddy (2017) explores another embodied cognition approach to representing data from the Irish financial crash and recession in the data–driven composition “The Good Ship Hibernia and the Hole in the Bottom of the World.” This piece makes use of soundscape recordings which are mapped to represent Irish GDP growth rate from 1979 to 2013. The piece is structured using the conceptual metaphor of a maritime journey where “smooth sailing” and “good weather” represent “good times” and “rough seas,” and “bad weather” represents “bad times,” a conceptual metaphor first discovered and studied by Izabela Zołnowska (2011). The piece is further structured on the basis of Johnson’s (1987) balance schema in so far as data is communicated as a function of the balance between two soundscape elements in the piece. This balancing was achieved by simple amplitude mapping. Harmonic material, consisting of an improvised guitar performance in response to the shifting soundscape of the sea journey, is also heard throughout the piece. There are two distinct forms of harmonic material present a foreground melodic component and a background chordal accompaniment. The individual notes and chords are not directly determined by the data but are rather an interpretative response to the shifting soundscape. This was intended to provide a background context against which the changes in GDP might be rendered more obvious. The perceived audio fidelity, salience, and timbral character of the improvised material is also determined by the GDP data, with low GDP mapping to a noisy timbre and loss of amplitude (so that the harmonic material is overcome by the sounds of the storm). The piece was composed as a conceptual blend where the maritime weather metaphor provided the framing structure and the data was mapped to determine how the piece unfolds. The improvised harmonic material was intended to provide a baseline against which changes in the soundscape could be interpreted. This approach is illustrated here.
