MAIN CONTEXT PLANT BIOACOUSTICS DATA SONIFICATION INSTALLATIONS REFERENCES
The representation of data using sound (among other means) can help exploit our sense of hearing’s effectiveness in grasping complex contexts through both immediate orientation in space and intuitive classification of sound characteristics (Hermann, Hunt, and Neuhoff 2011). Sonification offers a deep and broad insight into multidimensional data, enabling us to recognise patterns and providing an aesthetic and emotional experience of scientific discoveries.
Sonification and aesthetics
Many data sonification experiments in the scientific field are characterised by their shortcomings for inexperienced listeners (Grond and Hermann 2012). Very little weight is usually given to aesthetic judgements in the generation and use of sounds in scientific data sonification: what is mostly sought is simply a clear differentiation between individual sounds; consequently, the artistic design of sonifications is usually rudimentary. Our aim was to increase the accessibility of our data sonification insofar that we set it up from the outset in accordance with musical criteria – that is, the biological connections and correlations corresponded with the harmony, dramaturgy, and emphasis of the artistically arranged sounds. Through these means, a generative piece of music was created that is controlled by the data flows used and that gathers the individual processes and phenomena into a holistic musical experience. An important goal here was to create an experience for non-scientific listeners of the life and life conditions of a tree.
Artistic and technical implementation
We integrated the recordings of the acoustic emissions of the tree into the sonification system in two different ways. (1) The constantly present crackling sounds that it is suspected arise from water transport and/or air bubble movements (Laschimke, Burger, and Vallen 2006) in the water transport vessels are played back in loop mode; the playback volume is controlled by the sap flow rate data. (2) The louder cavitation pulses are played back as audio samples (using a single recording of one cavitation pulse); their amplitude and pitch is controlled by the original amplitude and pitch measurements.
Weather sounds are also played as a sample (wind and rain). All non-auditory measurement data have been sonified; that is, measurement data series of individual physiological processes control the characteristics of individual sounds (= parameter mapping procedure, see the table, right).
The different sonification modules are implemented in a set of Max/MSP patches that replay measurement data from one month in spring 2015. For an adequate (temporal) experience of the most important processes, the speed of the running system is increased up to thirty-six times the normal speed – that is, the logged ten-minute measuring intervals. This is because a real-time playback of even just one day would demand too much patience from listeners and certain processes like the increase of cavitations around noon would hardly be perceptible with a normal playback speed.
Environmental data is mapped onto the outer side of the spatial audio system of trees: Pinus sylvestris (spatial audio versions), while tree data is played back on the system’s single speakers of the system, in accordance with the spatial position and geographic orientation of the sensors on the plant.
The sounds that we used to sonify the measurement data can be divided into two groups: field recordings (rain, wind, and plant sounds) and synthetic sounds. The majority of these phenomena do not manifest themselves acoustically; thus, generating metaphorical sounds that optimally portrayed a single phenomenon, for instance sunlight or air humidity, was a challenging task. For instance, for sunlight we used a string-like sound, for air humidity, a transformed and filtered burbling sound of a creek. All sound sources have a specific static or dynamic location within the audio system. The sun sound moves across the firmament, from east to west; wind comes from varying directions, louder if strong, softer if weak. As the sun rises, transpiration within the tree starts, and sap flow noises become louder. When the diurnal course reaches its peak at noon, cavitation pulses start, later decreasing during the course of the afternoon.
The most dominant sounds in this artificial representation of a tree’s life and its environmental conditions (besides the actual tree sounds) are the sonifications of the sun’s energy as well as air and soil humidity. This is because they are the key drivers in a tree’s ecophysiological cycles, enabling nutrient uptake, gas exchange, and growth (Larcher 2003). The sonification system renders experienceable the sensitivity of a plant’s reactions to the smallest changes in microclimatic conditions: with changing sunlight exposure, for instance, there is an immediate audible increase or decrease of the water transport sounds and rate. At night, it gets quiet: the plant’s transpiration is inactive, lacking solar energy. Air humidity increases (the pitch of the played sound gets higher), the plant refills with water (heard as very quiet cracklings) that it lost during the day. There are also audible seasonal patterns: the cavitation pulses increase in spring (between May and June) and dissappear in summer. New needles sprout in May – growth processes need more water and the drought stress increases for the plant. As it gets warmer and drier in summer, the plant starts to protect itself from water loss and reduces transpiration and production (Zweifel, Rigling, and Dobbertin 2009).
Our sonification experiments show that immediate and intuitive access to measurement data through sound and its spatial positioning is very promising in that it offers new forms of data display and generative artworks. Spending time in the system listening to the interplay of sounds and the phenomena that they represent during the growth period is a fascinating experience, which lasts about twenty minutes. Besides the diurnal course of the tree’s response to sunlight, there are many other recognisable patterns: as it gets drier, the cavitation events grow longer, sometimes lasting deep into the night; the stressed plant needs more time to refill with water from the soil. Likewise, there are more cavitation sounds when the plant is well drained and exposed to full sunlight than in very dry periods.
