From Particle Data to Particular Sounds: Reflections on The Affordances of Contemporary Sonification Practices
Thomas Bjørnsten
Particle soundings – an introductory case
"You tend to personify things that you think about a lot […] I think electrons, perhaps, sound like a glockenspiel to me.” (Prince 2011)
The above quote stems from an NPR/Science article describing how scientists are trying to think of alternative ways to deal with huge amounts of data generated by particle physics experiments related to the Large Hadron Collider (LHC) project. An impressive technological and scientific endeavor of the 21st Century situated within CERN (the European Organization for Nuclear Research). LHC experiments seek to give us insight into very big and fundamental questions concerning, for instance, the origin of our universe. Equal to such big questions, the LHC currently produces a vast quantity of data, currently amounting to 30+ Petabytes per year and definitely qualifying for the otherwise loosely-fitted term “Big Data.”[1] The response of the scientist interviewed here, Lily Asquith, refers to a core question relating to one of the problems that many scientists have to tackle, namely what to do with the big data sets continuously generated from diverse experiments. The focus of the story concerns how Asquith has been working with employing a type of sonification method as something which involves certain affordances for making sense of the big data generated from the LHC. She explains this idea by referring to an often-used example within discussions of such affordances, relating to listening-as-monitoring, likening the idea of sonifying LHC-data with the functioning of signal sounds of heart monitors in a hospital: "You don’t have to watch the monitor because you can hear it without making any effort […] Just a steady beep—you can quite easily detect if it starts going quicker or if it stops even for a second” (Prince 2011).
It should be noted here that when the concept of “affordance” is used in this article, it denotes the variety of “possibilities for meaningful engagement with and within an encompassing situation,” as summed up by Mark Johnson, who ascribes this notion to the psychologist J.J. Gibson (Johnson 2015: 31). Johnson furthermore, and importantly, relates this view on affordance as experience and engagement with the conception of aesthetics “as a field of inquiry” which is seen as “an investigation into everything that makes these experiential affordances possible and gives them whatever meaning they have for us” (Johnson 2015: 28). Thus, the idea of “aesthetics” does not merely encompass questions about specific formalizations, for instance musical renderings, according to certain cultural traditions, but concerns the way we fundamentally sense, respond to and cognitively reflect upon the world around us and its plenitude of phenomena.
Bearing this in mind and returning to the rather short interview with Asquith, she delves briefly into the basics of the actual mapping procedures. This is where the informed reader might consider these explanations quite superficial, referring to very rudimentary concepts of sonification in relation to what she “thinks” electrons might sound like. However, the interview article also links to Asquith’s blog “LHCsound” on which a number of Asquith’s sonification experiments related to datasets derived from different LHC-procedures are described. Here it becomes clear that Asquith and her team have worked quite elaborately with this idea and that the work also comprises a, by now discontinued, website: “LHCSound – The Sound of Science.” The home page of the website is playfully designed to look like a working desk from which the visitor can access different elements of the project of sonifying LHC-data. For instance, a link guides you to the headline “When Protons Collide,” and on this page one finds three different sound examples to be listened to, accompanied by a text describing how basic aspects of the data have been mapped to “pitch, vibrato speed, attack and resonance.” The immediate impression of these sonifications is that they resemble simple compositions, containing varying timbral and rhythmical structures and, thus, involving a certain degree of aestheticization that we may also consider as including specific musical qualities. Elsewhere on the site, one furthermore comes across brief and general considerations about the sonification methods behind the actual sounding results. What is particularly interesting in the context of this article is, for instance, the description of how the “raw data (streams of ones and zeroes)” are converted “into ‘objects’” from which particles are “reconstructed.” This step is then followed by another step through which “the information” is converted into so-called “breakpoint files" which can finally “be read by compositional software.”
It is obvious that the website has been designed with a purpose of research dissemination, making the scientific project accessible to an interested, non-expert public. This leads us to the initial reflections about one of the central affordances of this type of sonification practice, namely that it can be used to communicate properties of otherwise abstract and intangible data, directed toward and received by our sensory apparatus. The general activities surrounding a huge scientific venture such as the LHC are experimental and of a highly complex character, which makes the outcome partly impossible to grasp for a broader public. The LHCsound website and its apparent purpose seems thus to parallel another example of sound-oriented project linked specifically to the LHC-experiments. This following example stems from an article and video clip posted on The Telegraph’s website under the title “The sound of science: Higgs boson data turned into music at CERN” (Knapton 2014). The written story presenting the accompanying clip starts with the following description:
It is the musical representation of some of the most cutting edge experiments in the world. Yet when seven physicists from CERN created a composition based on measurements from inside the Large Hadron Collider the results were surprisingly traditional, with even a hint of baroque. (Knapton 2014)
The article then goes on to account for this surprisingly traditional result as being generated from the collection of data from LHC-experiments by way of a process “known as sonification” (Knapton 2014). In the clip we see and hear a seven-piece ensemble playing traditional-sounding classical music using traditional instrumentation, situated within various settings in the LHC-environment. In the written part of the text, the person behind the composition explains it as “a metaphor for collaboration” thematizing the collective effort of making big science. However, the article ends with a clarification, much in the vein of the article mentioned above, of some of the basics of sonification in general, concerning, for instance, mapping and value assignment and concluding with how “any regularity in the scientific data can be naturally mapped to the melody,” in this case, Higgs Boson data related to the so-called ATLAS and CMS experiments, described in further detail as “a lead ion collision from 2010, and data from the first observation of a heavy-flavored spin–3 particle that was seen in July.”
The point of bringing this last example forward is, again, related to a kind of discrepancy, also brought out through the wording of the interview article, between the level of complexity of the LHC-experiments and the resulting, not very complex, sonic interpretations of the data derived from those experiments. Even though the leading composer hints at the project being a “metaphor,” the article ends by referring to the series of specific scientific experiments that have allegedly laid the ground for mapping the properties which would define the final musical composition. In both cases we are thus presented with a view of sonification as a practice that involves a high level of interpretative freedom (including the choice of musicalized expression and effects) as well as being intrinsically linked with specific scientific datasets.
[1] For an elaborate discussion of the term see for instance Geiselberger and Moorstedt (2013).
Sonification between exploration and (aesthetic) communication
The reason for including the two above examples as a point of departure for this article’s further analysis is that they can be seen as typical of the way in which ideas of sonification are communicated in a mainstream and popular cultural as well as popular scientific context. Considered from this perspective, before completely leaving the case of Lily Asquith and her team of sonification-interested particle physicists, we shortly turn the attention toward the LHCSound-blog that functions as a continuation of the aforementioned website. Here one finds less explicitly “musicalized” examples of LHC-data sonifications, for instance, under the title “Higgs: 2011 data – part 2” with a supplementary headline asking “Does time exist?”. Besides referring to what has been one of the central undertakings of the LHC, i.e. the search for the Boson Higgs particle, two aspects of the discourse about sonification related to these experiments are interesting here: For one thing, the playable sound files are associated with somewhat elaborate scientific notes documenting the particular experiments, and elsewhere on the blog’s sound file pages, links to long summaries of scientific reports can be found. Thus, a much more direct relationship is established between the strictly scientific findings of a highly specialized research community and the sounds we, as a general public, have access to listening to. For another thing, on the page mentioned, an articulate reflection on the sonification practice as such can be read. Most important here is the distinction between two “phases” of data sonification, the first of which is described as “exploratory.” This initial phase is described as a procedure where you “try lots of different mappings and, based on what you hear, what you know, and what you’d like to know, you modify those mappings or throw them away and try something completely different to see what you can see (or hear what you can hear).” Concerning the second or other “phase,” this is described in relation to the process of finding a “mapping that reveals a significant structure” which can then be “re-use[d] in a different context, one which is less exploratory and more communicative.”
These statements seem, then, to describe the sonification practice as something involving a continuous negotiation between exploring data openly through different mapping tools in order to arrive, as a kind of end result, at a particular poignant selection of those mapping attempts that have a strong communicative quality. However, this practice and the way its different steps are explained poses several questions that will be discussed in the following. For instance, to reiterate the above, when something like “raw data (streams of ones and zeroes)” is mentioned, and the further “reconstruction” from those data is termed as “information” which is subsequently converted to file formats that can be read by compositional software, then we may very well ask: what are we actually listening to on the LHC-sound website? Is it primarily a sonic representation of a software algorithm rather than an more directly correlating (in terms of granularity) sonic rendition of a dataset derived from the physical phenomenon of particle collisions? And what does it mean to conceive of data as “raw” compared to a notion of “information” which can be translated into sound through a designated interface?
These questions will be further discussed in the following parts of this article, but already at this point we can discern between two different modes or phases of sonification derived from the above description: first, an exploratory mode or phase of data sonification which is suggested to involve a closer connection to the “original” or “raw” data. And secondly, an intention of communicating significant findings via this exploratory practice. These two modes can, by now, generally be considered as accepted within a common sonification-terminology. In the most comprehensive volume yet published on the general topic of sonification, The Sonification Handbook from 2011, Barrass and Vickers furthermore propose that we consider representational connectivity to the data in terms of “tightness,” referring to a semiotic continuum of stronger or weaker indexical relationship with data (Hermann, Hunt and Neuhoff 2011).[2] A “tight connection” involves a high degree of correlating factors (i.e. granularity, temporality etc.) with the data while including many “artistic constraints,” whereas a less tight connection would, of course, entail weaker directly correlating factors, however allowing for “full artistic freedom” (Barrass and Vickers 2011: 157). Here the reference to artistic freedom corresponds with the authors’ overall discussion of “aesthetics,” which is linked both to notions of musical sound and to that which has to do with “sensuous perception” (Barrass and Vickers: 157).[3] Barrass and Vickers conclude their thorough discussion of the potential found within an aesthetic approach to sonification by suggesting that we may think of reconfiguring it as a practice “from an instrument solely for scientific enquiry into a mass medium for an audience with expectations of a functional and aesthetically satisfying experience,” partly as a result of the fact that, in the wording of the authors, “[d]espite the promise of sonification to provide new insights into data there is little to show in the way of scientific discoveries made through sonification in the past twenty years” (Barrass and Vickers: 165).
However, in the following discussion below, we will ask whether it can be considered sufficient for such an “aesthetically satisfying experience” to include composers solely for the purpose of designing auditory displays? And if making sonification “a medium for communicating information about data sets to a broad music-listening audience” is actually able to bring forth the potential achievements of exploring data through aesthetic and artistically-based processes? (Barrass and Vickers: 166).[4]
[2] It should also be noted that my discussions here are indebted to the important work on the topic by Alexandra Supper (cf. Supper 2012).
[3] However, I would like to add here that another kind of distinction can be made between sonification and ”audification,” the latter being – according to, for instance, Sterne and Akiyama – always conceived of as “indexical in that patterns of sound are directly caused by patterns in the data” (Sterne and Akiyama: 549).
[4] This is a discussion which also relates to what Sterne and Akiyama have referred to as the “push-pull between art and science […] played out in sonification discourse” (Sterne and Akiyama: 551).
Data imagined as data
Leaving these questions open for now, it will be a central argument here that before we attempt to tentatively answer suggestions about how to better incorporate aesthetic approaches within the sonification of data, we should ask what we understand by data in the first place? In the introduction to a recent anthology, Raw Data Is an Oxymoron, Lisa Gitelman outlines how a broader and more thorough understanding of the vast amounts of data that now make up part our everyday lives is needed; we have to realize that data is always “familiarly ‘collected,’ ‘entered,’ ‘compiled,’ ‘stored,’ ‘processed,’ ‘mined,’ and ‘interpreted’” (Gitelman 2014: 3). Gitelman asserts that the interpretation is often conceived of as a posterior phase of overall data-handling, but that it is nevertheless not obvious how this interpretative activity “haunts its predecessors.” Crucial to our discussion here is her emphasis that “[d]ata need to be imagined as data to exist and function as such, and the imagination of data entails an interpretive base” (Gitelman 2014: 3).
As was the case with the initial example, i.e. the LHC-data serving as point of departure for both exploratory and more ‘communicated’ or musicalized sonifications, that process and venture was explicitly described, in the wording cited above, as derived from “data” and “data sets.” According to Gitelman, we often tend to think of data as “the starting point for what we know, who we are, and how we communicate,” which repeatedly leads to “an unnoticed assumption that data are transparent, that information is self-evident, the fundamental stuff of truth itself" (Gitelman 2014: 2). Gitelman’s discussion and part of the anthology’s overall inquiry is, thus, seeking to highlight and make clearer the ways in which various disciplines’ use of data is based on “how different data sets harbor the interpretive structures of their own imagining” (Gitelman 2014: 3).
This observation is crucial for how we understand the relationship between the sonification of apparently “objective” or “primary” data and the knowledge-generating activity of hearing and listening, i.e. the interpretative act that is usually thought of as “coming after” the initial processing and transcoding procedure. Here we may also turn our attention to the discussion put forth by Veit Erlmann in which he addresses the a priori conditions for listening and meaning, which makes him ask “[w]hy and how certain orders of knowledge [do] make some aspects of our auditory experience more worthy of attention than others” (Erlmann 2010: 18). Erlmann points to how certain conditions must be given “for something to become recognized, labeled, and valorized as audible in the first place” (Erlmann 2010: 18). This sonic contextualization seems to be highly relevant when dealing with the affordances of sonification at large, but even more so when it comes to the transcoding of designated data sets generated by technical apparatuses. According to the above reflections about data as never being neutral or “raw” but always contextualized and negotiated, the non-audible data that one would intend to translate into meaningful information will in most cases, then, be selected from a basic pre-existing idea about the origin of those data, related to the use of certain technologies in specific contexts. Put in another way, we might say that data sonification presupposes an acquaintance with – and judgment of – the data or data sets used in order for the sonification process to be able to bring forth the patterns, significant events, etc. that could eventually lead to meaningful information. Information which could then subsequently enhance and converge (or falsify) rudimentary assumptions, through exploratory use of technology, into purposeful knowledge. This predicament can be said, again, to resonate with the concept of affordance mentioned earlier as something which designates the possibilities for meaningful engagement with and within an encompassing situation. In line with this, Have and Stougaard have recently re-introduced a theory of affordance from a sound studies perspective in their discussion of audio books (Have and Stougaard 2015). Here the authors point out how “affordance comprises a quality that belongs neither to the medium nor to the user, but emerges in interaction between the two” (Have and Stougaard 2015: 88). Furthermore, they argue that the term affordance entails “the possibilities not yet actualized by media users and producers” (Have and Stougaard 2015: 88). Both of these aspects are present when we consider one of the key technical issues at play in sonification processes: the actual coding or transcoding procedures involved.
“Transcoding” is also the term suggested by Sterne and Akiyama as central to describing how “a wide range of possibilities exists for both the manipulation of data and the interface itself” (Sterne and Akiyama: 549). The term is often referred to via Lev Manovich’s discussion of new media technologies and Sterne and Akiyama make use of it here, among other reasons, to put forward some basic considerations about the indexical representation. In a broader sense, transcoding as a concept also is closely related to the above discussion about the contextualization of data. Manovich has advanced the concept in order to argue for a relationship between cultural and computer technological domains, focusing on the possibilities of software programming for handling sound, images, and text in an extremely versatile manner across traditional medial borders (Manovich 2001: 45). Following this, doing more scrutinizing analysis of the level of software programmability used in sonification practices could potentially reveal some of the underlying principles for our interpretative engagement with abstract data sets. However, Adrian McKenzie has argued that we should be skeptical toward the concept of transcoding and claims that it might be an approach that “raise[s] more problems than it solves, since its terms are themselves derived from software design” (McKenzie 2006: 4). By this critique McKenzie means to point out how software design is equally pre-determined in how it renders data according to specific cultural presumptions and institutional circumstances.
Thus, when considering critically the affordances of sonification, one becomes aware of the complex mesh of interrelations that emerges – thinking again about our initial example – between the “objective” data produced through specific technological processes at CERN, the potential information assumed to be contained within those data, and the “official” knowledge generated through interpretative and re-mediating procedures, both individually and collectively by the particle physicists whose personal suppositions of what an electron might sound like can end up shaping the actual expression and aesthetics of the particular sonification of LHC-data (cf. the initial quote above). Here the words of Steven Connor may also resonate strongly in that he suggests how “[s]onification does not sound very far away from personification,” a statement put forth by Connor while explaining his highly skeptical attitude toward a current so-called “conversion hysteria” (Connor 2013: 137). Connor also discusses what he terms “the stubbornly genitive case of sound, its inseparability from the idea of an originating circumstance” (Connor 2013: 151) and identifies a main problem within current notions of sonification, namely a kind of “mysticism of the primal, a set of beliefs that sees translation into sound as a kind of making manifest of the latent truths, of a set of absolute but hidden primal conditions.” “The act of sonification,” he continues, ”is understood as a kind of re-enchantment of the world […] (Connor 2013: 147).
Nonetheless, as highly skeptical of the affordances of sonification Steven Connor appears to be here, we can take this as a cue for moving into the latter part and toward the conclusion of this article. Not least if Connor’s critique is seen in conjunction with how Erlmann has proposed that we revise our understanding of hearing and listening by looking for “a deeper interpenetration of the biological and the cultural” and “complicat[ing] simple tripartite sender-medium-receiver models” (Erlmann 2010: 17).[5] This denouncing of a classical model of communication suggests, among other things, that the sheer physical and material qualities of the media used for transferring data into acoustics should be considered further, even though this analytical task would not immediately seem to be occupied with clarifying the information and knowledge production processes involved in sonification.
[5] I will not delve further into the distinction between listening and hearing, which has been rehearsed in several recent writings. Suffice it here to refer to Erlmann’s own elaborate explanation of the “listening-hearing divide” of which a central component is the notion of listening as a “psychological act, distinguished from hearing, which is a purely physiological phenomenon” (Erlmann 2010: 21).
From re-enchantment to re-enactment
If the shortcomings of the affordances of data sonification is partly due to an underdeveloped awareness of what it is that we actually explore when listening to transcoded data, then we might learn something from practitioners working with conversions of sound from a slightly different approach than that of more traditionally trained scientists and software engineers, for instance. This includes, as well, certain artistic practices that would not consider it as their ideal to turn selected data sets into Mozartian pastiches, but would rather seek to circumvent such aesthetic conventions while being highly sentient of the situatedness and contextualization of the data they employ.
In their introduction to Das geschulte Ohr. Eine Kulturgeschichte der Sonifikation, an anthology addressing sonification from a variety of cultural theoretical and partly philosophical perspectives, Schoon and Volmar point out some of the basic questions relating to the affordances of sonification, similar to those mentioned above (Schoon and Volmar 2012: 9–26). As an introductory example they refer to the British Artist Graham Harwood’s art installation “Lungs,” which is described by Harwood himself as “a memorial to the 4,500 slave laborers who worked in Hall A of the former Deutschen Waffen und Munitionsfabriken A.G. during the Second World War.” One of the central aesthetic components of the installation is the continuous playback of the sound of breath, referring paratextually to statistically calculated data concerning the slave laborers, thus connecting the installation’s location in ZKM at Karlsruhe (the original site of the factories) to the history and destiny of those people. Schoon and Volmar use this to point toward a difference between sonification understood as either the expression of a conscious, individual intention (of the artist) or as a way of more generally providing access to and rendering meaningful a specific set of data. They refer here also to the artist’s own description of the aim to bridge: “the gap between the perception of data and social experience” and to “take computer records of local events or communities that have been reduced or demeaned to the status of information and to allow the people to re-experience and/or recover their own value.” Here we may say that the discussion about the tightness of data as seen from a semiotic point of view – as either enhancing an artistic and aesthetic quality/freedom or as underpinning the closeness to an original data set – becomes irrelevant. In Harwood’s work the data set referred to is heavily contextualized in that the artist creates an explicit narrative necessary for establishing the connection between perceived sound and original data. At the same time, the installation is directly, i.e. indexically, connected with the historic site, whereby the event, as described above, is subtly re-enacted in a current social (art institutional) context. The data is, in other words, embodied or given “body” in the form of the original architectural surroundings, serving now as a setting for the visiting public.
Harwood has elsewhere continued to extend this strategy of using data and datasets as a point of departure for a critical artistic practice, during more recent years as part of the artist duo YoHa (with Matsuko Yokokoji). For YoHa a returning core question is, thus, how to process abstract data qua information into meaningful knowledge, realized through a variety of unexpected and alternative media.[6] Not least by reversing how we conceive of the relationship between original data and the choice of formal, aesthetic reworking of those data. As part of this critically-engaged practice, YoHa have, for instance, conducted a long-term investigation named “Database Documentary” in which they sought “to understand how databases change our conduct.” This was done via research located within health databases, subsequently used to intelligently ask a number of principal questions, such as “[H]ow [does] the creation and implementation of database management systems (DBMS) affect the conduct of the authors and commissioners of such systems?” or “How can people who are affected and represented by the abstract models of health embodied in DBMS critically engage with their processes?” Even though no specific sonification aspects have been connected with this latter project, the theme and quotes here are representative for YoHa’s overall strategy of creating unexpected conversions of data sets, going far beyond traditional mapping models typically associated with transcoding procedures, as described above.
The artistic practice of YoHa, as well as the remarks that Schoon and Volmar present in their introductory discussion, also tap into a general discourse related to distinctions between data, information, and knowledge. Or, rather, the processes through which data becomes information that might be assimilated as meaningful knowledge that can, once again, be re-employed in order to arrive at a better understanding of the original data. Volmar and Schoon term this “ein auditives Wissen,” i.e. “auditory knowledge,” which can be seen as emerging on a continuum between interpretation and epistemology (Schoon and Volmar: 11). This brings us back again to the two central modes of sonification suggested above, namely an exploratory as well as a communicative mode of engaging with data. Focusing here on the first mode, i.e. data exploration, this may be considered an activity that circumvents a priori categories and conventional aesthetic formalizations, thereby leaving much space open for interpretation. The immediate task that one would have to carry out via this exploratory path involves different steps of selecting, re-listening, and re-selecting specifically poignant ranges of the sonified data. As a final example leading to the conclusion of this article, we turn our attention toward an artist working with data monitoring and data sets, while simultaneously attempting to avoid the type of conventional aesthetic formalization discussed previously (i.e. the classic musical orchestration as the end result of a data-processing procedure).
[6] For further analysis of YoHa’s works and installations related to data discourses, see Bjørnsten and Løhmann (2015).
Sounding beyond (common) sonification
Working collaboratively, Martin Howse and Shintaro Myazaki have developed a type of measuring gadget which is able to monitor electromagnetic emissions produced by electronic devices. In a smaller co-authored article describing the use of these so-called “Detektors,” Howse and Myazaki describe part of their project as dealing with an extension of “the spectrum of artistic concerns to embrace modern data space” (Howse and Myazaki 2010: 138). Furthermore, they characterize their effort as research that “attempts to bridge an impossible divide between the physical and the protocol (code)” (Howse and Myazaki 2010: 138). In practice, the Detektors allow the user to move through city spaces and similar settings while sonically registering the variety of electromagnetic fields and emissions, focusing on “WiFi, Bluetooth, GSM, UMTS, GPS and other transmission systems which are in the 100MHz–5GHz region of the spectrum” that are “transduced” into an audible output (Howse and Myazaki 2010: 138). One example of this is a recording entitled “Karl-Liebknecht Berlin” which is accompanied by the following caption: “A very short walk, dance along my favorite spectral zone Karl-Liebknecht-Strasse. HF recording with the Detektor and some varying, interesting signals.”[7]
Howse and Myazaki partly frame this practice by referring to the situationist concept of “dérive” while suggesting a “psychogeophysical” activity involving what Myazaki has termed “algorhythm” (Howse and Myazaki 2010: 138). According to the artists, this term comprises “the transversal thinking of algorithm with rhythmic, therefore to think the symbolico-calculational within the sonic, thus the physical” (Howse and Myazaki 2010: 137). Following these ideas, phrased in a blend of technical vocabulary and DIY-terminologies, Howse and Myazaki’s project as a whole hints at new ways of converting invisible and unheard data spaces into audible sounds. However, the tools for doing so are constructed so as to avoid traditional software interfaces and transcoding procedures that would otherwise potentially smooth out the signals registered, i.e. by rendering the signals, for instance, into musicalized or otherwise aesthetically-conventional forms. The Detektors-project has unfolded as a work in progress since its presentation in 2010. Howse has, prior to and following this particular collaboration, worked on a number of more or less related projects also involving signal and data transformation into both sound and images. Howse himself, however, has also rejected that what he is doing aligns with common ideas of sonification as such. He pointed this out in a recent interview, phrasing his critical approach toward sonification in the following way:
I would never refer to what I’m doing as ‘working with sense’ or any kind of sonification, because those are very directional processes. They do result in things that are visible and audible to the audience, but I try not to put a dividing line between a ‘box’ which will transform something into something else: this into this. Rather I want to extend its abilities – to have an electronic device which is ‘leaky’ and open to interferences. Something which is not a refined device and which can be operated by a body, by all that is close to it. (Quoted in Bjørnsten 2015)
If we recall again here the suggestion put forward by Erlmann, for thinking about sound and listening along the lines of “a deeper interpenetration of the biological and the cultural,” involving also a complication of “simple tripartite sender-medium-receiver models,” this seems to resonate with Howse’s intentions of interacting simultaneously with natural environments and coded technology. In this artistically-driven process, the end result is consequently not confined by specific notions of “sense” or “sensemaking” – the latter equating “sonification” if we follow Howse – but represents, rather, an alternative strategy of making data accessible through other types of interconnections between the technological apparatus, its output, and the user who operates and interprets it, thus fundamentally altering and expanding the affordances of the data-translating process. This would be a kind sonification practice, then, in which the regulating protocols and technologies are subsumed by the listening artist or user subject, but without reducing him to a mere pattern recognition machine.[8] Again, we can see how this lines up with an established notion of affordances as characterized by Johnson when he points out the limited range of the human sensory system for perceiving “sound frequencies” by referring to the way in which we “out of our bodily interaction with our environmental affordances […] take the meaning of things and events in certain specified ways, according to specific interactional patterns” (Johnson 2015: 30). Howse appears to object exactly to such easy “pattern-alignment” and even goes so far as to suggest that we leave behind any deliberate intentions of trying to make sense of specific data, as this kind of “measurement-work” amounts largely to a “fetishization of massive information” where one gets “tempted to find the most esoteric ways of dealing with it” – something which, according to Howse, is fundamentally not very interesting (Bjørnsten 2015).
[7] A number of similar recording examples can be found at the designated project website.
[8] This notion of the human subject, in a sonification-context, reduced to carrying out pattern recognition is also adressed by Schoon and Volmar, who refer to the human listener as Mustererkennungsmaschine from which follows a mode of listening that is emphatically goal-oriented (Schoon and Volmar 2012: 13).
Conclusion
Having included, as the last example in this article, the subtly critical proposal put forward by Howse against a common contemporary sonification practice (i.e. “measurement-work” and “information-fetishization”), this is obviously not to suggest that we give up on the potential productive outcomes of data sonification. However, it may remind us of the need to counterbalance the habit of, somewhat unconsciously, “smoothing out” or formalizing the end results of sonification processes, thus too radically reducing the complex relationship between data, translation, and knowledge generation outlined in the discussions above. However, we may here recall that sonification has emerged as a field and a practice for a reason in the first place, namely that our naturally-given auditory system allows us to perceive only part of the phenomena that make up the lifeworld that artists as well as scientists seek to uncover and understand through various methods. Sonification can, thus, basically be comprehended as a way of expanding our given affordances for interpreting, describing and engaging with the world. Seen in this perspective, the sonification work carried out both by the particle physicists at CERN, by Howse and Myazaki, as well as by Graham Harwood is about expanding, challenging, or changing those affordances even further – a task which is, as we have shown, accomplished from quite different perspectives and through rather dissimilar aesthetic paradigms.
Especially the artists mentioned here can be said to propose a truly explorative methodology for engaging with data beyond standardized software interfaces and algorithms in ways that we may use as an inspiration for thinking more imaginatively about both artistically- and academically-situated research work based on sonification processes. If we accept this type of critical and essentially avant-gardist approach toward data sonification, it can possibly prove more relevant than ever to open up the field toward such practices that go beyond mainstream aesthetic schemes in dealing with abstract data material.[9] Currently this seems to be an actual tendency which we find, also in the context of the motivation for this article. Since 2011 CERN has been hosting a number of experimental artists based on proposals for official calls via the program “Collide@CERN,” who have then been given a residency in order to explore CERN according to their artistic agendas. The outcome of these residencies has, so far, not been limited to sound-based works, but includes visual and combined audiovisual forms. While this article has not focused on that particular initiative, the practice and work by some of the associated artists, such as Ryoji Ikeda and Semiconductor, will be investigated further through a current research project called “Making Sense of Data - understanding digital reality through contemporary artistic practices of visualization and sonification.” One of the main goals of the project is, thus, to survey and bring together both creative avant-garde working procedures and more traditionally academic-scientific methods. This would obviously also involve advancing a more nuanced conception of sonification affordances, as we have proposed in the above.
[9] By referring to “schemes,” we partly invoke here the work of George Lakoff and Mark Johnson who use the term ”image schemas” for explaining “recurring patterns of interaction” (Johnson 2015: 30).
References
Barrass, Stephen and Paul Vickers (2011). “Sonification Design and Aesthetics,” In Thomas Hermann, Andy Hunt and John G. Neuhoff (eds.), The Sonification Handbook, (pp. 145–72). Berlin: Logos Publishing House.
Bjørnsten, Thomas (2015). “Art beyond the critical human perspective – an interview with Martin Howse.”
Bjørnsten, Thomas and Jan Løhmann (2015). “Renegotiating data ecologies through trees, soil, and pigs’ lungs.” Spheres: Journal for Digital Cultures (forthcoming).
Connor, Steven (2013). ”Photophonics.” SoundEffects 3/1–2: 137–52.
Geiselberger, Heinrich and Tobias Moorstedt (eds.) (2013). Big Data: Das neue Versprechen der Allwissenheit. Berlin: Suhrkamp Verlag.
Gitelman, Lisa (ed.) (2013). Raw Data is An Oxymoron. Cambridge, MA: MIT Press.
Erlmann, Veit (2010). Reason and Resonance: A History of Modern Aurality. New York: Zone Books.
Have, Iben and Birgitte Stougaard (2015). Digital Audiobooks: New Media, Users, and Experiences. London: Routledge.
Hermann, Thomas, Andy Hunt and John G. Neuhoff (eds.) (2011). The Sonification Handbook. Berlin: Logos Publishing House.
Howse, Martin and Shintaro Miyazaki (2010). “Detektors: Rhythms of Electromagnetic Emissions, their Psychogeophysics and Micrological Auscultation.” In Proceedings of the 16th International Symposium on Electronic Art ISEA 2010 RUHR (pp. 136–138). Berlin: Revolver.
Johnson, Mark (2015). “The Aesthetics of Embodied Life.” In Alfonsina Scarinzi (ed.), Aesthetics and the Embodied Mind: Beyond Art Theory and the Cartesian Mind-Body Dichotomy (pp. 23–28). London: Springer.
Knapton, Sarah (2014). “The sound of science: Higgs boson data turned into music at CERN.” The Telegraph, October 1.
http://lhcsound.hep.ucl.ac.uk/
http://lhcsound.hep.ucl.ac.uk/page_sounds/Sounds.html
http://lhcsound.hep.ucl.ac.uk/page_sonification/Sonification.html
https://lhcsound.wordpress.com/sonification-of-higgs-di-photon-channel-2012-data/
Manovich, Lev (2002). The Language of New Media. Cambridge, MA: MIT Press.
McKenzie, Adrian (2006). Cutting Code: Software and Sociality. New York: Peter Lang.
Prince, Andrew (2011). “Particle Pings: Sounds Of The Large Hadron Collider.”
Schoon, Andi and Axel Volmar (eds.) (2012). Das geschulte Ohr: Eine Kulturgeschichte der Sonifikation. Bielefeld: Transcript.
Sterne, Jonathan and Mitchell Akiyama (2012). ”The Recording that Never Wanted to Be Heard and Other Stories of Sonification.” In Trevor Pinch and Karin Bijsterveld (eds.), The Oxford Handbook of Sound Studies (pp. 544–560). New York: Oxford University Press.
Supper, Alexandra (2012). “The Search for the ‘Killer Application’: Drawing the Boundaries around the Sonification of Scientific Data.” In Trevor Pinch and Karin Bijsterveld (eds.), The Oxford Handbook of Sound Studies (pp. 249–270). New York: Oxford University Press.