ON MUSICAL SPACES


Space is existential,

existence is spatial
[Merleau-Ponty, 1966 (1945)]


Sounds are perceived not only as characteristics of a sound source and thus as indicators

of their location, but they can become more or less self-sufficient objects with real physical properties.

The result is a listening space, the dimensions of which are in a confusing relation to the real space.

[de la Motte-Haber, 1991]

 

Taking possession of space is the first gesture of living things, of men, and of animals.

The first proof of existence is the act of occupying space.

[Le Corbusier, 1946]

 

 

 

 

 

 

I. Traces in (Computer-)Musical Application

Space is another term that plays a central role in the compositional work with sculptural sound phenomena. Again, the research process begins with a search for clues. Along with the introduction of the multi-channel technique and the first performances with complex loudspeaker arrangements, e.g. at the 1958 World's Fair, the subject was increasingly discussed in the field of composition.

[...]In the context of analogue tape composition, the composer tries [...] to shape the direction and the movement of the sounds in space and to develop it as a new dimension for the musical experience. [...] We notice more and more that all musical ideas are becoming increasingly spatial.  [Stockhausen, 1958, 153]

The term space is not always used consistently in computer music. [...] the notion of "space" has become the paradigm of electroacoustic music. However, the terminological framework of electroacoustic "spaces" is not clearly defined  [Harley, 1994].

There is most of all a consensus that space plays a central role in acousmatic music. Acousmatic music is the only sonic medium that concentrates on space and spatial experience as aesthetically central. [Smalley, 2007, 35]

Chion distinguishes between internal space, which is created in the composition, and external space that arises during the performance [Chion, 1988]. Risset describes the fragility of an illusory space produced by the composer and the real space of the performance in which the illusory space is presented [Risset, 1998]. Smalley counts over 20 different spaces in electroacoustic music, for example the composed space, the listening space, and the superimposed space [Smalley, 2007, 35ff]. Emmerson speaks of nested spaces [Emmerson, 1989] and space frames [Emmerson, 2015], and Roads distinguishes between virtual and real spaces [Roads, 2015, 261].

In the standard work Was ist elektronische Musik? by Werner Kaegi, spatialization by means of multi-channel loudspeaker arrangements is explained as a spatial arrangement of sound sources, but space itself is not addressed [Kaegi, 1967, 23].

In the last chapter of his 1998 book, Elektroakustische Musik & Computermusik [Supper, 1998, 121], Martin Supper deals with space. It is interesting to note here the quotation by Rudolf Carnap, which proceeds the chapter titled "Music and Space": In my dissertation "der Raum" (1921), I tried to show that the contradictory theories about the nature of space represented by mathematicians, philosophers, and physicists stemmed from the fact that the authors talked about completely different things but were using the same term, "space."[1]  In the footnote, Supper makes it clear that he concentrates mainly on architectural space in his explanations. This reduction implies that there are various uses of the term space, the consideration of which would, of course, exceed the scope of the chapter space. The following subtitles in Supper's book are then: Space as an Instrument, Virtual and Simulated Space, and Movement of Sound in Space.

In the field of musicology, space has historically been mainly attributed to pitch ratios. Gunnar Hindrichs speaks of musical space as an order of the constellation of musical sounds, which in their network of relationships make the identity of each one in the music possible and creates spatial comparative variables. On the one hand, this results in the fact that music does not only sound in space, but also creates its own kind of space in itself [Hindrichs, 2014, 161]. In the book Musik und Raum, published in 2005, Karen Gloy describes space as a basic phenomenon, which is composed from the superposition of various space types. She describes tuned space, action space, perceptual space, mathematical space and metaphorical space as possible basic types. [Gloy, 2005, 11].

Due to the fact that the concert space could be designed and played in differently by the positioning of loudspeakers, other spatial concepts were considered. Pierre Boulez speaks in favor of exploring more flexible spatial concepts that can change over the course of a piece:

It seems to me that one of the most urgent objectives of present-day musical thought is the conception and realization of a relativity of the various musical spaces in use. […][T]he time has obviously come to explore variable spaces, spaces of mobile definition capable of evolving (by mutation or progressive transformation) during the course of a work. [Boulez, 1971 (1963)]

 

 

 

II. Spatial Turn

The versatile use of the term is not surprising, considering the fact that, parallel to the development of music, the concept of space has been given interdisciplinarily new and historically noteworthy consideration, so that one speaks of the so-called Spatial Turn.

An initial, very simplified definition of the spatial turn should take into account that something astonishing happened in the last decade of the 20th century, such as what might be regarded in the 21st century as one of the most significant intellectual and political events of the late 20th century. Some individuals, among them scientists, began to think about space and spatial elements of human life seriously and critically, similar in a way to what has long been thought about time and the historicity of human life. Over the last 150 years, we have become accustomed to seeing the world through a historical lens rather than a space-based one. But what happened now happened on an interdisciplinary, transdisciplinary, and, if so, a pandisciplinary level. In the late 20th century, space-related thinking broke out of the traditional disciplines - geography, architecture, urban development, regional sciences, and sometimes sociology and art history. The sudden width of the spatial turn is remarkable beyond all measure. [Soja, 1989, 243]

In view of the historically unique changes in dealing with and thinking about space, it would be desirable if the acoustic arts also had a different spatial concept. One which, on one hand, is less metaphorical and therefore more conceptually generalizable, and on the other hand, not only "Cartesian", and rather therefore more interdisciplinarily oriented.

 

III. Three Space Theories for the Constitution of a SPS

But which type of space, definition, or concept can make a practical contribution to composing with sculptural sound phenomena? Since computer music has always been interdisciplinary, it is obvious to include extra musical considerations of space in the composition of different spaces. If one wants to investigate plastic sound phenomena with a loudspeaker system conceived for mobile use and to be used in different situations, one must develop an understanding of room acoustics as well as the different spatial ideas and their conditions for the sound resonating therein.

Sound and space converse by multiplying and expanding the point of attention, or the source of a sound: the materiality of a given room shapes the contours of sound, moulding it according to reflection and absorption, reverberation and diffraction. [LaBelle, 2006, ix, xi]

 [At the same time] sound makes a given space appear beyond any total viewpoint: in echoing throughout the room, my clapping describes the space from a multiplicity of perspectives and locations, for the room is here, between my palms, and there, along the trajectory of sound … What we hear in this clapping is more than a single sound and its source, but rather a spatial event. [ibid., x ]

Here, models of psychoacoustics and room acoustics are superimposed with philosophical and sociological models of how we construct space in our perception. While electroacoustic music and with it the development of audio software and hardware in the last decades mainly follow the idea of Euclidean space, other spatial ideas are oriented away from this or understand physical space as a condition but more in the sense of a foil with other spaces unfolding in front of it.

The following three representatives introduced can provide a direction for further reflections on the compositional and performance practice of plastic sound objects and their exploration within the scope of this work: Henri Lefebvre, Michel de Certeau and Martina Löw.

Henri Lefebvre [Lefebvre, 1974] and Michel de Certeau [Certeau, 1980] developed their theories of space in the 1970s, which are well received today in architecture, geography and the social sciences. The sociologist Martina Löw developed her approach in her dissertation on Raumsoziologie in 2000 [Löw, 2001]. The starting point for all three approaches is a relational understanding of space.

The general theory of space distinguishes between absolutist and relativistic concepts of space. In the absolutistic concept space exists independently of matter. Movable bodies and things are in a space that remains unmoving itself. The space exists continuously, for itself, and forms an equal, homogeneous basis for action for all. This idea of a container space has been replaced in science with the development of the relativity theory of relativistic spatial concepts. However, it still characterizes the everyday understanding of space and is usually an indispensable condition for psycho-acoustical studies and ingenious scientific research in the area of spatial-audio. In the relativistic concept of space, the space does not exist independently of the bodies. Instead, space is understood as a relation, as a relational structure between bodies. The bodies, whose arrangements give rise to one another, are in constant motion. Thus the space itself is no longer static, but becomes processual and constantly changes over the course of time. Since the arrangement of bodies cannot be thought of independent from the observer’s reference system, space is not absolute, rather always exists relative to the consciousness of the observer.

 

 

III.1. Three Dimensions of Space Production by Henri Lefebvre

Lefebvre sees space as a communally produced, social product. He distinguishes three levels of the production of space, each connected dialectically.

Lefebvre describes the first dimension of spatial production as spatial practice [Lefebvre, 1974, 335]. The starting point is the material dimension of the space. The material elements and objects, which form the space are perceived sensually and linked to a spatial order of simultaneousness. Spatial practice refers to everything that humans do in and with space: what things and objects they build, how they move these and themselves in space, how they deal with topography, and how and why they use the space. To this end, in musical practice it would include the arrangement of stage and audience, the design of the stage, the installation or suspension of reflectors and the seating or design of passages in the context of installations, as well as the arrangement of musicians and loudspeakers on the basis of various principles of organization, including room acoustics and concepts such as audio vision and visu-audition. I also include the composition, arrangement and movement of plastic sound objects. All these are elements of spatial practice.

Lefebvre's second dimension of space production is conceived space [ibid., 336]. The connection of individually perceived, material elements to a space presupposes mental effort, having an idea of space. These representations of the space include linguistic descriptions, pictoral representations, maps, plans, and scientific definitions. Representations and definitions of space are based on social conventions and are negotiated discursively. Each in-depth investigation in the field of space-sound composition must deal with the definitions of space in its own fields accordingly and the adjacent fields, and, if necessary, offer terminology. Since, as shown above, these conceptions differ widely in the field of music and musicology, and space is not adequately defined, other concepts of spatial attributes must be used to classify the artistic results. These originate from the scholastic writings on sculpture, space theory and engineering sciences. It is also necessary to consider how the laboratory situation can be visualized and later evaluated graphically in the context of the test design and the presentation of results.

The third level of the production of space is the lived space, the level of meaning of the symbolic content [ibid.]. This level is decisive for the experiencing of spaces. Spaces are occupied by symbolic content; they can describe something outside themselves. The symbolic meaning of spaces expresses itself, for example, in the architecture of sacral spaces or spaces representative of political power. They show the importance of cultural landscapes in how a country sees itself, as well as places that are important for the individual biography. The construction of concert halls points to a certain superior or bourgeois attitude toward art, or to a certain musical tradition, meant to sustain and ensure its livelihood.

 

 

The choice of space and the spatial design of concert venues reflect the ups and downs of the autonomy of this art. The location of where music is practiced is often legitimized in a way foreign to music. [Kirchberg, 2009, 156]

Even if the concert hall is described as the "place of realization of autonomous music", such a place is always a "social institution". [Heister 1996: 42, 44] 

Symptomatic of this is the inscription on the Neuen Gewandhaus "res severa verum gaudium" - the serious thing is the true joy. [Kirchberg, 2009, 157]

 

 

This structure of reference is changing through the described changes in the conception of music, whether by the construction of loudspeaker clusters or spherical constructions or the invention of mobile loudspeaker orchestras (almost) without human performers, self-playing instruments and installative sound machines, or even the design of a new concert hall rebuilt and furnished in versatile and unconventional ways. The change of awareness in staging media art and theater has left traces in all musical disciplines, e.g. the use of video screens, light installations and illuminated (!) loudspeaker domes (ZKM). This leads to references to extrinsic coherences such as media technology, film and television, as well as politics, sciences, and other arts. In addition to this, concerts are increasingly being held at untypical places: foyers, old factories, empty department stores, basement vaults, shops - just to name a few examples. They form symbolic spaces through their original purpose. The symbolism is therefore also to be included in the reflections of the composition, if the place of the performance is to be regarded as decisive for the work, especially if it is conventionally visually stimulating. The audience's behavior will be guided by the expectation and the perspective of what is offered. If I play a loudspeaker concert in a gallery without any chairs, the acoustic characteristics combine with the dedication of the place differently than of the same concert in a chapel with benches.

Space arises therefore out of the interplay of these three poles. It is not to be understood as an arrangement of material objects and artefacts, but as the practical, mental, and symbolic establishment of relationships between these objects. Space is not dormant, immobile, or given, but a multi-layered fabric which is constantly produced and reproduced.

 

 

SPS

III. 2. Spaces and Places by Michel de Certeau

Spatial sound composition with the IKO incorporates the acoustic potential of places in the form of concert halls, entrance halls, gallery rooms, laboratories or shopping centers. Michel de Certeau's approach to practices in space (practiques de l’espace), in which he differentiates space and place one from another [Certeau, 1980, 217], provides a suitable framework based on everyday circumstances. His place and space concepts are embedded in his sociological theory of everyday life [ibid., 219]. In that sense, place is defined as its own, actually the original, which separates itself by definition from what it is not. Compared to such a stable constellation, space is a dynamic concept. Space emerges from place and indeed as an intervention, which makes the action theoretical framework of Certeau's argument plausible. Space is a result of activities that give it a direction, making it temporal. The use of directional vectors, which make the space function as an ambiguous unity of conflict programs and contractual agreements, shows clear proximity to formulations of the actor-network theory. In their texts, Michel Callon and Bruno Latour adopt multistep processes of the generation of consistency in networks, in the course of which different actors develop their interests and goals, change their course of action, assume plans of action and counterplans, and reintroduce, redefine or remove actors [cf. Schulz-Schaeffer, 2000, 187]. The conventions, negotiations, or, more generally speaking, the communicative exchange conducted with the aim of the establishment of situations is not only metaphorical for the composition of an acoustic spatialization of a place: acoustic conditions of the performance space, the IKO as a tool, reflectors, audience and audience seating are "actants" who program the appearance of spatial sounds and sound spaces as a result of their mutual influence. Certeaus' paradigm for the transformation of a place into space is the image of one walking, which dynamizes the geometric determination of a place [Certeau, 1980, 218]. The IKO also practices this transformation, in that sounds that react to specific existing locations become performers, which in their movement coin new vectors and thus create space.

 

 

III.3. Spacing and Synthesis by Martina Löw

Martina Löw also develops her theory of space from a sociological perspective. Her position serves as an example for the current "spatial turn" in the humanities and social sciences. Löw understands space as a relational arrangement of social goods [Löw, 2001, 158]: of material elements and human beings. Space is not given, but is produced by arranging elements (in relation to other arrangements), that is, by means of actions. The constitution of space is to be understood as a process. Löw distinguishes two different processes of spatial constitution: Spacing (furnishing) and Synthesis. These are comparable to spatial practice and the conception of space by Lefebvre. Spacing refers not only to the placement, erection, construction, or positioning of buildings, but also of moving goods. Through processes of perception, imagination and memory, the individual elements placed in the space are linked together and combined into spacings [Löw, 2001, 166]. Here, similarly to Lefebvre, the placing of plastic sound objects, their movement, arrangement, and separation are the practices of spacing, which constitute space. Composed sound can thus help to connect the different spatial components in the sense of furnishing [Brüstle, 2009, 115]. The so-called "concert" would arise when buildings, instruments, performers, loudspeakers, media technology and audience are arranged and placed in a specific, intended relationship to one another, and are linked together by the people who move in it and perceive it as space. Löw calls this linking process a synthesis. In everyday practice, the two processes cannot be separated. Building, constructing, and moving in space is not possible without the simultaneous linking of the surrounding elements. Conceptions of what a space is and can be (for example, a street or a place), determine what and where to build.

Spacing and synthesis are repetitive in everyday life. The arrangement of elements for a specific type of space follows one ordering principle. The elements that form the space “road” are always arranged and perceived according to a comparable pattern. Those processes of spatial constitution, which take place according to predetermined rules, which are socially and institutionally secured, form spatial structures. Space, understood as a relational arrangement, as a relationship between elements, has no material quality. It is, however, experienced as an object when the formation of relations, that is, the arrangement of the elements, is institutionalized, therefore always following the same rules.

In this respect, a study on plastic sound objects would have to look for a rule based on the perception, provided one expects a synthesis capacity from the listener: A stage and audience seating refer to a concert setting. Even though the installation and use of a loudspeaker system may be over 50 years after the Philips Pavilion and "Poème Eléctronique", They can count as a synthetic space. It becomes problematic when we work with sound objects whose descriptions and experiences diverge and even in expert circles are unequally dealt with. Thus the institutionalization is minimal and detached from other "shareholders,” such as the audience.

A compositional spatial synthesis can therefore only occur when the elements used are perceptible, placeable and lead to a description. This definition corresponds to the descriptions in the electroacoustic space-sound composition literature, e.g. Nystrøm's description of his distribution schemas [Nystrøm, 2013, 46]. But Emmerson's space frames [Emmerson, 2015, 13] are also able to perform these synthetic tasks. The spacing, placing, building, and setting up of elements are tied to a specific place. This specific, mostly geographically marked, concrete place makes the emergence of different spaces possible. It is only in the synthesis of these elements that Varèse’s zones of intensities or Bayles' region of influence [Bayle, 2007, 243] and Emmerson's area of interest [Emmerson, 1999, 138] can appear.

 

 

IV. The Concept of Space within the Framework of XXX

The space beyond the framework of the body may appear to be a priority to most people, but in fact it must be traversed to be properly grasped. The movement of the body, the movement of objects, the movement of other people - all of these contribute to an understanding of space through spatial actions and behaviours. [Kendall, 2010, 232]

I use a relational concept of space within this work. This is, on the one hand, tied to spatial practices but takes into account conceptual spatial concepts and the symbolic. Therefore space does not exist independently of the bodies. Instead, space is understood all the more as a relation, having the structure of the relationship between the bodies. I understand spatial concepts to mean the sound objects along with their spatial extent and placement by the composer who must be present during the compositional process in order to be able to compose the sculptural sound objects, the IKO and its placement, those who research the phenomena, the audience and the description of spatial phenomena as spatial concepts. Thus, the SPS is a necessary condition for the emergence and perception of sculptural sound phenomena, which in turn first form this space. There is therefore a relationship of mutual dependency. This applies to sculpture’s classic body-space constellation:

The importance of the all-round totality of a full three-dimensional piece is that it should “take possession” of its space by one means or another. [Rawson, 1997, 66]

This does not mean simply occupying space; all objects do that. It means developing its shapes and their implications so as to build in readable connections between the piece and the spatial environment. [ibid. 67]

As well as for sculptural sound phenomena in computer music:

We must note here in passing the reciprocal tie, which unites the space inhabited by the senses and the spatial sense, so that to understand what a thing is, is to work out the space suggested by the dimensions of that thing; and in the same way, listening to it involves a space and time for the object listened to. [Bayle, 2007, 242]

In this respect, I contradict the often-accepted view in acousmatic music, that it is an art dealing with "music in space" or "sound in space."[2] Instead, against the background of Smalley's debated spectromorphological space definitions and Nystrøm's elaborated topologies of spatial textures, as well as the depicted spatial practices from Lefebvre and Certeau to Löw, I plea here for the fundamental assumption that electroacoustic space-sound composition is "music as space" in the sense of a space-forming art in which, in particular, the sculptural sound phenomena are capable of constituting space.



[1] Rudolf Carnap, Mein Weg in die Philosophie, Stuttgart: Reclam, 1993, p. 18 f, cited in Supper 1998.

[2] Cf. as an example: next generation symposium report 2007

ON IMMERSION

Surrounded by Immersion - Means of Post-Democratic Warfare

(gksh, excerpts from: The Ultrablack of Music, mille plateaux, Frankfurt 2020)

 

 

Here, perhaps the frightful expression "consumption of music" really does apply after all.

For perhaps this continuous tinkle, regardless of whether anyone wants to hear it or not,

whether anyone can take it in, whether anyone can use it, will lead to a state where

all music has been consumed, …

(Arnold Schoenberg, Style and Idea)

 

It is not a case of worrying or hoping for the best, but finding new weapons.

(Gille Deleuze, The Societies of Control)

 

[…] immersion is not achieved through assault; it’s achieved by inducing surrender.

(Frank Rose, strategy + business magazine)

 

 

 

// --

Within the past 10 years, immersion has become a frequently used term in concert venues and studios with multichannel-loudspeaker arrays, in the context of audio-visual caves, VR, AR and fine arts. Manufacturers of loudspeaker systems as well as the gaming industry are using the term as a feature that heralds a new step in “multi-media” experiences, and academia is claiming a kind of expertise in this field based on years of scientific experimentation and avant-garde practice[1]. At the same time, exhibition halls thematise immersion in VR as a socio-political issue of the present[2] and contemporary club culture is making a shift towards immersive, new worlds of experiences ranging from the most subtle and nuanced to the most intensely visceral, from healing and meditative to thought provoking and radical[3]. These days, protagonists of Pop music, Ambient and experimental forms of Club and Rock music are idolised by loudspeaker companies and produce, within very short rehearsal times, “immersive experiences” for festivals and museums. However, also global marketing for all kinds of consumer products has understood the modes of action and thoroughly researched the intrinsic qualities of the field to create a cohesive and all-encompassing experience through Immersion Branding.  Even data analysis recently entered Immersion Analytics[4] with the help of VR. Immersion is without doubt a subject of debate and current relevance. But what can we expect from this terminology and its applications when it comes to an extended artistic articulation as well as advanced production, and what are its implications for the shared perceptive situation of artists, engineers and audiences in the Now? Within recent years, the use of spatial audio (e.g. Ambisonics) has come into the focus of game design, online platforms such as YouTube, and companies like Google and Facebook, as well as consumer electronics, event locations, and architecture. Quite likely in the near future artificial auditory environments will be part of everyday life, and for a great number of people, a part of their reality. The emerging question is, who creates these environments and with which intentions, and how can music, sound art, and design contribute with their own strategies, to such a reality?

 

 

// The Contemporary -  Production of Space

Today, we are witnessing an unprecedented awareness of space, be it in philosophy, art, geography, and other such as academic discourses. However, after ages of discussion about time and its descriptions, what is behind all this? Why space, and why today? Since organising, redistributing, promoting, annexing, and defending territories are basic exercises of political as well as of economical and military decisions, to enthrone space as the new all-round theory means no less than to refer to the most traditional power dynamics we can imagine. This may be no surprise in view of strong effects of de-territorialisation as one main feature of the world’s status quo. Now, the awareness of space seems to have affected all kinds of systems, including academic discourses and art, to subordinate their targets under a new paradigm within a wide range of empirical, deductive, discursive, historical, scientific and intuitive methods. Spatialisation, the synthesis of spaces and spatial properties of sounds for a listener, is a growing field of interest for researchers, sound engineers, composers, and audiophiles. Due to broad and diverse viewpoints and requirements, the understanding and application of spatial sound is developing in various ways. How do we think spatialized art positions itself in terms of power dynamics? It is still unclear what this initial and explicit way of spatialized music means in terms of perception, composition, aesthetics, engineering and culture. And how its existence affects the history of ideas around technology. 

 

 

// Virtual Reality = Total Control

The expression ‘‘virtual reality’’ is a paradox, a contradiction in terms, and it describes a space of possibility or impossibility formed by illusionary addresses to the senses.[5]

 

Since virtual acoustics, as one of the technological conditions for current and future spatialisations, is part of a huge, utopian vision, a reproduction of what some might call “reality” we have to re-consider: What is it about the project of virtual reality that exists much longer than the period we name digitization?  The long list of end users and/or operators of immersive virtual reality products presents military equipment, modelling in all kind of engineering procedures, medical technology, educational technology, technology of rehabilitation and assistance, cybersex and, of course, consumer electronics. After having left its mark for centuries, the idea of simulating natural processes artificially was named virtual reality by Damien Broderick in The Judas Mandala in the year 1982. It seems that the four features Broderick listed, have covered all epochs: immersion, plausibility, interactivity, high fidelity. However, “virtual reality” never meant “the other world“, or “the new world“, but the perfectly controllable reality. Consequently, as artist we have to ask: What is the final use of perfect simulations? What is really its important and lasting contribution, and to whom or to what?

 

 

// Absorption – Being there – Being Surrounded – Being a Campaign

Although the term immersion is used widely within music journals and game studies literature, there seems to be little consensus on what the term precisely means. It is often used in its non-media specific context of “deep absorption”, but that ignores how the particular characteristics of the medium in question influence audience’s engagements with it. This equating of immersion with general absorption overlooks the important fact that the term was applied to digital games and virtual environments to identify a specific type of experiential phenomenon. However, experiencing immersion as a form of deep involvement fails to differentiate the experience of being absorbed in solving a crossword puzzle from the sense of inhabiting a compelling virtual environment. Virtual reality technology carries with it the promise of making similar experiences an accessible part of everyday life, the culmination of a long history in the creation of representational spaces that attempted to give viewers and listeners the sensation that they were actually “inside” them. According to Grau[6] this tendency to create hermetically closed-off image spaces of illusion has a rich history that can be traced back to Roman villa images in 60 B.C. This notion of creating an all encompassing media experience was also a concern of Bazin, whose 1946 influential essay, The Myth of Total Cinema, argues that the ultimate goal for cinema and all techniques of mechanical reproduction was the creation of […] a total and complete representation of reality… a perfect illusion of the outside world in sound, colour and relief. [7]

But as the representational power of graphics and sound technically expanded, game and 3D audio companies adopted “immersive” as a promotional adjective to market their games and multichannel sound systems. This strategy was initially employed almost exclusively to promote (photo-)realistic graphics or sound re-production, and although this is still often the case, it is nowadays also used to market the more prominent or innovative features of the relevant software, like size and scope of the game world, computer controlled characters’ artificial intelligence, engaging storyline, weather effects, dramatic sound effects, open-endedness, or the realism of its physics engine as well as the fact that loudspeaker arrays are physically surrounding the listener. The common thread running through these conceptions of immersion is their ability to generate captivating experiences often with a close relation to the mimetic qualities they share with the chosen genre. [8]

The development of advanced loudspeaker environments is most frequently directed by technological concerns, a focus upon reproduction of the physical signal, and less often towards the aesthetic or perceptual characteristics and the creative potentials that are inherent to these systems. This points to a conception of media technologies that ignores not only the key role that agency plays in creating presence, but also the interpretative role of the participant. So called high fidelity systems are an important part of enhancing the intensity of the experience but in themselves cannot create a sense of immersion!

One has to point out that even though immersion is enabled by technology, it is ultimately a personal experience of the user/listener and cannot be reduced to the characteristics of the technology that enable it and the expensive venues that are part of a so called creative industry, which is first of all a marketing machine’s environment.

 

 

// State of the Art or State of Mind?

[…] computer-managed signal processing offers unprecedented possibilities in the control of sound fields, and the promise of three-dimensional music is on the horizon[9].

 

The technical evolvement has been enormous over the past 30 years when it comes to sound field reproduction and also the creation of fictive spatial constellations. For example, how different reverb-qualities within one and the same pop song or ambient track became a part of the artistic sonic vocabulary. But the basic idea is not only to take a common piece of music and arrange it - spatially, with the listener surrounded by the musical elements (flying snares, circling sirens?) and many loudspeakers. By the use of loudspeaker arrays artists can produce spaces and sculpture-like spatial phenomena never-before perceived outside the technical setup, as such. We can determine that with today’s technologies, engineers’ and artists’ shared utopias from the past came close, and we are only now experiencing the beginning of possibilities.

Thus, historically we are now inside Edgard Varèse’s utopia from 1936 (!), when he wrote[10]:

 

When new instruments will allow me to write music as I conceive it, the movement of sound-masses, of shining planes, will be clearly perceived in my work, […]. Certain transmutations taking place on certain planes will seem to be projected onto other planes, [...] We have actually three dimensions in music: horizontal, vertical, and dynamic swelling or decreasing. I shall add a fourth, sound projection ... [the sense] of a journey into space. […] Today, with the technical means that exist and are easily adaptable, the differentiation of the various masses and different planes as well as these beams of sound could be made discernible to the listener by means of certain acoustical arrangements ... [permitting] the delimitation of what I call ‘zones of intensities’.

 

With so called “immersive sound systems” we now can enter these zones of intensities. We can experience three-dimensional sound planes and masses. The materiality and the spatiality of sound have been changed dramatically. And instead of the convention of the listener sitting passively facing a sound stage, we are conceiving music that a listener can get inside, even explore, being an active participant rather than recipient. However, in addition to physical constraints, there are the conceptual ones, inherited from previous situations.

A paradigm shift is under way; as technological constraints are rolled back, so must conceptual constraints be re-evaluated. Some of these are concerned with what spatiality actually is. Although we think of three-dimensional Euclidean space, it is by no means clear that this is anything other than a conceptual latecomer (however useful).[11]

 

These new possibilities opened pathways to a terra incognita in a time when every voxel of the world seemed to be charted, scanned and categorised. However, the substances of these spaces are multi-layered and need different knowledge compared to former ideas of navigation, composition and research.

As Kendall and Ardila implied in 2008[12], […] we do not want to simply copy the real world; we want to build on it. But, we do want to use everything we can of what we know about human spatial perception, how it works in different contexts, why it works, and so on.

 

 

// No Sky – No Land – Doors of Perception?

But how do we critically engage with artificial environments that are made as products of mass consumption and as means of control, conceived with the ideology of isolating individuals physically and mentally to become highly efficient consumers, instead of being citizens of a society that fosters and benefits from the arts for centuries?

After all, […] artist’s work is originally engaged in the question of the sensibility of the other.[13]

 

The early pioneers of electro-acoustic music pushed the frontiers of spatial audio and achieved remarkable successes in the artistic use of space. Varèse, Stockhausen, Schaeffer and Poullin, Bayle with the Acousmonium, Chowning (Turenas!) and onwards - spatial audio has been an expanding area of artistic expression. On the other hand, the great advantages in computer and audio technology that we enjoy today have not necessarily led to greater advances in spatial audio. What is a spatio-sonic utopia of 2019? Quite possibly, pushing back the frontiers of spatial audio today depends more on understanding spatial perception and cognition than on raw computing power and tools available. In everyday life, every person is able to navigate around in a spatial world, to talk about space and even to imagine unknown spaces. We can say that spatial thinking is one of our most deeply embedded cognitive capacities. But the ease with which we think about space is possibly a miscue as to how easily spatial ideas can be translated into spatial audio, which has its own unique capacities, intrinsic nature, and inherent limitations. The instrumentality of the tools, if not used for reproductive purposes, stays unclear and the artistic knowledge base is very thin. Not every spatial idea can be reverse engineered into sound. Clearly, our expectations about spatial audio should be in alignment with the fundamental capacities of spatial hearing. Traditionally, and through the centuries, artists performed experiments on perception and perceptual abilities, turning common tools into their instruments within this research process. At present, it seems that we entered a phase of vast, loud, colourful and impressive tableaus of clichés, overwhelming and highly sensational but less sensorially rich and diverse, with a focus on a software-hardware-in-use-debate, instead of critical strategies of (mis-)using tools, experimenting with subtle spatial formations, or claiming alternative (listener) positions. Therefore, we would need artistic research environments that enable us to understand the perceptibility of different listener backgrounds in very different and varying environmental designs. These research environments exist partly in some media art and music universities around the globe, but there they are bound to strict and limited rehearsal times and facility management, funding constraints, and quite often they stay hidden or closed for most artists in the ivory towers of academia.

 

 

// Art – Immersion – In – Compatibility

The successful involvement in an art experience is based on trust in the framework of art and its protective boundaries. Art may seriously provoke, but not seriously injure. Conventionally, art remains committed to the “as if” mode: the worlds it creates are real - but not real. At the same time, art renews itself and thrives on the border of violation and the frame break. With concept art, the idea replaces artful execution; with pop art, the difference between art and non-art falls; performance art grasps the border between life and art; Science competes with self-confident practices such as Lecture Performance or Artistic Research. Thus, we come to the question whether and, if so, how, these framework breaks are compatible with the laws of immersion itself. We should be particularly interested in the potential of intrinsic strategies, which point in and with the enjoyment of immersion on the questionableness of the same.




// Presence with Absence of Failure and Friction?

We can find and describe this questionableness present in most audio-visual media compositions from the past century. The common relations between image and sound (film music/video clip/VJing & DJing, audio-visual installation, and VR) routinely comprise little more than functionality and illustration. The field of possibilities of “imaginable” relations mostly stays un-approached. A mutual dramaturgy existing outside of the omnipresent and frequently established synchronicity e.g. of cut, morph and musical meter, is hardly to be found. Within the past 15 years, audio-analysis (e.g. volume, frequency, beat detection) plug-ins produced an additional “re-active” strategy of making something happen. The claim of an audio-visual language hovers over almost every work relating an audio track to a picture, or vice versa. But is this promise ever going to be acquitted? The arising anxiety in the moment of failure or sudden absence of synchronicity, or sound, or picture, leads very soon to judgments of defectiveness, instability, a “non-composition”. However, at the same time, a short moment of increased attention, a fissure, a disturbance in the field of the mass media shaped to-be-expected is emerging. The attention in the moment of friction, the latency between audio and video, is a gate for the searching of artistically utilizable narrative styles. To consider the specific characteristics of function and perception, is to introduce a (new) terrain. Therefore, how much detachment between sound and image can be produced, or is “allowed”, and what is happening with, and in, this gap? In other words: Can VR and AR with its basic technical immersion requirements make this happen? A terrain of defectiveness, failure and instability?

 
 

// Marketing’s Avant-garde – Dissolving Subcultures – Imperceptible Warfare?

[…] aesthetic ambition in this sense has largely collapsed. And this is because a huge portion of the population is totally subjected to the aesthetic conditioning of marketing, now hegemonic for the vast majority of the world, and is, therefore, estranged from any experience of aesthetic investigation.[14]

 

Immersion is by far not the exclusive domain of engineering, sciences, arts, music or media theory. The avant-garde of immersion can be observed somewhere else, insofar as immersion became one of the most important models for economy and its marketing strategies. Therefore, the term cannot be marginalized as a feature of state of the art media technologies, or the successful absorption into an artwork. The implications are deeper and more serious. Marketing refers to the process through which businesses and organizations promote themselves and their products by communication with potential customers. Global marketing has understood immersion as an unprecedented field of influence that changes the paradigm of traditional customer relation and market building.  In From The Power of Immersive Media [15] Frank Rose states: The most successful advertising today convincingly takes on the qualities of real experience. These experiences may be concerts, multi-media events, games and VR-applications. Rose asks: From 3D to VR, the goal is to eliminate any barrier between person and experience. What if we could have an unmediated experience?[16]  This unmediated experience can be translated as the substitute that Bernard Stiegler coins as conditioning. Aesthetics has become both theatre and weapon in economic war. This has resulted in a misery where conditioning substitutes for experience[17].

For Rose, immersion takes place when the audience forgets that it’s an audience at all. With the advent of social media, the hard sell and even the soft sell are giving way to what he calls the conspirational whisper[18].  Through immersion, marketing takes its chance at becoming invisible by omitting the medium that used to transmit the message. We could say that immersive technologies are producing the “stealth mode” of contemporary marketing strategies. Artists’ experiences in utilizing media technologies aesthetically – by researching and developing effects and affects to create artefacts, became the knowledge base for companies like Google, Facebook and Amazon striving to understand the fundamental rules of conditioning, to predict and control consumer behaviour 24/7 using the drift of technology toward the imperceptible.

Stiegler continues: In today’s control societies aesthetic weapons play an essential role: it has become a matter of controlling the technologies of aísthēsis and, in this way, controlling the conscious and unconscious rhythms of bodies and souls; modulating through the control of flows these rhythms of consciousness and life.[19]

As a matter of fact, in this economy of attention the reality of many has become tuneable by a few companies.

 

[…] this economy is an alienation of desire and of affects, where the weaponry is organized by marketing: Marketing is now the instrument of social control[20].

Accordingly Goodman states: The micropolitics of frequency points toward the waves and particles that abduct consumers immersed in both the transensory and nonsensory soup of vibro-capitalism.[21]

 

All this is concealed by the effects of the imperceptible medium itself, and hidden behind the language of PR.

As this is what Rose might call an immersive, technology-driven brand experience.

By living with and from YouTube as a multipurpose, every day tool of information, and by subscribing to Facebook and Google, we have started to work and live in post-democratic environments – voluntarily.  Facebook has become the closest thing many countries have to a public square. It is the largest, and one of the most important, places where citizens exchange views. And every interaction is governed by the terms of service of a hierarchically organized private corporation whose priority has to be profits. The eagerness with which these private corporations work[22] on the immersiveness of their products (“Enabling true social presence in VR”, “Introducing full-body Codec Avatars”, “Shared spaces in VR”) is unprecedented and measured by their economical power to overlap research, design practice, and human resources, they will be setting the predominant features and expectations in this field. And Rose concludes: […] Immersion is not achieved through assault; it’s achieved by inducing surrender. […] When the spell is broken, the audience snaps back to reality. The job of the 21st-century marketer is to make sure that does not happen[23].

 

 

 

// An Art of Immersion – Circumscribing – Borderline

There is great difficulty in separating out musical ideas, technological opportunities, and socio-political

and aesthetic nuances, but this also beckons great adventure. [24]

 

Art is free, shall not be part of any political campaign, nor shall it need to express a contemporary ideology, but we cannot close our eyes and ears in the current situation: sound and music are immersive parameters of reality – this is for real. The reason why I am deeply interested in technology is because it has the potentiality to let us learn more about our perceptive capabilities as human beings. I see technologies as training tools for aesthetic research for artists and scientists, and most ideally for artistic researchers. And in the process of research we turn tools into instruments, and with these instruments we are perceiving music out of something that some might call „The Contemporary“. These instruments can work like antennas, receivers.

Therefore, I do not subscribe to the common sense that technology is neither good nor bad. It is a question of their potentialities and who is going to explore and use them, with which motivation. So, in what sense do artists contribute to alternative world perceptions today, or are we just trying to be a part of "it"? Being sponsored by energy drink companies, lifestyle magazines and car companies means nothing more than being a part of their immersive strategies and campaigns. As composers, musicians, and sound artists we have to focus the paradox of immersion itself, which defines a core parameter for the understanding of today’s media development, but only occurs when the medium becomes imperceptible and disappears. Obviously, there is not a simple relationship of ‘‘either - or’’ between critical distance and immersion. But thematising exactly this paradox and the thin line between two worlds can be a contemporary artistic strategy for the utilisation of immersive technology.

An art of immersion should describe and stage the moment of transition from one world description into the other. The object of investigation is the brink, the borderline between “in” and “out”.

Immersion can be an intellectually stimulating process; however, in the present as in the past, in most cases immersion is mentally absorbing and a process, a change, a passage from one mental state to another. It is characterized by diminishing critical distance to what is shown and increasing emotional involvement in what is happening.[25] Similar to how film has made time manipulable, virtual acoustics as part of virtual reality makes this possible for space. Creating n-dimensional spaces - in the future, more than ever, we will decide what we surround ourselves with, and conceptualise the qualities of experiences possible. But by that we will not alter a virtual reality, we will change the everyday and how we perceive and understand it. We will have to overcome the current situation in which we are using the same tools like every marketing specialist with a team of very well paid and fully equipped engineers and designers, to instantly fulfil every strategic wish to manipulate bodies and minds, constantly and with entrance to the private everyday life of so many. We have to find a way of sharing experiences in immersive environments. What do we know about perception and perceptibility of spatial phenomena in n-D environments? Surely there is more than basic directional descriptions indexed to outdated reminiscences like “Kick” or “Snare” or “Voice”. So how can we communicate spatial qualities intrinsic to these systems? To benefit from varying technical and artistic viewpoints, individuals involved in artistic practice and those involved in theoretical or applied research, would need to engage in regular dialogue. This would have to happen on a level from which we could derive an “original” use of the immersive tools of our time, producing fundamentally different and challenging artefacts than marketing and PR are now offering - large scale and ubiquitous in public places, at fairs, in shops, smart homes or in the www. What else can we produce than overwhelming tableaus, overpowering multi-media products for piles of n-speakers, arranged as neat weapon-like arrays that we are not supposed to see and hear/feel/know about? In other words, we have to become more than, and very different from, advanced users. To answer this question, we practically need 24/7 access to tools and production environments. With the recent development in free downloadable software tools[26] for any kind of loudspeaker arrays and the upcoming wave of binaural software environments for head-tracked headphones, we can at least establish a continuous artistic practice. Actually, there are no excuses left. To produce and refine qualitative aesthetic differences in immersive environments, let us not decline and demonise the mainstream. Rather, let us embrace clichés to become experts on non-clichés. For example, by establishing and re-inventing surreal spatiality in VR - as normal. Maybe some will find a way to professionalise strategies of amateurism in VR and higher order loudspeaker environments, to contrast the spotless appearances of common media products and foster a fundamental aesthetic debate on immersive effects and phenomena. We could start by re-labelling our own “products”: Musical work is not necessarily engaged anymore with the traditional composition of songs, pieces or tracks, but with the conception of sonic spatial models (ssm?).  Whether we are writing music, composing, hacking sound, programming, creating tracks, most likely one would find a spatial strategy involved in the creative process (e.g. background-foreground shifts, left-right morphologies, spectromorphologies as vertical descriptions, sculptural qualities of sound, use of environmental sounds, use of convolution reverb).

 

 

// firniss_redux

We can describe the thin layer[27] between the one and the other world that preserves and finalises, but forms the completion and remains invisible. The border that no one perceives is the interface between two places that merge at it and thus become real. This layer can be circumscribed, tested, vaulted in multiple directions in order to hear what can be dilated, for how long and where, until the membrane becomes porous, cracked, permeable and clearly perceptible, thus thematising spatial acoustic perception itself: Making it vulnerable exposes it as defenceless and thus emancipates it at the same time. The original is created outside the common product. It remains unrealistic to the last and has the impertinence to be.

 

 

 

 

 

// --

Deleuze writes: “It is not a case of worrying or hoping for the best, but finding new weapons.”

 

Perhaps in the future, however, the artist will be concerned with something else. Identifying the weapon as such and confronting it with its greatest nightmare: The unmasking of its involvement and entanglement in everyday life, its exposure in the unconscious, by forcing it into consciousness through artistic actions.

 
 
 
 
 
 
 
 


[1] E.g. Book 2018: Immersive Sound: The Art and Science of Binaural and Multi-Channel Audio, Audio Engineering Society, Ed. Agnieszka Roginska, Paul Geluso,  Computer Music Journal, Vol. 41, No.1, High-Density Loudspeaker Arrays, Part 2: Spatial Perception and Creative Practice, https://www.mitpressjournals.org/toc/comj/41/1.

[2] E.g. exhibitions 2018 at Kuenstlerhaus Graz - Immersion in space and time, Berliner Festspiele Immersion 2018.

[3]  https://www.monomsound.com/about/.

[4] https://www.magicleap.com/news/partner-stories/bring-data-to-four-dimensions-with-immersion-analytics-visualizer-software, (all links accessed Nov. 2019).

[5] Grau, O. (2003). Virtual Art: From Illusion to Immersion (Rev. ed.). Cambridge, Mass., London: Mit., p. 49.

 

[6] Grau, O., (2003), p. 54.

[7] Bazin, A. (1967), What Is Cinema? Essays. Berkeley: Univ. of California, pp. 20-21.

[8] Calleja, G., (2007), Thesis: Digital Games as Designed Experience: Reframing the Concept of Immersion, Victoria University of Wellington, p. 89.

[9] Lennox, P., (2009) The Oxford Handbook of Computer Music, edited by R. Dean, New York 2009. p. 259

[10] Varèse, E., (1936), "The Liberation of Sound", Contemporary Composers on Contemporary Music, Perseus Publishing, 1998, pp. 197.

[11] Lennox, P., (2009) Oxford HdbK, p. 259.

[12] Kendall, G. S., and Ardila, M., (2008). The artistic play of spatial organization: Spatial attributes, scene analysis and auditory spatial schemata. In CMMR 2007, Lecture Notes in Computer Science 4969, ed. R. Kronland-Martinet, S. Ystad, and K. Jensen. Berlin: Springer, pp. 125–138.

[13] Stiegler, B., (2014), Symbolic Misery, Volume 1, The Hyper-Industrial Epoch, p.1.

[14] Stiegler, B., (2014), p. 3.

[15] Rose, F., ISSUE Spring (2015) 2015, strategy + business magazine, published by PwC Strategy& Inc., front page.

[16] Rose, F., (2015), p. 5

[17] Stiegler, B., (2014), vii.

[18] Rose, F., (2015), p. 8.

[19] Stiegler, B., (2014), p. 2.

[20] Stiegler, B., (2014), p. 13.

[21] Goodman, S., (2010), Sonic Warfare, Sound, Affect, and the Ecology of Fear, The MIT Press, Cambridge, Massachusetts, London, England, p. 188.

[22] facebook engineering, realistic sound: https://engineering.fb.com/virtual-reality/spatial-audio-bringing-realistic-sound-to-360-video/, facebook reality labs: https://tech.fb.com/inside-facebook-reality-labs-research-updates-and-the-future-of-social-connection/, Google AR/VR: https://arvr.google.com/, (all links accessed Nov. 2019).

[23] Rose, F., (2015), p. 10.

[24] Collins, N.  Oxford Hdbk, p. 350.

[25] Grau, O., (2003), p. 13.

[26] E.g. https://plugins.iem.at/,  (link accessed Nov. 2019).

[27] The following lines are also the liner notes for the sonic spatial model firniss_redux (binaural) published on this CD