Emotional Machines - Composing for Unstable Media.

Thorolf Thuestad

Emotional machines – composing for unstable media. Thorolf Thuestad In partial fulfilment of PhD in artistic research The Grieg Academy, Department of Music, Faculty of Art, Music, and Design, University of Bergen, 2021 The artistic research project Emotional machines - composing for unstable media explores how non-representational kinetic objects may afford affective and emotional responses, in particular experiences of kinship and relation between an audience and animated objects. Further possibilities for artistic expression are opened through composition and dramaturgy, in works where the animated objects perform alongside human performers and other media, such as light and sound. How may such animated non-representational physical objects be developed? How does animating the physical objects change the potential for experience of kinship and relation between audience and object? How does context modulate the potential for experience of kinship and relation between audience and animated physical object? How does the potential experience of kinship with the animated object differ for the creator of the object, and audience? These questions have been investigated through the creation of a set of animated kinetic figures. These figures have performed in a series of artistic presentations such as gallery exhibitions and stage presentations. The artistic research concludes with two complementary public artistic presentations taking place in bergen in June/July 2021; a time-based stage production at Cornerteatret and a gallery exhibition at Lydgalleriet. The presentations are considered connected and have the common title For one – for many – for all. This is an exposition of the research that presents the kinetic figures, the process of developing them, the questions they have contributed to explore, resulting artistic works (with particular emphasis on the final artistic presentations taking place in June/July 2021), the additional artists contributing to these public presentations, and a series of reflections resulting from the research. --------------------------------------------------------------------------- Emotional machines – composing for unstable media Thorolf Thuestad. Denne eksposisjonen presenteres som del av gjennomføring av doktorgrad i kunstnerisk utviklingsarbeid ved Griegakademiet, Institutt for musikk, Fakultet for kunst, musikk og design, Universitetet i Bergen, 2021. Emotional machines - composing for unstable media utforsker hvordan ikke-representative kinetiske objekter kan skape affektive og emosjonelle responser hos mennesker. Spesifikt opplevelse av slektskap og relasjon mellom et publikum og animerte objekter. Det åpnes ytterligere muligheter for kunstneriske uttrykk gjennom komposisjon og dramaturgi i verk der de animerte objektene opptrer sammen med menneskelige utøvere og andre medier, som lys og lyd. Hvordan kan animerte ikke-representative fysiske objekter utvikles med dette startpunktet som inspirasjon? Hvordan endrer animasjon av de fysiske objektene potensialet for opplevelse av slektskap og relasjon mellom publikum og objekt? Hvordan modulerer konteksten de animerte objektene presenteres i potensialet for opplevelse av slektskap og relasjon? Hvordan er den potensielle opplevelsen av slektskap med det animerte objektet forskjellig for skaperen(e) av objektet, og publikum? I Emotional machines – composing for unstable media har spørsmålene har blitt undersøkt ved hjelp av en gruppe animerte kinetiske figurer. Disse figurene har opptrådt i en rekke kunstneriske sammenhenger hovedsakelig galleriutstillinger og scenepresentasjoner. Prosjektet ble avsluttet med to utfyllende offentlige kunstneriske presentasjoner som fant sted i Bergen i juni/juli 2021: En tidsbasert sceneproduksjon vist på Cornerteatret og en galleriutstilling vist på Lydgalleriet. Presentasjonene er tilknyttet hverandre og har fellestittelen For one – for many – for all. Denne eksposisjonen presenterer de kinetiske figurene, prosessen med å utvikle dem, spørsmålene de har bidratt til å utforske, resulterende kunstneriske verk (med særlig vekt på de endelige kunstneriske presentasjonene i juni/juli 2021), kunstnerne som bidro til disse offentlige presentasjonene samt en rekke refleksjonstekster drevet frem av arbeidet med prosjektet.

Sonic Score

The compositions, for the most part, make extensive use of sound. When discussing the time-based organization and placement of these sounds in relation to themselves, to the figures and to the physical movements the figures may make, I use the term score. Score is here used exclusively to discuss the sonic part of the compositions.

This term is intended to facilitate discussing of the sound components independently. In the creative process towards a composition, the development of non-sonic materials, such as creating the figures or movement material, takes place in a parallel process to that of the sound. The sonic and non-sonic meet as the components are organized in time according to what I experience as their aesthetic weight.

There are other seemingly fundamental differences between the sound and everything else; the sound primarily operates as an invisible force (unless it is caused by the actions of a figure (non-human or human)). When creating the score, I hope for it to be ambiguous. The sounds within, like the figures themselves, strive to avoid direct recognition. Some of the sounds may appear diegetic, synchronized with movements. But what does that term mean if the object a given sound purports as its source, like the sound itself, is non-recognizable? I hope it will be ambiguous whether the score is intended to be an integrated part of the world and the time we see unfolding in a given composition, or whether it is as Michel Chion would describe it: “pit music”.1

I have chosen the term score since I consider it in this context to have similar meaning to how the term is often used in contemporary discussions of film sound. Sound in film is multi-layered with diegetic and non-diegetic layers of sound traditionally organized into a music track, sound design track (ambience, foley and sound effects) and dialogue track. A score in the context of a film has different connotations than for example concert music. A film score is always understood as music that reaches outside of itself, grounding itself in something outside (the film). This is how I think of the sounds in the compositions. The score may take on additional functions such as being a narrative or dramaturgical tool commenting on the activity happening with other components in the composition, for example the activities of figures. It could be illustrative of movement, or aid in creating an atmospheric ambience surrounding other elements in the compositions, forming an eco-system for them. It may connect actions occurring in physical space, creating coherence .

The integrated score

Danijela Kulezic-Wilson2 discusses how soundtracks in films in later years increasingly may be considered integrated, meaning that the sound design, dialogue, musical score, and set sound can be seen interdependent as a unified soundtrack. With that outlook, sound design may take on musical qualities, and sounds traditionally regarded as musical are given diegetic potential. This corresponds to how I think of the score part of the compositions. I consider the figures, their actions and the sonic events in the score as interdependent and musically motivated. Moreover, Walter Murch points out that film is a musical art form by nature, and therefore the soundtrack also becomes musical3. This statement resonates with how I think about the score for the compositions.

The impetus and inspiration for the creation of the scores are the figures. The score seeks to embellish and expand the reality in which the figures operate within a composition, and develop their ability to impart affect. The figures also seek to embellish and expand the reality of the sounds in the score and how they may form part of a composition. The score is therefore not intended as accompaniment.

An ambition for the project is to, as far as possible, avoid recognizable physical objects. By recognizable, I mean objects that most onlookers associate with a specific function or origin. Simultaneously, my hope is that the audience themselves experience recognition of "something" in the figures' movement and similarly in the sounds making up the score, but that those things will be beyond the limits of precise description. While seeking to avoid what I consider direct recognizability, the compositions still employ recognizability as one of their main expressive tools. That is, I hope that the audience may experience mimesis enabling engagement and imaginative perception resulting from the openness inherent in such mimetic recognition. This ambition is shared by the figures, their movements, and the sonic material in the score.

The sounds used in the score are also not intended to be directly representative in the sense of being recognizable as the sound of specific objects or instruments. Therefore, discussing what constitutes the musical part of the score and what constitutes sound design seems like a moot point. There is no intentional divide between the two, and I consider such a divide often seen in discussions of film soundtracks as irrelevant in the context of this project. For me, all sounds forming the score are both intended as part of a musical whole and as worldbuilding through sound-design. Together they make up the score of any given composition.

Reproductions?

Every part of a film is accepted to be a reproduction in some way. Therefore, a film score is separated from what we perceive as the real by the nature of the medium. It is viewed on a screen, and the sounds heard are created by speakers. Through suspension of disbelief, most of us seek to accept the images on the screen and sounds emanating from speakers as a reality we can be transported into. A film is often trying to convince us that the world it draws up for us is a reality, making it in effect a virtuality that wants to be “real”.

A primary difference between film and the compositions of this project is that the latter feature objects that would generally be considered "real". Material objects seem to be regarded as real by most. Ironically and unlike a film, the figures and even the humans in this project strive to escape their reality; they want to be ambiguous and non-specific in order to create the potential for the experience of multiple narratives and realities.

Even if the figures succeed in being experienced as non-representative, there is one thing they will likely not be able to avoid representing: a "real" physical object. As several of the figures also generate audible sounds as they move, there will also be a component in the score that will be heard as being caused by something "real" by most. This intrusion of the "real" into the realm of the non-representative is, for me, another key difference between the score in my compositions compared to the integrated score found in a film.

Although some of the sounds and all of the physical objects featured in the compositions are bound by "realness", the actions and sounds seek to explore some of the same possibilities afforded by the audience's acceptance of fiction as it would in a film.

A note on notation

I am aware that the word score as it is used in western musical practice usually relates to musical notation. This project also makes use of notation in various forms to organize compositional material. For example, I work with several excel documents that describe compositional segments, their durations and content. These are documents that look like what would, in the realm of film production, be described as shooting scripts.

Movement segments performed by the non-human animated figures are generally stored as sequences in the control system. Since the control system stores information about when and how the figures move and when and what sounds are played, I think it prudent to consider this as compositional notation as well. Lastly, there are certain segments in a few of the compositions where musical notation has been used to communicate melodies to be performed by a recorder player or singers.

Developing the time-based compositions is done in close dialogue with the figures, the physical materials from which the figures emerge, the human performers and the affordances of the tools and technologies used. As the figures and humans are unique to the compositions, there is no notational system that captures the nuances of the compositions. The compositions are inscribed in the figures themselves, in the memories of the humans and their unique presence in the performances.

The compositions are not intended to be reproducible or interpretable by other figures and human performers. Any notation is therefore internal and meant only for the contributors to the compositions.

Tip

Tip

Sonic Score

The integrated score

Reproductions?

A note on notation