Emotional Machines - Composing for Unstable Media.

Thorolf Thuestad

Emotional machines – composing for unstable media. Thorolf Thuestad In partial fulfilment of PhD in artistic research The Grieg Academy, Department of Music, Faculty of Art, Music, and Design, University of Bergen, 2021 The artistic research project Emotional machines - composing for unstable media explores how non-representational kinetic objects may afford affective and emotional responses, in particular experiences of kinship and relation between an audience and animated objects. Further possibilities for artistic expression are opened through composition and dramaturgy, in works where the animated objects perform alongside human performers and other media, such as light and sound. How may such animated non-representational physical objects be developed? How does animating the physical objects change the potential for experience of kinship and relation between audience and object? How does context modulate the potential for experience of kinship and relation between audience and animated physical object? How does the potential experience of kinship with the animated object differ for the creator of the object, and audience? These questions have been investigated through the creation of a set of animated kinetic figures. These figures have performed in a series of artistic presentations such as gallery exhibitions and stage presentations. The artistic research concludes with two complementary public artistic presentations taking place in bergen in June/July 2021; a time-based stage production at Cornerteatret and a gallery exhibition at Lydgalleriet. The presentations are considered connected and have the common title For one – for many – for all. This is an exposition of the research that presents the kinetic figures, the process of developing them, the questions they have contributed to explore, resulting artistic works (with particular emphasis on the final artistic presentations taking place in June/July 2021), the additional artists contributing to these public presentations, and a series of reflections resulting from the research. --------------------------------------------------------------------------- Emotional machines – composing for unstable media Thorolf Thuestad. Denne eksposisjonen presenteres som del av gjennomføring av doktorgrad i kunstnerisk utviklingsarbeid ved Griegakademiet, Institutt for musikk, Fakultet for kunst, musikk og design, Universitetet i Bergen, 2021. Emotional machines - composing for unstable media utforsker hvordan ikke-representative kinetiske objekter kan skape affektive og emosjonelle responser hos mennesker. Spesifikt opplevelse av slektskap og relasjon mellom et publikum og animerte objekter. Det åpnes ytterligere muligheter for kunstneriske uttrykk gjennom komposisjon og dramaturgi i verk der de animerte objektene opptrer sammen med menneskelige utøvere og andre medier, som lys og lyd. Hvordan kan animerte ikke-representative fysiske objekter utvikles med dette startpunktet som inspirasjon? Hvordan endrer animasjon av de fysiske objektene potensialet for opplevelse av slektskap og relasjon mellom publikum og objekt? Hvordan modulerer konteksten de animerte objektene presenteres i potensialet for opplevelse av slektskap og relasjon? Hvordan er den potensielle opplevelsen av slektskap med det animerte objektet forskjellig for skaperen(e) av objektet, og publikum? I Emotional machines – composing for unstable media har spørsmålene har blitt undersøkt ved hjelp av en gruppe animerte kinetiske figurer. Disse figurene har opptrådt i en rekke kunstneriske sammenhenger hovedsakelig galleriutstillinger og scenepresentasjoner. Prosjektet ble avsluttet med to utfyllende offentlige kunstneriske presentasjoner som fant sted i Bergen i juni/juli 2021: En tidsbasert sceneproduksjon vist på Cornerteatret og en galleriutstilling vist på Lydgalleriet. Presentasjonene er tilknyttet hverandre og har fellestittelen For one – for many – for all. Denne eksposisjonen presenterer de kinetiske figurene, prosessen med å utvikle dem, spørsmålene de har bidratt til å utforske, resulterende kunstneriske verk (med særlig vekt på de endelige kunstneriske presentasjonene i juni/juli 2021), kunstnerne som bidro til disse offentlige presentasjonene samt en rekke refleksjonstekster drevet frem av arbeidet med prosjektet.

For one for many – Bomuldsfabrikken

Arendal 26 – 27.10.2018

Huset Spiller (The House Plays) is a cross-disciplinary art project initiated by the musician and composer collective Lemur1. It makes use of the entire Bomuldsfabrikken gallery facility for concerts exhibitions and performances. For this iteration, they invited me, Siri Jøntvedt, Ellen J. Røed, Jan Halvor Bjørnseth and Ida Habbestad as guest artists.2

I was allocated a large white gallery space on the first floor for installing an exhibition of the animated figures I had created thus far as part of the research project Emotional machines – composing for unstable media. I had a sound system consisting of 4 full-range studio monitors and a couple of matching subwoofers available in the space.

The gallery space was a large white cube. In front of the entrance door, a large wall section had been constructed that compelled the audience to either make a left or right turn to enter the gallery space.

This offered the opportunity to create a route through the exhibition and thereby gain some control over the order in which the audience would encounter the figures, therefore affording some formal choices regarding how the audience experienced the exhibition.

In the first public exhibition of animated figures at Thresholds of the Algorithmic3, all figures had to be bunched together due to space constraints. The venue at Bomuldsfabrikken offered the possibility of creating space around each figure and for the audience to focus on both the characteristics of each as well as taking in the whole exhibition space.

The opportunity that this spatial arrangement offered prompted me to decide that I would create a path through the exhibition space that would enforce some linearity on how the audience encountered the works. I decided to install the sculptures in a manner that occupied most of the area available and created a natural audience path of sorts through the exhibition, albeit unmarked, for the audience to wander through the exhibition, passing each of the figures in a predetermined order before exiting the other side of the door.

With the help of a local technician I installed the figures in a distributed manner around the space.

The distribution of figures in the space was a subtle attempt at a non-specific social control over the audience. It was a big room, but the figures' distribution was made with the intent that it would be somewhat uncomfortable for the audience to step outside the only areas in the centre that were not "crowded" by sculptures. This was done to coax the audience into following the route I intended when making their way through the exhibition. This, for the most part, had the desired effect. Most adhered to the social cues that the placement of the figures implied. However, there were quite a few children in the audience, and they seemed in no small part oblivious to the social guidance provided by the room layout.

Featuring

The exhibition consisted of eight sculptures in pairs of two that are of similar design.

When first entering the exhibition space, the first pair of sculptures encountered was figure #1 and #2. If following the intended route, progressing into the room, figure #3 and 4 was placed to the left from the audience perspective with figure #5 and #6 placed on the right. As the audience made their way through the gallery space, they would at some point encounter figure #8 and as they approached the far wall, figure #7.

Interacting with the exhibited

Figures #7 and 8 instil familiarity because of their recognizable body configuration, yet unrecognizable because of substantial differences in how that configuration is constructed in the finer details. Similar from afar, different up close.

To me, this indicates two things: Firstly, that the familiarity of the body disposition gives an immediate almost instinctual affinity with the sculpture, secondly that this affinity is somewhat sabotaged by the differences that quickly become apparent on a closer viewing. There seems to be tension between the familiarity and the discrepancy. This seems related to something akin to the uncanny valley4 , but as it would relate to a machine looking like a non-human animal, we are familiar with rather than a human.

This indicates two things; firstly, the audience interacts with this sculpture in a manner similar to how they would be if they encountered an animal whose behaviour they would be able to a certain degree predict because of encounters with similar animals in the past. The size and body disposition place the figure into a category where the audience encounters it with a whole set of preconceptions about how it should behave. This gives this sculpture another way for the audience to form a relation to it compared to the other figures in the exhibition. They are much more wholly unfamiliar, and therefore expectations and conventions as to how they are to be taken are different or non-existent.

The untethered nature of figure #8 and to some degree #7, combined with their familiarity-uncanniness, created interesting audience-figure dynamics. The mobility forces the audience to have a much more acute awareness of #8 than any other figure. The other figures remain objects to be observed from a distance as a spectator. Curly becomes an object requiring awareness and therefore requires much more of a two-way relationship from the audience. The level and type of audience behaviour to accommodate it vary depending on its movements. This is fundamentally different from the other sculptures where the only audience accommodation required is to keep a safe distance.

Sound and light

I had access to 4 full-range studio monitors as well as two matching subwoofers for this exhibition. I created a sustained ambient soundscape with some slightly abrasive elements. For the most part, played back at a low level.

The soundscape was designed to create an ambience to comment on and enhance the experience of the sculptures and as an accompaniment for the sounds that several of the sculptures make. The sounds emanating from the figures were most prevalent in the pitched sounds of the air pumps, and also heard in the crackling of plastic contracting and being released, of plastic temporarily sticking to plastic skeletons as the air is sucked out by a pneumatic actuator and then let back in. When moving, the footsteps and motor sounds of figure #8 could be heard from wherever it may be located in the space. The audio played back over the speaker-system was mostly intended as a backdrop for the figures' sounds and movements.

Lighting was provided by available gallery lights. These were LED lights with a pronounced white colouring. I found the cold white light to be a good match for the figures. Since the light sources where physically relatively small they also offered the possibility to create shadows cast by the figures, mostly on the grey painted wooden floor, and in the case of figures #1 and #2, on the walls.

Talking to shadows

Figures #7 and 8 seem to inhabit a different stratum in the exhibition. If we consider the figures of the exhibition as part of an ecosystem, these were the dominant lifeforms by force of their mobility. The shadows cast by the other sculptures provided obstacles and objects they could seek out, that the immobile figures never could. I interpret figure #8 as in communication with the shadows cast by the other figures. In the case of the suspended figures #5 and 6, maybe it knows them only from their shadows?

Affinity

I received many comments from audience members experiencing affinity that the shape of the objects and the mechanism and movement evoke. Most relate to the physicality of the mechanism. For instance, for the air actuated figures, many of the audience reported being able to imagine the effect of having the air sucked out of them. Such physical empathy seems to operate on a more primitive level for the simpler figures in the sense that it mostly relates to a cognitive empathy with the physicality of the figures rather than with an imagined intent the figures might harbour. For figure #8: fingers, this relationship seems to be more ambiguous. It shares some actuator mechanisms with several other figures, particularly the part of its movement repertoire that is air-powered. Still, it also has movement capabilities quite familiar to us, like walking. This, combined with its ability to move "freely" seems to place it in a different perceptual category than the rest of the figures. Audience relations form towards an imagined intent. Figures #7 and #8 have both familiar and unfamiliar shape, which seems to create tension.

The humans and the machines

For the greater part of the exhibition period humans where spectators observing the activities of the figures. They would come in one side of the entrance and would by an large make their way along an intended path and out the other side of the entrance, observing the figures as they went along. It was reminiscent of how people are in a zoo, mostly looking, but with some attempts at establishing communication with the animals in the cages.5 Figure #8 represented a clearly different relation to the audience. Like the cages in a zoo limiting the zoo animal's movement, figures suspended by wires or without the power of locomotion are equally trapped. Figure #8 is untethered with its lack of physical wires and relative speed and ability and does not have this limit. This enforces a greater awareness of figure #8's presence on an audience. If #8 walks into the path an audience follows, it can disrupt the social obedience the audience has towards the gallery space conventions and force them to, for example, take a different route. A route that feels less comfortable since the audience now questions whether they are doing the "right" thing. Figure #8 smallish cuteness is amusing, but there is also a threat in the disruption of the social conventions of a situation in which the audience feels socially exposed.

Figure #8 is controlled manually, meaning that I operate a small midi controller, which triggers software sending commands that cause it to move. The other figures in the exhibition were automated and for the most part, controlled by a looped sequence of actions played back by a DAW. That meant that as rest of the installation was repeating (more or less) the same 25-minute segment, again and again, I could use figure #8 as an interlocutor taking part in a variable exchange with the other sculptures even if their trajectory was set. It also meant that I could reconfigure the layout of the space by moving #8 around.

This afforded some experimentation when it came to the handling of the flow of the audience. For instance, I was able to "intercept" audience members as they made their way through the exhibition space. The audience member would then have to stop or otherwise focus their attention on figure #8 rather than taking in the rest of the sculptures' activity. This type of action forced audience to be more proactive in their relationship towards figure #8. The other figures were taken to be viewed from some distance, but not interacted with beyond that. Figure #8's intervention ability gave the ability to traverse planes of audience interaction by being both an object for viewing from a distance(like the other figures) and being a force for action an audience member must be aware of and accommodate. I observed an exciting shift in the transition from an object of observation to an object of interaction. Figure #8 could demand attention from the audience, whereas the other figures relied on attention given. The awareness #8 demanded came temporarily at the expense of the attention afforded the other sculptures. That meant that not only did the role of figure #8 change from an object to observe into an object to interact with (albeit not physically), the attention shift meant that the part of the other sculptures also changed. From being objects, one could observe and try to understand, or look for a symbolic message or meaning in, into items of accompaniment to the activities of figure #8.

A central possibility was guiding the audience's attention to a much greater degree by using the activity of figure #8 to point out and comment on the activities of the other figures. Larger movements of the other sculptures in the exhibition could be used for a similar effect. Still, because they are stationary, they can never have the same ability to disrupt. They can, if the audience so decides, be ignored. When figure #8 acted in what I consider a confrontative manner, by, for example, positioning itself in the audience's path, there were no audience members that ignored them. By varying the attention demanded by figure #8, the exhibition gained a much more extensive "dynamic" range. The range of effects to be affected on the audience was greater.

The musician in the machines

At one point in the exhibition, a human musician made her way into the gallery space. Hild Tafjord joined the sculptures and the gallery space's soundscape for an improvised musical performance with or alongside the sculptures. Hild's improvised musical material consisted of quite expressive horn solo responding to the gallery space and sounds from other musicians performing elsewhere in the building.

Tribes and islands

Eight figures made up the Arendal exhibition in groups of two figures of similar design, so four distinct species were participating. The figures were distributed in the space to make the exhibition feel as spacious as possible. Each of the two members of each species was placed in quite close proximity to each other. Each sculpture was therefore through this proximity mostly viewed together with a variation of itself.

The movement segment's composition for all figures except figure #7 and #8 was played back as a loop. This segment was mostly structured to guide attention towards each species of figure successively. If there was a lot of activity in one "tribe", there would generally be less in another. My intent was to guide the spectator's focus from one species to the next so that their attention would wander throughout the space not at the audience members pacing but at a pace imposed by the timewise composition. By distributing the sculptures widely and having the audience move through the central area, the audience couldn't view all the figures simultaneously. The sequence of movements had some occurrences of timewise synchronization. Still, with the audience spending most of their time in the centre surrounded by the figures, these occurrences of synchronized activity would more be something that the audience sensed rather than saw. I felt they were useful but for other reasons than would be the case if it where possible to see the whole movement from the required distance. They provided more of an immersive experience because even if your eyes could not see every figure when such actions took place, the rest of the sensorium could pick it up. For an audience member in the centre of the space, there is undeniably a sense exposure but being surrounded there is also a sense of immersion.

Tip

Tip