The Memory Dealer as Pervasive Sound Narrative
Digital artist Rik Lander first developed The Memory Dealer (TMD) in 2010 as a small scale, experimental project designed to test whether mobile technologies could create an emotionally engaging narrative. An initial performance in the streets of Nottingham established the potential for innovative narrative experiences that merge personal media technologies (see Evans 2014). In May 2013 an expanded version was performed in the Harbourside region of Bristol. Creator Rik Lander describes TMD’s storyworld as follows:
The Memory Dealer is a ‘pervasive’ drama. Rather than sit in a darkened theatre, the audience (we call them players) have to move through the real world, visit locations, meet characters and exercise some agency over the outcome of the tale. The plot has the level of complexity of a movie and the player is a character in that movie, the scenes of which happen in sets or real places. An indication of the level of complexity and number of scenes can be gleaned from the following flow diagram. Player actions are in blue boxes. For example: ‘Player walks freely listening to app,’ ‘Player walks to hotel,’ etc. The red ovals represent the file names of the app audio. The audio soundtrack, listened to by the players on headphones, is the primary story-telling tool. It takes the form of a narration in a male voice, a musical score, sound effects, and character voices.
Figure 1: Flow diagram for TMD, Bristol, May 2013.
The player is told that they have come to the Watershed Cinema (where the event starts) to meet an old friend. As they have an hour to wait, they are taking a stroll and recalling her – how they met and how she became embroiled in “the whole memory thing.” Mobile phones can record and replay memories as “mems”, and there is an illegal subculture of memory exchange via “memory dealers”. The government plans to legalize this market and has sold the license to the Mevokia Corporation. Eve, a former “mem-head” (someone who became addicted to mems) is the leader of a campaign to stop legalization. Her organization, the XM, has been made illegal, and she is on the run. She has only days left to scupper the deal. The player is invited to meet her in her hotel room and to visit a police surveillance vehicle. In each of these locations they find media installations that offer fragments of her narrative. Eve has just discovered that her boyfriend is an undercover cop. Eve leaves a message from the police station. She’s been arrested and she needs the player to find a dealer who has her mem of yesterday. It proves her innocent of the charge, and the player must get it before the police can erase it. The Dealer will give the player the mem, but only if they experience it for themselves. Up to this point the players have been themselves; now they are both themselves and “Eve yesterday”.
The first thing Eve did yesterday was visit the dentist. As she (i.e., the player being Eve) meets the dentist, she realizes she is being betrayed. The player passes through several installations: a jewelers shop where Eve is arrested and a tent containing memories of Eve’s past. The player emerges from the tent in a police surveillance area where Eve can be seen on a TV screen being interviewed by a senior police officer. The player is then joined by all the other players partaking in the experience – they are all friends of Eve. On the screen they see Eve do a deal for her freedom. Eve gives the police the identities of all the senior members of the XM – in fact it is all the players. Their photos have been secretly taken during the course of the narrative, and they see them on the screen. The deal is that if they stay and are arrested Eve will be free to finish her work “saving memory”, but if they leave Eve will be charged. The players are told they have seven minutes before the police arrive.
Once they make their choice to leave or stay, the drama concludes with a text message that comes to each player’s phone. The content of the message depends on whether they have chosen to save Eve or not.
VideoObject 1: The Memory Dealer Trailer.
TMD sits within an emerging field of experimental, “pervasive” theatre that exploits digital and personal technologies to create fictional narratives that are then layered onto real world spaces. Such dramas remove narratives from the controlled, carefully constructed spaces of the theatre, cinema, or home and place them in public streets and buildings. There is no screen or proscenium arch separating the diegesis from its audience. Players move through and between different spaces that are transformed from their daily existence – a hotel room, a café – into a space layered with narrative detail and meaning. This involves a fundamental shift in both the form of storytelling and the positing of the audience. Rather than the narrative being told to the audience in the form of exposition or dialogue, the storyworld unfolds as the audience experiences it (see Evans 2014). Instead of being observers of the diegesis, audience members become participants in it, enabling narrative events to happen by their own actions. The narrative of pervasive drama is dynamic, with several “author-agents”, including Lander, his cast, and crew and the participants themselves, all contributing to each participant’s individual and unique experience. Work on such pervasive narratives is slowly emerging from fields such as Human-Computer-Interaction (Benford 2011; Evans, Martindale and Flintham 2014) or in relation to alternate reality games (McGonigal 2003, 2008, 2011). Much of this work focuses on issues of design or on managing the relationship between creator and audience (O’Hara, Grian and Williams 2008). The innovations in terms of storytelling techniques and technological exploitation at play in pervasive drama, however, mean that it is a form that is ripe for continued deeper interrogation.
Especially key to pervasive dramas is the use of personal technologies and an emphasis on sound (and the ear) as the locus of the narrative. In many ways, this sonic emphasis is inherent to the very nature of pervasive drama. On the one hand, the blurring of real and fictional space is a central pleasure and pursuit of pervasive drama and facilitates a range of subject positions as discussed throughout this special issue. To that extent, TMD offers researchers a new angle not hitherto covered by scholarship dealing with other, more conventional, sound-dependent narrative genres which actively seek to avoid this blurring, e.g. film and opera. On the other hand, there is also a pragmatic element. In order for players to navigate successfully through busy streets, complete with members of the general public and traffic, part of their attention must be on those streets to prevent accidents or injuries. This form of split attention, referred to by Jussi Parikka and Jaako Suominen (Parikka and Suominen 2006) as “bi-psyche”, requires a balance between the senses, especially sight and hearing. By focusing on sound, and so the participant’s hearing, dramas such as TMD are able to manage this balance, ensuring that enough attention, through sight, remains directed toward the real world around the participant and not on the fictional world inside their head. To construct a pervasive drama using mobile video would remove too much of the player’s sensory attention from their surrounding environment, removing both a key pleasure and increasing its inherent danger.
We argue that pervasive dramas such as TMD and “subtle mobs” such as As If It Were the Last Time (created by Duncan Speakman in 2009) or Coderunner (created by RocketChicken Interactive in 2013) can therefore be thought of as “sound narratives” in the way that they privilege sound as a, if not the, primary source of both the narrative and the participant’s experience. As such, they sit within a nexus of sound based media – most commonly personal music players, radio, and audio books – that are in a unique position to reshape our relationship to the space around us. Michael Bull articulates this in the model of a personal “sound bubble” (Bull 2004; see also Wilken 2010) through an analysis of the personal stereo. Different media forms (and different forms of sound narratives), however, construct different kinds of bubbles and, in turn, different experiences for individuals and different relationships between that individual, the device and their surroundings.
As the TMD player navigates this bubble and is exposed to the blurring of the real-fictional boundary in a sequence of sensory stimuli (sonic, visual, and – while moving – kinesthetic), what remains the most direct experience of the drama is undoubtedly the soundtrack: “We can close our eyes, but we cannot shut our ears” (Cobussen and Nielsen 2012: 8). And this “direct listening” has the capacity to be not just attentive, but engage us physiologically in sensual and sensory affective processes, as Anahid Kassabian (Kassabian 2013) has argued in her exploration of our developing capacity to engage with the “ubiquitous musics” of everyday life. The possibility for such an interactive sensory and affective engagement is summed up well in percussionist Evelyn Glennie's words that hearing is a “specialized form of touch” (Glennie 1993: 1). And once considered a more all-encompassing engagement, the oft-cited distinction between hearing (as physiological) and listening (as conscious and attentive) is erased: all listening can be considered physiological and includes several kinds of consciousness and attention (Kassabian 2013). As a sonic narrative, partaking in TMD thus brings to the fore the blurring of further boundaries; it requires the active participation of the whole human being, physically and mentally, in a world where technology regularly provides us with ubiquitous sound more often than not. This both heightens and challenges our capacity to interact sonically. It is through listening that the engagement in TMD becomes fully immersive.
Because it provides a fictional world layered onto a real world and because it plays with participants’ relationships with everyday ubiquitous listening in novel ways, TMD presents new perspectives for sound studies. Recent scholarship has attempted to map theoretically the ways in which individual auditory environments interact with the dynamic soundscapes of the outside world and transgress the boundaries between private and public spheres (e.g. Fluegge 2011); in its own unique way, TMD offers important empirical material for this kind of theoretical groundwork. The research into TMD presented here can also be considered a timely response to the recent call (in this very journal) for uncovering and understanding particular “sonic forms of knowledge”, i.e. understanding – in the first instance – how we respond and act according to auditory impulses that may lead to new knowledge:
[I]nstead of fixed identities and meanings, stability, nouns, and stasis, the sonic exposes us to action and movement, to fleeting understandings, verbs, and contingent possibilities. The ear’s focus is on process, on objects and events existing in time. (Cobussen, Schulze and Meelberg 2013)
As a thoroughly process-based, interactive pervasive drama, TMD opens up possibilities for different engagement with a variety of (internal and external) sonic spaces and thus becomes part of this current move towards a “new sonic materialism” (Cobussen, Schulze and Meelberg 2013) where the listening subject is central and where understanding the sonic involvement can tell us something important about our engagement with the world as well as the knowledge that emerges from this engagement. In this sense, as a novel sound world prompting new responses – responses that can be explored and analyzed – TMD is a rare form of documentation, providing fertile ground for exploration.
Epistemology aside, another crucial contribution to current sound studies emerges from the way in which TMD – especially because of its setting in real life, involving encounters with real people – requires participants to mediate disbelief in a constant juggle between reality and imagination rather than suspend it. Suspension of disbelief is normally the case with more conventional genres like film, opera, or video games, where the successful experience is dependent on a constant imaginary leap that remains passively situated in one physical place (e.g. the cinema, opera house, or bed room). The experience of pervasive drama is mediated in a radically new way because it is dynamic and in flux, rather than passive and static: the drama unfolds through a constant negotiation between different kinds of awareness and engagement, where the reality (and unreality) of each player is constructed as the narrative develops. Because of TMD’s intrinsic realism, the imaginary involvement is challenged in novel ways. Furthermore, this mediation occurs through a particular kind of subjective involvement (via the direct experience of sound), where immersion is directly related to the player’s subject position, i.e. where the player assumes either the self of a character (when “in character” as Eve) or the real self, e.g. when receiving particular instructions or reflecting on the next part of the action. If the classic narrative Hollywood film genre has paved the way for understanding the soundtrack as a narrative aid, the experience of pervasive drama is a multi-layered one, and TMD provides new scopes for research within the growing area of subjectivity studies and invites new approaches to uncovering the phenomenology of listening. While exploring the new kind of media engagement that an innovative drama such as TMD offers, this collection seeks to address the current lack of interrogation on the nature of sound narratives and the consequences of placing sound at the center of the narrative experience.
Methodology: scope and limitation
The Moving Experience research group was established at the University of Nottingham to work with Rik Lander on exploring the relationship between technology, place, and narrative that pervasive dramas such as TMD interrogate. At the center of this triangle are those who are engaging with TMD as a piece of pervasive theatre. The fact that we refer to participants as players reflects TMD’s reliance on interaction, task completion, and puzzle solving within the narrative in addition to listening and watching. Although in many ways this is a semantic cheat, for these players are also listeners, viewers, readers and actors, this is also done for the purposes of clarity. As Sean Hammond, Helen Pain, and Tim J. Smith argue, “player” can encompass multiple subcategories of role:
A player in an interactive narrative can be a spectator in the sense that she is a witness to the dramatic spectacle. She can be an actor in the sense that she plays the role of one of the characters in the narrative. And she can be an author in the sense that she collaborates with the system (and perhaps with other players) to produce the resulting narrative experience. (Hammond, Pain and Smith 2007: 2)
These players, their experiences whilst engaging with TMD, and their relationships to its diegetic elements have been of key interest to members of the Moving Experience group. This focus, however, has raised a number of methodological questions concerning how best to monitor and capture data on players involved in an innovative form of drama (often for the first time), a genre that did not have a ready-made cultural framework for discussion or interrogation.
Conventional qualitative and quantitative methodologies, such as focus groups or questionnaires, have long been the staple approach for media or cultural studies scholars to analyze audience engagement with narrative texts. Such approaches provide the opportunity to gather responsive data, to examine the pleasures or problems that participants experienced by allowing players to talk through and reflect on their experiences. In the case of TMD, focus groups were run after each player-group to explore key questions about the nature of participants’ experiences, their responses to the music, and their engagement with the narrative and its characters. These focus groups involved a total of 124 participants over 30 groups.[1]
However, the nature of pervasive drama raises a number of issues concerning the limits of post hoc approaches to empirical research. The innovative nature of TMD, which was the first pervasive drama experience for many players, means that it can be hard for research participants to articulate their responses and feelings in the limited space and time afforded to a written questionnaire. Verbally conducted focus groups, although far more useful in allowing opportunities for participants to explore ambiguous or contradictory responses to the piece than questionnaires, rely on memory and recall. Although the interviews were conducted immediately after each performance, participants were still required to recall events that they had not yet had time to reflect on and that may have started to merge together or become confused. Another factor was the group setting, which was reflected in some answers: responses could be enthusiastically confirmed collectively as well as silenced by the most verbal in the group, and the interviewer had to carefully steer discussions accordingly, bringing out as many nuances as possible. At the same time, as Matt Hills has argued in relation to understanding the pleasures of horror films, focus groups rely on “affective discourses” which are “the result of any cultural performance of affect […] rather than how or whether a specific pleasure is actually felt by the subject” (Hills 2005: 7). TMD opened up the opportunity to consider alternative methodologies that could make use of the communal nature of the research team, most notably more observational forms of research.
Conventional forms of narrative – such as television, film, or theatre – make observational techniques difficult. They require either access to private domestic spaces, such as the home, or observing large groups of people simultaneously whilst in darkened auditoria. Pervasive dramas such as TMD raise an alternative, fruitful space for observational data. The geographical structure of TMD over a number of public locations made it possible for the research team to select key observational “posts” where an observer could remain relatively inconspicuous, so as not to disrupt players’ experiences, but visible, to prevent ethical issues with covert observation and maintain a clear view of how each player acted during the actual engagement in the drama. Each of these moments were also key turning points in the narrative, when players were being directed into a new space or into a new form of engagement. The first was at the bridge, where players were directed to the car-based installation. Here the research team was able to observe their interaction with a character. For half of the group, this was the first moment in which they were required to become performers, to interact directly with a cast member rather than listen to the narrative via headphones. The second location was the café, where players were instructed to acquire Eve’s memory from the Dealer. Unlike the first moment, players were here positioned as a group, and so it was possible to see small moments of interaction between players. Both of these first two observational posts allowed the research team to see a moment of transition as players were required to locate where they were supposed to be, who they were supposed to talk to, and what they were supposed to say or do. The final observational post was in the dentist’s reception space. Unlike the other locations, this was a semi-private space, but the diegetic meaning layered onto it (of a waiting room) allowed for the presence of an additional person in the room. This observational data is not without its own limitations. Such observations of player actions lack a contextual framing that may explain why an individual acted the way they did. However, when combined with the focus groups through a process of triangulation (Rosengren 1996; Schrøder 1999), they offer an additional, useful dimension to the research data that is not filtered through the participants’ memory or discourse.
There is, of course, an additional methodology at play in the following articles as well, one of personal and professional reflection on the part of Rik Lander, creator of TMD, and Alex Kolassa, its composer. Such work seldom finds space within academic scholarship, outside of work relating to game design written by game designers (McGonigal 2008). Lander’s work here offers an opportunity to interrogate practice from an auto-ethnographical perspective. Such an approach provides insights into how creative practitioners actively engage with their own work as well as giving a unique perspective on the questions that concern academic scholarship.
[1] In the following articles, each focus group participant is referred to anonymously by an alphanumeric code, denoting which group they took part in, with the first three letters referring to the performance day and the fourth letter referring to the individual performance. For instance, “FRC1” refers to participant number 1 in the third focus group on the Friday.
Interrogating The Memory Dealer
The Moving Experience group is deliberately interdisciplinary and consists of scholars from music, media, and cultural studies. Mirroring the authors’ interdisciplinary backgrounds, the following articles reveal varying relationships to sound studies as a discipline. However, each deals explicitly with the consequences of privileging sonic storytelling. The opening article of this special issue, by TMD’s creator Rik Lander, reflects on the process of designing the piece and considers how best to engage players in new and innovative narrative experiences. He uses focus group data and his own practice to contemplate the role of the creator in pervasive drama “sound narratives” and offers guidance for future productions.
Evans and Giraud take perspectives from transmedia, literacy, and cultural studies. Evans examines the processes by which TMD’s otherwise “literate” players were confronted with their lack of skills when engaging with a sound narrative. She asks what innovative forms like TMD reveal about the way adults learn and adapt to unexpected narrative forms and subject positions, identifying the convergence of multiple forms of literacy encompassing narrative, technology, and the geo-logistical nature of public space. She explores the way players articulated moments when they were confronted with their own illiteracy and how, for many, this became a source of anxiety as they felt they should be able to successfully navigate their way through the drama. Giraud, meanwhile, considers the cultural politics of using mobile audio technologies to layer narrative onto public space. Rather than seeing personal sound technologies as constructing a negative relation to public space, as is often presented within cultural studies research, she instead considers the potential to use such technologies to engage the public critically. By reshaping and “defamiliarizing” public space, dramas such as TMD highlight the “agency” of technology and offer a highly valuable space for politically engaging with the everyday.
Hibberd, Nielsen, and Kolassa take perspectives from musicology, philosophy, and sound studies, explore the musical and sonic sides to TMD, and analyze the workings of the soundtrack. As the composer for TMD, Kolassa reflects on producing a soundtrack for this new, experiential drama, and situates his methodology in a selection of aesthetic approaches within the avant-garde and electroacoustic music, including those related to soundwalking and acoustic ecology. Kolassa also discusses the contribution of video games and film scoring to his compositional techniques for TMD. Having served as musical advisers for the project, Hibberd and Nielsen interrogate the extent to which the TMD soundtrack can be argued to fit the model of narrative film music, or whether it shows itself to defy some of the more traditional patterns. Based on responses from the players, they argue that the TMD sound world is best understood through a framework focusing on construction of subjectivity (real and imagined) through sound and by recognizing the multiple ways in which a pervasive drama like TMD enables different kinds of engagement with a variety of sonic spaces. While TMD on some levels proves to be irreducible to traditional narrative approaches, it opens up new, exciting possibilities for guiding and constructing subjectivity through sound.
We do not wish to present this special issue as a definitive study of pervasive drama or “sound narratives”, as we call them. The design of TMD demanded creativity within the limitations of current mass-produced technology (in order to ensure that as many players as possible could take part) and necessary budget constraints, something that has been a part of early production in most mediated forms (see Jacobs 2000 for a discussion of early television’s limitations and creativity). Whilst ultimately a successful example of a pervasive drama and sound narrative, the genre has much scope for further expansion and innovation as audio and personal technologies develop and become more accessible. As such, the debate around sound narratives’ structure and value only begins here. We do not even wish to claim that these five articles are the only way to critically interrogate TMD. What these articles do offer, however, is a timely interdisciplinary exploration of an emerging form as it begins to develop and exploit equally emergent mobile media technologies. The experimentation within dramas such as TMD requires continued critical reflection and evaluation in order for scholars and practitioners to understand the key questions they raise about the relationship between the individual, textual forms, technology, and public space. We hope that they will raise questions that cross (between) disciplinary boundaries, particularly as digital and personal technologies open up greater potential for innovative forms that tell stories in unique ways.
References
Benford, Steve and Gabriella Giannachi (2011). Performing Mixed Reality. Cambridge, MA: MIT Press.
Bull, Michael (2004). “‘To each their own bubble’: Mobile Spaces of Sound in the City.” In Nick Couldry and Anna McCarthy (eds.), MediaSpace: Place, Scale and Culture in a Media Age (pp. 275-293). London: Routledge.
Cobussen, Marcel, Holger Schulze, and Vincent Meelberg (2013). “Editorial: Towards New Sonic Epistemologies.” Journal of Sonic Studies 4.
Cobussen, Marcel and Nanette Nielsen (2012). Music and Ethics. Farnham: Ashgate.
Evans, Elizabeth, Martin Flintham and Sarah Martindale (2014). “The Malthusian Paradox: Performance in an Alternate Reality Game.” Pervasive Ubiquitous Computing. Issue to be confirmed.
Fluegge, Elen (2011). “The Consideration of Personal Sound Space: Toward a Practical Perspective on Individualized Auditory Experience.” Journal of Sonic Studies 1.
Glennie, Evelyn (1993). “Hearing Essay.”
Hammond, Sean, Helen Pain and Tim J. Smith (2007). “Player Agency in Interactive Narrative: Audience, Actor and Author.”
Hills, Matt (2005). The Pleasures of Horror. London: Continuum.
Jacobs, Jason (2000). The Intimate Screen: Early British Television Drama. Oxford: Oxford University Press.
Kassabian, Anahid (2013). Ubiquitous Listening: Affect, Attention, and Distributed Subjectivity. Berkeley: University of California Press.
McGonigal, Jane (2003). “A Real Little Game: The Performance of Belief in Pervasive Play.” 030303: Collective Play. Research Colloquium, University of California at Berkeley.
McGonigal Jane (2011). Reality is Broken. New York: Penguin Press.
McGonigal, Jane (2008). “Why I Love Bees: A Case Study in Collective Intelligence Gaming.” In Katie Salen (ed.), The Ecology of Games: Connecting Youth, Games, and Learning (pp. 199-227). Cambridge, MA: MIT Press.
O’Hara, Kental, Hazel Grian and John Williams (2008). “Participation, Collaboration and Spectatorship in an Alternate Reality Game.” In Proceedings of the 20th Australasian Conference on Computer-Human Interaction: Designing for Habitus and Habitat (OzCHI ‘08) (pp. 130-139). New York: ACM.
Parikka, Jussi and Jaako Suominen (2006). “Victorian Snake? Towards a Cultural History of Mobile Gaming and the Experience of Movement.” Game Studies 6/1.
Rosengren, Karl Erik (1996). “Combinations, Comparison and Confrontations: Toward a Comprehensive Theory of Audience Research.” In James Hay, Lawrence Grossberg, and Ellen Wartella (eds.), The Audience and its Landscape(pp. 23-52). Boulder, CO: Westview Press.
Schrøder, Kim Christian (1999). “The Best of Both Worlds?: Media Audience Research between Rival Paradigms.” In Pertti Alasutarri (ed.), Rethinking the Media Audience (pp. 38-68). London: Sage.
Wilken, Rowan (2010). “A Community of Strangers? Mobile Media, Art and Urban Encounters with the Other.” Mobilities 5/4: 449-468.