Download article as pdf

Storyworld 2.0

A brief exploration of developing digital transmedial universes.

Prologue

" When I first started you would pitch a story because without a good story, you didn’t really have a film. Later, once sequels started to take off, you pitched a character because a good character could support multiple stories. and now, you pitch a world because a world can support multiple characters and multiple stories across multiple media" Henry Jenkins quoting an unnamed Hollywood scriptwriter in both ‘Transmedia Storytelling’ (2007) and in [Convergence Culture]{.underline} (2009).

Since Henry Jenkins wrote his groundbreaking books and articles about transmedial storytelling a decade ago, the digital revolution has changed both the way that we experience fiction and how we fictionalize experiences across multiple formats.

For me exploring digital Storyworlds is about creating quality in the future, and my hope is:

Research design

In this artistic research project I am exploring how I can build digital Storyworlds (digital transmedial universes). I have divided my research into three phases:

Phase 1: How are Storyworlds made?

Phase 2: How can I develop a Storyworld?

Phase 3: How can I develop a digital Storyworld?

Between each phase I will redefine the next phases, the research questions, and even the overall aim of the project if necessary.

In the course of my project I will create a concrete digital Storyworld. But this will only serve as a test bed in the broader search for tools and methods to create digital Storyworlds. The product of this project is a paper with practical suggestions for my colleague artists, and I have also included three Storyworld exercises that can be used by teachers and students.

PHASE 1: how are Storyworlds made?

Phase 1 is an exploration of the Storyworld field.

How? By researching universe and Storyworld creation historically, theoretically, and in the practices of both myself and other artists.

My own practice

Looking back on myself as a child I was always covered in LEGO, playing with teddy bears, Barbie dolls, cars, building blocks and anything else I could use to tell stories and build worlds. This playful passion has continued through my professional life as a director, building worlds and creating stories for radio, documentary films, TV series, interactive narratives, games, books, and apps.

In 2016 I was asked to head the directing program (Animation) and DADIU (Danish Academy of Interactive Digital Entertainment) at the National Danish Film School. Here I met some students who told me that they were working on something called Storyworlds. The word STORYWORLD itself sounded super sexy to me. And after digging into the teaching material, I found the name of a professor at Filmakademie Baden-Württemberg in Germany, Prof. Inga von Staden, who I decided I had to talk to!

I first contacted Prof. von Staden in 2017 and found out, not only that we had friends in common, but that her school even uses some of my earliest games in their teaching. We first met via Skype, then face to face, to discuss the idea of Storyworlds. I especially remember a conversation about whether one could create a universe without having a specific format in mind. I said that it would be impossible for me, as I have always created my stories for one or, maximum, two formats (i.e. a game and a book). Inga claimed that since I have worked with so many different audiovisual formats, I should take up the challenge and try to work with format-agnostic universes. This was a new way of thinking, at least for me.

Inspired by our talks, I began to explore the phenomenon of Storyworlds in 2018. The first project consisted primarily of a series of student workshops and small, practice-based experiments with three teacher colleagues and a group of professionals.

Then I began to research the subject further.

Storyworld thinking

In my research I discovered that, though quite a few researchers have thought and written about Storyworlds, there were still a range of different definitions. I have summarized them below.

“When we can take green from grass, blue from heaven, and red from blood, we have already an enchanter’s power … in such “fantasy,” as it is called, new form is made; Faerie begins; Man becomes a sub-creator.” (J.R.R. Tolkien, ‘On Fairies’, 1939).

According to J.R.R Tolkien, author of The Lord of the Rings (1954) and other fantasy books, who both created universes and theorized about them, a Storyworld is anything we can imagine that is beyond our own world (i.e. Trolls, Ghosts, Sci Fi). Tolkien defined Storyworlds as sub-creations. Because he was religious, he believed that God was the creator of our world and that we as humans can only be sub-creators; and as such, that we can only create worlds that are imaginary - or fantasy.

“The story world isn’t a copy of life as it is. It’s life as human beings imagine it could be”. (John Truby, The Anatomy of Story: 22 Steps to Becoming a Master Storyteller, 2007)

John Truby, a professional Hollywood script doctor, has also written about Storyworlds. According to him, Storyworlds are created from imagination, but can be any fictional world that we invent, realistic or not. In other words, he broadens Tolkien’s definition to include realistic worlds as well - so long they are fictional.

“The design of the logic of the world precedes the telling of the story…The design of the world is as important to the development of the narrative as the script is to the development of the design… The world becomes a container for multiple narratives.” (Alex McDowell, ‘PD4C21 - Production Design for the 21st Century’, 2017)

Alex McDowell is another prominent filmmaker who has theorized about Storyworlds. He is one of the world’s leading production designers and has founded the Worldbuilding Institute at USC (University of Southern California). His take on Storyworlds comes from a design perspective, and one of his main points is that the audiovisual design of a universe (i.e. space, look, and feel) is just as important as the underlying script.

Jesper Juul, a prominent game researcher and one of two programmers on my very first game (Blackout, 1997), has defined Storyworlds from the perspective of a game designer. According to Jesper, the world for a game is not necessarily the same as the world for a movie. The world of a game is more like a playground in which the player puzzles together his own story. Or, as Jesper puts it, a fiction can be “any world, regardless of whether it is fixed, linear, interactive”. (Jesper Juul, quoted at Let’s Play conference in Copenhagen, 2019). In this way he expands the concept of ‘Storyworlds’ beyond the boundaries of the strictly narrative.

“Storyworlds as possible worlds of fiction (and non-fiction) and as cognitive constructions in the mind of the “reader”.” (Marie-Laure Ryan, ‘Story/Worlds/Media: Tuning the Instruments of a Media-Conscious Narratology’, 2014).

And then there is Marie Laure Ryan, who is a literary scholar and critic. Her definition of the Storyworld is that it stands between the audience and the media, binding the two together through experience. In her definition, she includes the users’ own imagination and the experience of exploring a universe across formats, i.e. playing a Ninjago game, then watching a Ninjago Movie, and then playing with the Ninjago LEGO bricks. In this way, Ryan expands the Storyworld definition beyond the creator.

Storyworld creation

My initial research into the theory and history of Storyworlds has given me insights on Storyworld thinking and perspectives. But in my research, I found little or no information on how Storyworlds are actually created. This kind of surprised me, as you can find tons of material on dramaturgical tools to create stories. Even with a young format, such as game creation, you can find piles of books about how to design them.

So I wondered, how do you actually create a Storyworld? Which tools can you use? And which methods can you apply when you want to create a Storyworld? These are some of the questions I have discussed with colleague teachers and researchers.

In 2018 my colleague, Prof. Jakob Wille of the Royal Danish Academy of Design, and I, went to Los Angeles to explore these questions. The trip was part of a project called “World Building Club Denmark,” which was funded by Nordisk Film Fonden. During the trip, we met with various professionals and researchers from UCLA and other institutions. And though we did not find the tools and methods we were looking for, we did have many inspirational conversations. One of these occurred on a long flight, during which Jakob interviewed me about my own creative pipeline and how I have worked with universes when directing in different media, from documentary film to games. This led me to the idea that, instead of looking to academia, I could hold a series of interviews and conversations with artists working across different media formats in order to find artistic methods and tools with which to create Storyworlds.

Every conversation would occur with two artists from the same field (film, theater etc.), at the same time, and the two artists had to be as different as possible.

To make sure I would arrive at structured material, I set certain rules for the interviews. Namely, that each interview would take only half an hour and have exactly the same structure:

The artists had to have a substantial practice, thus to have developed their methods over many years.

The formats I explored were:

Comic artists Ole Comoll and Palle Schmidt

Film directors Esben Toft and Rasmus Klosterbro

Theater artists Pelle Koppel and Siggi Oli Palmason.

Game artists Michael Ruud and Charlene Putney.

TV series creators Hannah Lundblad and Mads Grage.

I had hoped to get some concrete universe crafting tools from these interview, yet I found that the artists’ methods were so craft-specific that I would have been unable to master them without a deeper practical knowledge and understanding of their work and formats. Just as it takes many years for a carpenter to properly use a plane, it takes many years to master writing scripts or designing games. I did, however, discover something else:

All of the conversations were recorded; and after listening to them several times I realized that, though the artists’ tools and methods differed, there was a common trait: each artist had already created the majority of their Storyworld at the very beginning of the artistic process! I don’t think it would be wrong to say that most artists know more than 80% of the core theme, characters, and places already during the earliest stages of development, regardless of the artistic format.

Another significant common trait is that every artist has their own method for outlining, which they use to test their universes before embarking on the “hard”, “boring”, “heavy” work of scripting/designing the format. In the interviews, several of the artists moaned when talking about drawing, writing manuscripts or storyboarding. As comic book artist Palle Scmidt says; “You need to know your story and do thumbnails before you start doing the hard and tedious work of drawing”. This also seems to be true for game and theater artists, who work in a more cyclical (iterative) fashion. They, too, have unique outline methods to nail down the core concept and the respective world early on in their work-flow.

Some types of outlines mentioned by the artists were:

Phase 1 Summary:

In the first phase I asked myself the question: “How are Storyworlds made?” Here is what I discovered:

My own practice: I have always thought up universes when working on radio, film, tv, games and other formats, but I have never thought about how I do it. Nor, as with most artists, have I ever documented my own methods.

Theory, history and thinking: According to Tolkien, this makes me a sub creator. And according to various thinkers, Storyworlds can be fictional, factual, or game worlds. And according to Marie Laure Ryan, Storyworlds can even be defined as the intersection of the common elements from several different formats, simultaneously serving as transmedial universes, or even as universes that exist within the imagination of the audience themselves, who create their own experience within and across various artistic formats.

Artist practice: Even though I did not, in the course of my interviews with the artists, find any concrete tools or dramaturgical methods for creating Storyworlds, I did find inspiration on how to create Storyworlds through the use of outlines. But this idea poses a new question: What is an outline for a Storyworld?

EXERCISE 1:

I have created and used the following exercise with my students. The aim is that you, a single person, can - in 3 hours - learn to analyze and expand an existing Storyworld using the ideas of thinkers like J.R.R. Tolkien and Marie Laure Ryan.

1: Take one of your favorite stories with a rich universe (from a book, game, etc)

2: On five separate A3 pieces of paper, draw:

3: Reflect on the potential for expanding the universe into various other formats and develop concrete ideas/outlines for these (50 min).

4: Reflect on how these publications could help the audience to construct worlds within their own imaginations (between formats). Then, write a second iteration of the format outlines focused on enhancing the Storyworld within the users’ imaginations (50 min).

What are your insights?

PHASE 2: how can I create Storyworlds?

Phase 2 is a search for how I can develop a Storyworld outline method that could be used in both my own practice and that of others.

By replicating the idea-outline-change loop used by the artists I interviewed, and then testing it by developing an actual universe, I hope to create an outline method for developing entire Storyworlds. In this section, I call it a ‘world-outline-change loop’.

Creating a Storyworld

In this section, I will develop an outline for a Storyworld titled Borgergade (literally, “Citizen Street”). This is based on an idea that I had many years ago while developing the comic-inspired game series, Crosstown (‘Giften’, 1999 and ‘Englen’, 2000).

Borgergade takes place on the street of the same name, and focuses on the very poor community that lives there. The citizens of Borgergade lack even the most basic resources necessary to sustain their lives. This leads them to be both completely dependent upon each other and, simultaneously, completely unable to trust the other equally desperate people around them. This creates a toxic dilemma and the dramatic fuel for this Storyworld.

The Borgergade Storyworld was partly inspired by Lorte Renden (“Shit Creek”), a contemporary shanty town in Copenhagen’s Sydhavn (South Harbour) district. Lorte Renden is inhabited by people who live outside of society. Some of the residents are

dangerous, have spent years of their lives locked up in jails or mental institutions, and all of them are generally in deep shit.

Lorte Renden shares many things in common with the old Copenhagen neighborhood of Borgergade, which was demolished around 1950. Here, about 10,000 people lived at the bottom of society. 8,000 of them were children, and one in six adults worked as prostitutes.

From that starting point, I designed a two-day workshop where the goal was to create a Storyworld without thinking about whether it would be used for a movie, game, or another format. The participants were a writer (Michael Valeur), a designer (Isac Craaford), and myself in the role of facilitator and director. Michael and Isac had never met each other and, like with the artists I had interviewed before, I asked them to present their personal artistic methods to each other (from idea to product). These methods then formed the set of “tools” we would work with as a team throughout the workshop.

On the first day, we walked around the area of Copenhagen where the Borgergade neighborhood used to be, and later went to the Workers’ Museum where we studied archival material about Borgergade and the lives of its impoverished inhabitants.

The second day was divided into two phases. First, we spent two hours brainstorming by asking ourselves five simple questions:

The brainstorm took roughly 2 hours, and the method we used was to draw a five-pointed star, and then write each question at one of the star’s points. This “star model” is inspired by a method developed by the German scriptwriter and world builder, Jörg Ihle.

After brainstorming, we spent another two hours developing contributions to the Storyworld outline. In this phase we split up, each outlining a different part of the Borgergade universe. We decided that each one of us could decide for himself how and in what shape he wanted to make a contribution to the Storyworld outline. As we did not have a format to define which specific elements a Storyworld outline should have, we hoped that, if we each had the freedom to choose our own method, we might organically generate ideas for how a Storyworld outline could be structured.

Michael, the author, chose to write.

Isac, the designer, chose to make illustrations.

I (the director) chose to make an overview of the history, time, and events in the street.

We gathered the brainstorm notes, and our individual contributions, together into one document. And we discovered that, even though we each tried to create our contributions in a format-agnostic manner, we had actually carried our own formats directly over into the Storyworld outline.

For example, although Michael was known for his work writing both theater and games, his text was distinctly literary and gave the impression that he was, above all, a literary writer.

Likewise, Isac’s concept drawings had qualities betraying the fact that, even though he had done production design for both films and other linear formats, he is primarily a game artist. This was most visible in the way he worked with light and light particles, which is a distinct strength when working with real-time computer generated light in game technology.

For my own part, it was clear from the overview I created that I am, first and foremost, a director, and that the structure and dramaturgy of the narrative and the universe are a central part of my own practice.

Creating a Storyworld

Working with the Borgergade Storyworld outline gave me an answer to how I could develop a Storyworld by using the personal artistic pipeline and tools of my fellow creatives. But it also hinted at something deeper: that each format can contribute its own unique elements to a Storyworld.

“Ideally, each medium makes its own unique contribution to the unfolding of the story” (Henry Jenkins, ‘Transmedia 202: Further Reflections’, 2011)

In my research, I repeatedly stumbled over the researcher and journalist, Henry Jenkins, who invented the term “transmedia storytelling”. On his blog I read an article about how he believes that artistic formats are just as much a product of the universes they contain, as the universes are a product of the formats.

Working on the Borgergade Storyworld and studying Henry Jenkins inspired me. I had the idea that instead of using the Borgergade Storyworld document as an outline, I could use format outlines in the idea-outline-change loop I was trying to recreate.

With the idea of being able to test and develop a universe by developing format outlines, I hired some experienced artists to develop formats based on the Borgergade Storyworld outline. To ensure that it really was only outlines that I asked them to make, I deliberately allocated limited funds and time for the artists to develop their ideas (2 working days per outline).

In order to represent a wide range of formats, I choose one each of a linear, an interactive, and a serial format: radio drama, board games, and comic/graphic novel.

For the audio drama, I hired the award-winning director Carsten Rudolf. For the board game I hired the talented, upcoming Plotmaker Games team, and for the comic I hired the prominent cartoonist Peter Snejbjerg.

The results were as follows:

Radio drama: A coming of age story about a beautiful prostitute’s son. Written as an outline that is a mix between a cinematic treatment and a step outline.

Comic: Three visual narratives of citizen relationships across time. Outline made as a layout of individual page concepts for comics.

Boardgame: A game with the title Kongen af Borgergade (The King of Citizen Street), in which the goal is to win influence and power by controlling henchmen, prostitutes, and pickpockets. The outline is a three-page layout that describes the characters, rules, and board.

I was very content with the artistic level of the work, and in order to find out how I could use the format outlines as inspiration for the Storyworld-format-change loop, I invited the artists for a two-hour conversation organized as follows:

The conversations were recorded and I have summarized them below:

Show and tell:

The artists showed, and talked about, their ideas and how they had worked with those ideas. All three expressed that the Storyworld concept had made it easy for them to work, and that they had found it inspiring. They had each felt that it was hard to stop working on their projects and had all worked a little more than required. The artists and I were surprised to see how much the format outlines aligned with one another.

Transmedial inspiration:

Universe contributions that the artists felt were missing from my own Storyworld outline:

On a final note, my own experience of the talk was a big surprise, as all the formats developed by the artists seemed to fit the universe I had envisioned. None of the three format outlines seemed to be “off topic” or out of direction. All contributors agreed that the three format ideas would fit very well together, and the artists stated that they would want to continue working on the Borgergade projects if they were to be made for real.

Documenting a Storyworld

Encouraged by the results and the artists’ response, I began to wonder how I would proceed with the project. In particular, I wondered how I could actually document, describe, and keep track of all the many changes, inputs, and format ideas that would be developed as work progressed on the broader Borgergade universe. Going back to the smaller formats, there are several ways that a format and its universe may be described and documented. Examples include:

In my own practice I have worked with all of the above types of documentation. And even though you can combine them, or take a similar approach to documenting a world as Tolkien did, it very easily becomes a cacophony of materials.

During the trip to Los Angeles in 2018, Prof. Jakob Wille and I asked Alex McDowell about a format for documenting a Storyworld. McDowell had just finished working on a massive world-building project for the Star Wars franchise. When I asked him about a format for creating a “world bible” his answer was pretty simple: “Why don’t you just build it in a game engine?” As a game veteran, the idea of creating a Storyworld in real-time technology seemed so obvious that it made me feel like I had “not seen the forest for the trees”.

Alex McDowell was using a method he called D-viz (design visualization), in which he would first build the physical part of the Storyworld in a game engine (landscapes, houses, etc.) and then, together with the film crew, go on a location recce using VR goggles in order to create the story. Alex elaborated: “It’s a very theatrical approach. But it does the same thing as a story room in animation and maybe as a rehearsal does with acting and the director. And it is very close to prototyping for games.”.

Phase 2 Summary:

In the second phase I asked the question: “How can I develop a Storyworld?”. I discovered that:

Storyworld concept: Through the process of creating a concept for the Borgergade universe, I realised that the artists involved, and the tools and skills they bring into the table, are actually an incredibly important part of creating a Storyworld.

Format outlines: By having skilled artists create format outlines, I found a cheap and effective method for using the strengths of a format to strengthen the Storyworld through the use of a world-outline-change loop.

Storyworld documentation: Through the process of exploring various documentation formats, and through my talk with Alex McDowell, I became inspired to create/document the Borgergade universe digitally using a game engine. But this idea poses the next question: What is a digital Storyworld?

EXERCISE 2

I have created and used the following exercise with my students. The aim is that you (a group) can, in one day, create an original Storyworld and then use format outlines to strengthen that Storyworld.

Cross format/competence group (one day):

1: Each participant writes down their own artistic pipeline (from idea to product) in five to seven bullets on an A3 paper (10 min). The group then does a “show and tell” of what they have written, in order to get an overview of which tools they have combined between them. (40 min)

2: Each participant writes down a personal taboo, or something they have never talked to anyone else about (10 min). Show and tell of taboos. Then, the group chooses one taboo that they feel is important and could be used for creating a Storyworld (40 min).

3: Brainstorm on where, who, when, what, and why? Use five, A3 sheets of paper and draw:

4: Reflect on the potential for creating three formats (for instance a game, a play/film, and a novel) and develop concrete ideas/outlines for these (50 min).

5: Reflect on what these formats can bring to the Storyworld and then do a second iteration of the main Storyworld concept, this time incorporating the strengths of the formats you’ve just developed (50 min).

PHASE 3: How can I create a realtime digital Storyworld?

Phase 3 is a search for how I can create Storyworlds digitally, by building an actual Storyworld using real-time technology such as motion tracking, game engines, and VR.

In this section I experimented with technology such as game engines, motion tracking, and VR to create the Borgergade Storyworld

The ‘game engine’ allows me to build the scenography in 3D and move around in it. In the case of Borgergade, that would be the buildings, the street, the backyards, and the props.

‘Motion tracking’ is a technology one can use to record movements. For Borgergade, this was primarily used to record the movements of actors’ bodies and facial expressions. Motion tracking can also be used to track camera movements in cinematic sequences.

‘Immersive media’ is a technology that lets you experience vision and sound in a three dimensional virtual space. Besides visuals and sound, you can also interact with and control the point of view or camera with, for instance, a VR headset, VR trackers, or a mouse.

The explorations I performed were:

Hackathon: A two day workshop where a group of artists and myself, together with technology people, created a small rough scene/glimpse into a digital version of the Borgergade Storyworld. The aim was to find [tools]{.underline} useful for creating digital Storyworlds.

Model: Based on the hackathon, I created a model process for developing digital Storyworlds. The aim was that this model could be used by other artists and myself, including students.

Prototype: The hackathon was to give me an indication of how I could actually craft the Borgergade Storyworld digitally - or at least prototype a slice of the street. The model would give me a method to work from. The prototype itself contained 90% of the features/functionality of the digital Storyworld, and had to have a fully realized [artistic]{.underline} look and feel in order to be a valid proof of concept for the model and method.

Hacking a digital sketch

In game development, an exploratory workshop is often called a hackathon. A hackathon is a session where, instead of creating a game with finalized clean code, you make everything quick and dirty (“hack”) with the sole purpose of finding answers to questions and getting something to work on screen as quickly as possible. You could say that a hackathon can give you a glimpse of the game you are about to make a long time in advance of the actual premiere. More importantly, a hackathon can help you decide which technical and artistic tools will be most useful in a full production.

In order to find the right tools for creating a digital Storyworld, I used methods and ideas in the hackathon that I had developed in phase 1 and 2 of the artistic research project:

Team: To make the virtual segment of Borgergade, I invited three actors, one sound and music composer, a production designer, and three technical directors/producers. I was part of the team as well having the tole as a director.

Technology: For hardware, I booked a tv studio, computers, sound equipment, and motion capture equipment (used to record body and face movement). For software, we used a random assortment of game and film software packages such as the Unreal Game engine, Ikinema motion capture software, and the Pro Tools sound software.

Goal: The idea of the hackathon was that we would work together to develop a small glimpse into the digital version of Borgergade, working with acting, story, music, sound, light, and a virtual set… and just see how far we could get.

The work flow I planned to use was a combination of rapid prototyping, as is used in game development, and improvisation, as employed by actors in film and theater.

Rapid prototyping (or “agile development”) is a way to develop software, where you know more or less what phase you want to go through and how short a time it should take (“sprints”). But you don’t know the outcome of the creative work. The team can work either together, as a large group, or might split up into small teams (“SWAT teams”). After each sprint, the goals are re-defined and the teams re-group. At the end of the process the director, together with the team, decides which of the results will actually go into the final production.

Improvisation: As with rapid prototyping, the improvisation is organised into rehearsals (similar to sprints) where you do not know the outcome of the creative work during the majority of the process. And at the end, again, the director and/or the writer decides what material makes the most sense to carry forward.

In the Borgergade hackathon I planned four sprints:

Sprint 1: Inspired by the improvisation methods used by the British film director Mike Leigh, in which he improvises and creates scenes based on the actors’ own experiences (e.g. in Life is Sweet, 1990), I asked all of the team members to take 15 min to write down a real-life memory or story they had experienced or heard. The memory or story should be inspired by the central dilemma of Borgergade (trust vs poverty). After this, the team was introduced to the Borgergade Storyworld, as described in the 6-page Storyworld-document. We then discussed which story or memory we wanted to use as the foundation on which to create a glimpse into the life of the Borgergade Storyworld.

Sprint 2: After we found the story, which dealt with an assault, and chose a backyard as the setting, we split up and started working in SWAT teams:

All of this was done simultaneously.

Sprint 3: In the third sprint we worked in smaller SWAT teams, in short loops, to create digital characters, a digital set, sound, music, and light. In this sprint, the fact that we were working on all the different aspects simultaneously meant that the artistic creations (music, acting, scenography) started to inspire each other across the floor. Because we were building the scenography and rehearsing the acting at the same time, we could move walls or change movements in real-time, in order to emphasise the feeling and tone of the scene accordingly.

Sprint 4: In the last sprint we recorded the sound and movement of the acting using traditional sound recording and motion tracking technology (face and body animations). We did not have the technology to see the 3D characters modelled in real-time with the acting, but this would have been possible with a more technically advanced setup. We could, however, get a feeling of the space by using tables and by putting tape on the floor to show specific objects (like walls and doors) that existed in the ‘backyard’ virtual set.

After working one day with the actors, and one day to gather the material, the main results from the hackathon were:

We did have some technical issues, which made the work feel less than real-time at some points. But during the hackathon the actors noticed that the process reminded them of a well-established theater method called “devising”. In devising, similar to the working process of a game development team, a theatre ensemble develops a performance by creating small segments, in a non-linear fashion, using improvisation, while working in rehearsals structured similarly to our “sprints”.

As the actor Alex Lehman noted in the debrief, “Really nice to bring the theater devising techniques into a digital production context. Devised theater is about the actor being empowered as part of the creative process, which isn’t usually the case for digital work.”

But what surprised me the most was that the flow of iterations between theater and game techniques all but eliminated the need to outline anything, as we were able to see the material that we created, and any changes, instantly in real-time.

A model for digital Storyworlds?

In the hackathon, I used gaming methodology and tools to hack a scene in a digital Storyworld. By having the actors onboard, I learned about theater devising. I was surprised to see how well game methodology and theater devising fit together. In Danish, you might say that game and theater development “passer som hånd i handske og fod i hose” (“it fits like a hand in a glove and a foot in a shoe”).

This relationship was easy to fall in love with. And then I remembered that Alex McDowell, when I visited him in 2018, also mentioned that he had experienced a similar relationship between developing theater and games and worldbuilding. So in order to get the theater aspect into a model for creating digital Storyworlds, I invited the theater scenographer Siggi Oli Palmasson (who participated in the artist interviews in phase 1 of this project) to help me formulate a model.

The two of us could seem like a bit of an odd couple, as I am a complete theater ignorant and Siggi is a complete digital ignorant. But to us the lack of knowledge and ‘virginity’ only made it more fun to show and tell each other about our respective fields. It actually made it easier for us to find common ground.

In order to sketch out a model, we arranged three sessions in early 2020 that took place all over Copenhagen: on Siggi’s houseboat, at Aalborg University’s digital Vizart Lab, in Siggi’s workshop, at the National Filmschool of Denmark, and at the Danish National School of Performing Arts where Siggi was teaching at the time.

Session 1 - Game: In the space of three hours, I introduced Siggi to game development and digital worldbuilding, using examples from both my own work and that of others, with the help of technicians from Aalborg university. Siggi learned how game engines and motion tracking work. During this introduction we noted that many of the interface elements/windows in the game engine Unreal used terminology from theater, i.e. one window is called “the stage,” another “the cast,”, another “props,” and so forth. We were surprised by how easily we communicated and how much Siggi actually felt at home with the way that the game engine and methodology worked. We shared language and ways of working.

Session 2 - Theater: In the space of three hours, Siggi then introduced me to theater creation and devising through his own work. We went through some of the plays that Siggi has worked on. He talked to me about the devising process, and it was then that I began to realize just how close it really is to the iterative processes and rapid development of games. Before this I thought theater was only made in a very different, and more linear, step by step fashion similar to film (synopsis, script, shooting draft, recordings, post, etc). It is definitely true that, when working from a concrete text, the theater creation process more closely resembles that of standard film-making. But when devising plays, which Siggi has done for more than 25 of the 50 productions he has worked on, it is iterative and very much like developing a game.

Session 3 - Fusion: Using the knowledge gained from both the prototyping phase, and the two knowledge sharing sessions, Siggi and I continued to meet over the course of a few weeks at both Den Danske Scenekunstskole and Den Danske Filmskole (two of the Danish theater and film schools), which happen to be neighbors. We started to brainstorm how we could create a model for devising worlds which could encapsulate all of the best experiences that we had had. While working on the model we became aware that, even though my focus is digital in this project, a world devising model could just as easily be used to create analogue Storyworlds.

The model we have created consists of three steps:

WORLD: A process of drafting the world by using a star diagram to answer the five basic questions: who, when, where, what, and why?

STORY: A process of creating ‘dots’ as story scenes/glimpses/tableaux into the Storyworld by using improvisation.

FORMAT: A process of linking those dots in order to create existing linear (rectangle) and interactive (triangle) formats, as well as new formats about which we do not yet know (circle).

We baptized this model, UNIVERSE DEVISING.

Inspired by our model I decided to do the prototyping of the Borgergade Storyworld by combining 3D worldbuilding from games and devising from theater. I invited Siggi to help me test the model by making a prototype of the Borgergade Storyworld.

Prototyping a Storyworld

In game development, prototyping is used both to test the individual features, as well as to test the full game itself. This is often called a “vertical slice” and is the first part of an actual game production. This often comes after the game reaches roughly ⅓ completion. The vertical slice is a section of the game (which may be only a few minutes of play) which contains almost all features (functionality) of a game. This means that the vertical slice gives you an experience of the final artistic look and feel of the game. But it also gives you a lot of answers for the continued production of the game, both in regards to scope (size and workload) as well as the technical and artistic challenges/solutions you are likely to face in the process of completing the full game.

In order to test the UNIVERSE DEVISING model, I decided to create a 3D vertical slice of the Storyworld and explore how this could be used to create formats:

THE WORLD: Based on the collaboratively created Storyworld outline for Borgergade (created in phase 2), I started researching materials and tools for creating a digital version. Ten years ago, creating the digital world of Borgergade would have required a large team. But today, the democratization and technical development of real-time technology in certain game engines can actually turn you into a one-man worldbuilding army. Even a director like me, who has no programming skills and only mediocre technical skills. And moreover, the technology today is free to use, making it possible for even school children to become worldbuilders using top range professional game engines such as Epic Games’ Unreal Engine and the Danish game engine Unity.

Once I started developing the look and feel of the Borgergade world, I employed the pretty traditional method of studying the real historical Borgergade for my inspiration (incl. the original research done in the 6-page world outline created by me, Michael Valeur, and Isac Craaford). From this material I made a database of reference images and selected a few to build the world from.

To find the artistic tone I looked into other artists’ works, in particular the movie Delicatessen (1991, by Jean-Pierre Jeunet and Marc Caro) and the comic series Dropsie Avenue (1995, by Will Eisner). Both have a grotesque (gruesome and funny) tone that I was looking for - Delicatessen in it’s set and character designs, and Dropsie Avenue in both it’s story and setting.

Building a section of the scenography for the world, I decided to do a part of the main street and a series of backyards. During the hackathon we only had two days to build the scenography, so we only built one backyard using a method called kit-bashing. In kit-bashing, you buy or find 3D models on the internet and stick them together in a game engine. It is almost like playing with LEGO. In a professional game or 3D-animation production you would have a whole team of modelers to build your models from the ground up. But with the internet and digital distribution, more and more high quality 3D assets can be purchased online for a fraction of the price of labor. Today you see more and more 3D worlds for both film and games built with a combination of purchased 3D objects and hero props (selfmade 3D objects).

In my case, I bought a kit called ‘Nordic Harbour’ for 15 dollars. I used the "LEGO brick’’ buildings from this kit to reconstruct a section of the real Borgergade from the old black and white photos. And then I customized it by removing colors and textures from the kit models to create a more graphic look similar to Dropsie Avenue. I then added advanced real-time lighting and post effects (computer generated visual effects) to generate a cinematic look inspired by Delicatessen. These advanced effects can be added with the click of a mouse in Unreal.

Creating stories in the Borgergade world also requires characters. To achieve this, I employed the same method I had used to create the scenography. I took archival historical references, and artistic references from Delicatessen and Dropsie Avenue, and once again employed kit-bashing - this time finding and buying character models on the internet and customizing them. There are several options to buy and customize characters, such as Adobe Fusion, Reallusion Character Creator and more. Creating a character in this type of software lets you change everything: from physical features of the body and face, to clothes and the graphic appearance of the look. All with the click of a button. Again, today this is so easy that any child (or adult) who has played the virtual doll house game, The Sims (2001 by Will Wright), will be able to create their own characters for creating Storyworlds.

THE STORY: Siggi and I invited two artists for a two-day story jam session workshop. The idea of the story jam was to devise a series of story scenes by using the 3D-scenography and characters. This workshop occurred in May 2020, and the artists we invited were Michael Valeur (writer and participant in creating the first outline for the Borgergade Storyworld in phase 2), and Zoe Nguyen Tan (an experienced animation and game art director).

Inspired by the result we had achieved by combining devising techniques, rapid prototyping, and kitbashing, we created a workflow in which we would:

1: Let Michael and Zoe work together for a full day - kitbashing, choosing, and creating 3D assets: 3 objects, 3 characters, 3 locations

2: All four of us work collaboratively for one day using the chosen assets in three story sprints, improvising three story scenes in Borgergade.

3: Document the scenes directly in the engine as staged story scenes.

From a technical standpoint, the prototype was made by working on one laptop computer linked to a big screen so that everybody could see every change immediately as it was implemented in the 3D scenography and story.

After the two days, we had created three story scenes that took place in the Storyworld. The three story scenes we created were:

A: ‘King of the Yard’

Start: A man wants to enter a backyard with his wagon, but the drunk smith (the titular “king of the backyard”) has blocked it with a bench. The man gives the smith a bottle of schnapps to move the bench.

Middle: The smith doesn’t know where to put the bench and places it in front of the door to a latrine in the backyard. A very angry lady is now stuck in the latrine. The smith first tries to get a bottle from her as well, but quickly removes the bench when the lady starts swearing.

End: The smith, now standing in the middle of the yard with the bench in his arms, gets an idea. He walks to the other end of the yard and places the bench in a port blocking the way for the man and his wagon to get OUT of the backyard again. The smith now waits here for the reward.

B: ‘The Dog and the Swindler’

Start: A mother is talking to a man. Next to her are her daughter and a dog. The mother and daughter are moving away from Borgergade because they can’t afford to live in the city. The girl talks to the little dog and tells the dog that the man has promised to take good care of it.

Middle: The mother tries to make sure that the man will take good care of the dog. The man charges 10 kroner, which is a fortune. The woman tells him she can’t afford it. He laughs and tells her he is just joking.“Promise!”

End: The woman asks, if there is anything else she can do. He gives here an up and down stare. She steps back and tells him that she is not that kind of a girl. He laughs and tells her he is just joking. And that he will take good care of the dog. Everybody loves him. Dogs, children, women - everybody. “Promise,” he says!

C: ‘The Fire Lady’

Start: A mad woman, wearing strangely extravagant clothing and a tall man’s hat, is angry at a house. She rambles and we see that there is a chimney with smoke, sparks, and a little fire coming out. We hear her say: “… it’s eating everything… The hungry fire… you take care. You take care. It’s never satisfied, no, no, never satisfied. Remember it? … Smoke… you can´t breathe…”

Middle: A group of girls stand close to her, watching. They swing their arms back and forth rhythmically in an eerie child-like choreography while singing:

My mother giving her the hat

Do you really need that? Do you really need that?

Fire lady Fire lady, do you really need that?

My father giving her the dress

If she’d just confess. If she’d just confess.

Fire lady Fire lady, will you just confess?

My sister giving her a shirt

Keeping warm could never hurt! Keeping warm could never hurt

Fire lady Fire lady, keeping warm could never hurt!

But no one gives the baby back

Always feeling very sad! Always feeling very sad!

Fire lady fire lady, always feeling very sad!

End: A boy stands a little further away, not singing. He just watches. He takes a bucket of water and throws it after the girls, who run laughingly away down Borgergade. The Fire Lady stops babbling and walks over to the little boy taking his hand. “Let’s go home,” she says.

Besides the song, which Michael Valeur wrote for ‘The Fire Lady’, we did not write a text/script. Over the course of the two days, we implemented the scenes directly in the digital Storyworld using the scenography, characters, posed animations, and even some pre-made sound. The result could be compared with a movie storyboard. But it was made and staged in a 3D space, with special effects like dust, computer generated light, animated characters, and even 3D sound, making it less abstract than working with just a text.

After the two days we were on a high from the work that we had done together. First of all, the artistic level of the scenes felt both original, and of an artistic level that we were very happy with. Secondly, we had eliminated the need for a text-based script or printed images, and as such the digital Storyworld became the “legendarium” containing all of our ideas.

But the biggest advantage in working in this way was that we could all immediately see what we had created, and could be inspired by the real-time creation process, which made room for “happy accidents” and ideas that we would never have had if working only from a theoretical framework or from words on paper.

During the debrief, Zoe Nguyen noted: “Seeing it in the engine while we did it, it felt more natural than when reading about the rules and concept on paper.” Michael Valeur added: “It boosted the creativity everytime we let a mistake become a virtue - a solution.”

THE FORMAT: Each of us worked pretty seamlessly in regards to formats during the story jam, and neither I, nor Siggi, nor Michael, ever once mentioned our native game, theater or literature format. The only one aware of formats was Zoe. Zoe is a French art director and, besides working on animated films and games, she has also worked on major VR productions. While we were creating the story scenes she was very aware of where the camera should be staged and which type of point of view (POV) that would work best for each scene.

When we started working, I had instructed the story team to work with a fixed 360-camera in what I called a 360-tableaux. This meant that the camera was fixed to a spot, but could be adjusted/moved by the viewer in 360° by using a mouse, or when watching the scene through VR goggles. But with Zoe’s artistic suggestions, we quickly started using the camera dramatically and artistically for each story instead:

A: ‘King of the Yard’

POV: Staged Camera

King of the yard plays out in a limited space. And even though we could place the camera in a suitable spot within the yard where one could watch the entire story scene, it is not ideal for creating the potential comedy of the smith moving the bench around. A better solution for the camera might be to do it as a traditional cinematic shoot, with edits, or to stage it as a cinematic game camera with some freedom to move and look around. This type of camera is starting to be used in very cinematic games like God of War (2005, Sony). In God of War, the camera moves from intimate close-ups to grand wide shots, and changes focus lengths depending on the action happening in the story and game. The entire game, which takes more than 20 hours to complete, is staged as one single, graceful camera shot.

B: ‘The Dog and the Swindler’

POV: 360/VR Camera

In this story scene we decided to place the fixed 360-camera at the little dog’s POV, meaning that you never see the dog. On the one side you are looking up at the little girl who reassures you, the dog, that you will be safe with the swindler. Turning the camera to the other side, you will see, and hear, the mother trying to make sure that the swindler will take care of you. At all times, the girl has eye contact with you and the effect is very powerful. But you will also notice that the mother is never looking at you (the dog), but only at her daughter, which indicates her motivation for talking to the swindler. The swindler, you will notice, is looking only at the mother - in particular when staring lustfully at her legs.

C: ‘The Firelady’

POV: Theater Camera

For ‘The Firelady’, we placed a fixed 360-camera a bit away from the scene, almost at the same distance as the audience in a small theater. The little boy is closest and facing the camera and the action of the scene occurs behind him, as if on a stage. In the beginning of the story you don’t know who this little boy is, but it starts to become clearer as the scene progresses. At the end, as you begin to realize the unbearable role of being the son of the Firelady, the two move away from you down Borgergade, leaving you in an apathetic state.

As Zoe pointed out, each scene POV points to a specific format. Maybe the camera itself is the key to figuring out which format best suits each dot/scene. One could claim that the scene with the smith has both a game or a cinematic potential. ‘The Dog and the Swindler’ has VR potential. And perhaps ‘The Firelady’ would work best as a theatrical experience?

Phase 3 Summary:

In the third phase I asked the question: “How can I create a digital Storyworld?” And the resulting exploration lead me to:

Hackathon: By gathering a group of artists and technicians we created a truly collaborative process, creating artistic digital work by using a game engine and motion tracking.

Model: By exchanging artistic experience and knowledge between games and theater, we created the UNIVERSE DEVISING model - a Storyworld development method that can be used for both analogue and digital world building.

Prototype: By creating a vertical slice of the digital Borgergade Storyworld, we tested the UNIVERSE DEVISING model and experienced how theater devising and game technology/methodology together have the potential to lift the artistic quality of digital Storyworlds. The truly real-time “what you see is what you get” nature of this collaborative creative process allowed for happy accidents to occur, and for a rapid idea-scene-change loop that eliminated the need for a text script or outline. The process also indicated that the camera POV could be relevant for the further development of formats from the Storyworld.

EXERCISE 3

Siggi and I have created the following exercise for our theater scenography and film-directing students. Due to the Corona-virus situation we have not yet been able to put it into practice. But the aim is that a group of cross disciplinary artists, can, in a week, create an original digital 3D realtime Storyworld and use POV to create format ideas.

Single person or group (one week):

1: Create a Storyworld outline answering the five questions: who, where, when, what and why? (Half a day)

2: Challenge each point (make it more extreme) by finding both the least obvious option, and the most obvious option, for each question. Then revise your Storyworld outline accordingly. (Half a day)

3: Find 3D-assets on the internet (3 pieces of scenography, 3 sets of props and 3 sets of characters). Don’t think about how they fit together, as long as you think they will fit within your world. (Two days)

4: Create tableaux scenes (connecting the dots) with the assets you have found. Try to mix them up by rolling a die and seeing what it gives you, in order to avoid choosing the obvious combinations. Look out for happy accidents, but don’t expect those accidents to solve your problems. (One day)

5: Create ideas for one format to be published on paper (for example, a book or board game), one format as a screen publication (film, tv, app, game, etc.), and one live format (theater, performance, exibition, etc.). Consider the point of view (camera) for each scene and what this says about how each of your scenes should be experienced. Reflect on what this tells you about the potential formats.

Epilogue

The Storyworld 2.0 project was funded by the National Filmschool of Denmark and the Ministry of Culture. The research project The Worldbuilding Club was funded by Nordisk Film Fonden. Both projects were pre-projects, which is why the Storyworld 2.0 article stops before entering the actual Storyworld-to-format development process. In early 2020, both projects were granted additional funding to focus on the next steps.

The new project that follows the Storyworld 2.0 project is named Devising Worlds, and it will consist of a close collaboration between myself and Siggi Oli Palmason. Our primary aim is to instrumentalize the UNIVERSE DEVISING model by developing a new, original, devised Storyworld and then bringing it to the publishers of various formats (e.g. book publishers, TV stations, theaters, etc). Our hope is that, by taking it all the way to format publishers, the UNIVERSE DEVISING model and Storyworld thinking will come closer to actually being implemented in the professional world, and thereby raise new questions.

This project will consist of the following phases:

Phase 1: Research theater devising and game world building in order to expand and consolidate the UNIVERSE DEVISING model, and to engage in practice-based tests of the model with students in order to propose a concrete, collaborative artistic pipeline.

Phase 2: Prototype this pipeline by creating a new original Storyworld digitally in its entirety (and not just a vertical slice like Borgergade).

Phase 3: Have major publishers, representing three different formats (tv, theater, and books), put forth actual proposals for realistic format productions taking place within our Storyworld, which would be financed as well as published by them.

If we achieve these goals, however, it will lead to new questions, which I did not consider in the original Storyworld 2.0 project. These include:

These are some of the questions that we hope to touch upon in the Devising World project, which is scheduled for completion in 2022.

The successor to the Worldbuilding Club project is named WorldSum. The aim of this project is to further explore how artists around the world, and across formats, are creating Storyworlds. The aim is to create and publish a book about the subject, with practical advice for the reader and interviews with some of the most prominent artists around the world.

The project is organized in three research phases:

1: Japan: Focusing on Storyworlds in games and animation.

2: USA: Focusing on Storyworlds in film and television.

3: Europe: Focusing on Storyworlds in books and theater.

The project will run throughout 2020-2022. The aim is to publish the book in 2022.

Recommended reading:

Building Imaginary Worlds: The Theory and History of Subcreation (Mark J.P. Wolf)

Storyworlds Across Media: Towards a Media Conscience Narratology (Marie-Laure Ryan)

Convergence Culture: Where Old and New Media Collide (Henry Jenkins)

Download as pdf