-----------WORK IN PROGRESS------WORK IN PROGRESS------
-----------WORK IN PROGRESS------WORK IN PROGRESS------
SHADOW OF WAR
VR-Ethics and storytelling in the spectrum of representation, immersion and agency
ARTISTIC RESERACH PROJECT: INSIDE THE NARRATIVE
Authors: Gustav Kvaal, Torkell Bernsen, Simon Gilbertson
Artistic work/ SHADOW OF WAR/ I SKYGGEN AV KRIG (NO)
Torkell Bernsen – Visual storytelling, Animation, Compositing
Gustav Kvaal – Visual storytelling, Illustration, Animation
Odd Torleiv Furnes – Music and VR-Sound
Rafal Hanzl – Unreal Engine facilitator and Realtime Effects
See and experience SHADOW OF WAR at SIDEQUEST:
https://sidequestvr.com/app/34157/shadow-of-war
SoW Trailer on Youtube:
https://www.youtube.com/watch?v=7wl06l_-lWw
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
ABSTRACT:
The aim of this artistic research project is to create an illustrated VR documentary experience that narrates the story of a time witness from the second world war in Bodø, Norway. The project explores questions concerning ethics and visual storytelling in the encounter between the VR-audience, interviewed subjects and the audiovisual spatial design. Artistic and qualitative research methods have been employed to explore how different visual modes and contexts alter the experience of the narrator and narrative in a media form characterized by its ability to place the viewer in a state of immersion, intimacy, and a sense of presence. Theoretically, this study is situated in an artistic landscape connected to storytelling, media theory, journalism and ethics. Concepts such as media witness ethics, with the so-called risk of improper distance are relevant for the reflection associated with the project. The study proposes a view on ethics towards VR- storytelling that is depending on its position in a spectrum of different types of VR-experiences, from the totally immersive and interactive VR-simulations to VR-experiences that is more into the direction of cinematic screen based experiences. The Shadow of War VR-experience represents the latter and suggest a mindset to designing for virtual reality that is more looking to be utilizing opportunities in the technology rather than trying to establish a recipy for VR-storytelling this early in the ongoing history of VR-technology experiences.
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
CONTENTS:
ABSTRACT
INTRODUCTION
- project position
- Intentions and objectives
- an early VR-documentary concept prototype
- context and scope
- research question
BACKGROUND
- postphenomenology
- VR-experiences & ethics, metaverse VS representation( Murray)
- media theory / VR-documentary ( Chris Milk- Empathy machine,
Maud Ceuterick & Chris Ingraham
)
- VR-storytelling
METHODS
- Artistic Research methods and storytelling ethics
- comparative study / focus group interview/ user testing
- Building the experience & telling the story
- methodical progress & development - story versus ethics
- educational implementation / VR-stories by students
- theoretical development / Articulating a tool for ethical storytelling
FINDINGS AND DISCUSSION
- methodical findings
- educational findings / experiences
- theoretical findings / VR-ethics, in a spectrum of immersion and agency
About the integrity of the interviewed subject and how the story comes across to the VR-audience
INTRODUCTION:
The study presented in this exposition belongs to the artistic research project, Inside the Narrative. It derives its name and focus from the narrative point of view of the audience engaging in VR-experiences. The research project curiously joins in on the ongoing exploration of storytelling for immersive media and particularly virtual reality as a storytelling platform. What happens to storytelling when the audience is experiencing the story from within, immersed in the storys own world of audiovisual surroundings? Leaving behind the familiar rectangular framing of books and films, into simulated surroundings and interaction, new ways of guiding the audience through a story is needed. As the rules of storytelling are reorganized in a newer media format, attention to challenges concerning ethics comes along as well. Questions concerning VR-storytelling ethics is the focus in the further discussions.
The project Inside the Narrative has its academic position in the subject area of visual communication at the Faculty of Art, Music, and Design at the University of Bergen, Norway. The project's position within artistic fields such as illustration, animation and graphic design gives us a unique opportunity to help bring forth new perspectives on VR storytelling that may differ from insights gathered from, for example the computer games industry, film, medicine or engineering. These perspectives highlights communication through visual concepts, visual expressions, written language in combination with interactivity, moving images and sound. Working with VR-experinces within the field of visual communication allows for experimentation with new media experinces outside of the established standards of for example computer game or film experiences. This study is maybe paradoxially occupied by creating something that can be categorized as a type of film documentary, but the way the experince is constructed is equally engaging in modes of communication found in books or printed matter.
The VR-technology has been around for many decades but it is only in the last decade that theory concering storytelling has started to emerge through publications such as, The end of storytelling (Stephanie Riggs 2019) and Crafing stories for Virtual reality (Bosworth & Sarah 2019). This project belongs to this emerging field of research concerning immersive storytelling and in particular storytelling using virtual reality technology as a medium of communication. The Inside the Narrative project aim to provide both students and professionals within the field of illustration, graphic design a platform to start developing experiences, knowledge, skills and work methods towards designing for VR-experiences. As a part of this professionalism and knowledge is understanding the media technology and it effects on the audience and stakeholders in the production processes. In a recent artistic project called Shadow of War, a non-fiction VR-documentary became an opportunity to investigate the ethics of VR-storytelling.
VR-technology is known for creating a strong sense of presence for the users in virtual reality experiences (…). The feeling of presence also in non-fiction VR-experiences have amongst other aspects prompted the well known notion of VR as an “empathy machine” that say something about how VR as a medium of communication have a potential power that goes beyond former media viewing situations like watching the news on a screen. The feeling of presence seperates this medium from others and challenges debate on what producers of media content for VR-experiences should consider in the production process. This production process opens up for new possibilities of storytelling but also new challenges.
When approaching a discussion of storytelling ethics concerned with the experience of VR-mediaproduction it has been important for us in this study to to make a clear statement about our understanding of what type of human experience a VR-experience is at this point in time. The differrent views about how impactful, convincing or overwhelming a immersive VR-experience is will stear us in different directions on how to understand an ethical discussion. As a way to describe the different perspectives one might say that we are discussing wether or not a VR-experience makes us lose the distinction between what is real and what is virtual. A world of VR-experiences that will almost physically blur the lines of what is real and what is virtual will need very different measures in society in comparison to viewing VR-experiences as mostly a new extension to former audio visual screen technology. Nuancing this picture is important for the further discussion whwere still most of the current VR- experinces is still involving a user concious of the real world (Janet H. Murray 2020). As a way to move from traditional media formats of film and print the project is funded in a viewing of VR-experiences as a new arena for multiple types of experiences, not only the fully immersive, realistic real world simulations. We address the discussion about ethics and VR-technology in storytelling based on VR-experiences as a spectrum of various types of experiences that often might fit into the category of beeing a medium of representation. Still the feeling of presence in every VR experience will have to be taken seriously as a clear distinction from previous screen based mediums, also affecting the issues concerning ethics.
--------------------
Exploring virtual reality as a storytelling platform
The understanding of ways of telling storytelling in the VR-medium have grown clearer in the last decade. Media theoretical publications offer analytical overviews of the VR storyplex and key aspect of the storytellers palette and possibilities.
.......OM CRAFTING STORIES & END OF STORYTELLING HER...
It has become clearer that VR- experiences based on how it is created can end up in a variety of ways, from the simplest form of 360 immersive image and no functional interaction to a full real world highly interactive and immersive experince. When entering into the area of creating VR-experiences you can do this on a very complex and advanced way, but it can also be something close to publishing a video online. When discussing ethics concerning production of VR-experinces it might be relevant to distinguish between the various types of experinces and and understand what factors that decide wether you are working on a production that is exposed to a higher degree of ethical challenges or not.
...OM JANET MURRAY HER.........
RESEARCH QUESTION:
How can visual storytellers navigate ethical considerations in non-fiction VR-experiences?
This question is both challenging questions of how to carefully and ethically include partisipants into a documentary film type production, but it is also, at the same time exploring how the storytelling is affected, balanced and maintained.
PART 01
AN EARLY VR- PROTOTYPE: OFF-PISTE 2019
At an eight-day multidisciplinary VR storytelling workshop at Hafjell near Lillehammer in 2019, called Off-Piste Virtual Reality Storytelling Lab, the storytelling itself was put into focus. No Matter Where is the title of an experimental prototype of a VR documentary where you get to meet with various characters and their personal stories in a VR space. The project explores how to obtain, process and present documentary material in the VR format. With this prototype, we wanted to find out what it would be like to be face-to-face with someone who openly shared their life experiences? We meet three different characters within this VR installation. The thematic focus of what is being told is the various individuals' relationship to places and how these have had an impact on memories and their lives.
Wearing VR goggles with headphones in this prototype immerses the viewer into a misty and snowy landscape. The sound of a gentle breeze enhances the experience of the place's presence. The attention of the audience is directed towards a person sitting in the landscape, by means of an enticing sound. As you focus your attention on the person, the person stands up and begins to tell a story. While the person is talking, the audience can move around the space to look at the interview subject from different angles. You can also choose to look away. When you get close to the figure talking, you see in this prototype that the human figure is just a kind of 3-dimensional hollow shell that lacks the backside. This deficiency is related to the volumetric filming technique used to generate the 3-dimensional figure in the VR space. After about 3 minutes, the person ends the monologue and sits down in a waiting position. With the help of another enticing sound, the audience is led on to a new story told by another character in the same space. This time it's the sound of a radio we hear. When you turn around, another character in the landscape is appearing, which in this case is given a kitchen environment that is located in the middle of the misty landscape. As you approach, this person gets up and starts a monologue in the same manner as the first one. In this prototype, each of the stories and sounds that lead the attention is triggered by keystrokes from a technician in the room who observes the progress and course of action of the VR experience. The use of a technician in this project is a possible way to run an event course in a VR experience. The advantage of this solution is that the experience here is personalized to each viewer based on the viewer's body language and attention. This could also be automated to make production less resource-intensive and more easily divisible.
As an approach towards documentary content in this project, we mainly wanted to relate to the real world and photorealism as a starting point for the visual expression. Environmental descriptive elements were therefore gathered with photographic techniques which were later processed into 3D elements and assembled in the game engine called Unity. The snowy and misty landscape backdrop was created by a 360-degree panorama so that we could fill the whole world around the audience. This could just as well have been a 360 video recording if that had been desirable. To create the ground and the place where you stand and where the interview subjects are located, a technique called photogrammetry was used. This technique creates digital 3-dimensional objects by allowing software to analyze depth ratios in the images based on a plethora of photographs taken systematically of a place or object. This way, one can quickly capture and integrate photorealistic objects into a 3D program without necessarily being a 3D specialist.
To give the audience the opportunity to feel that they were in the same room as the interview subjects, we wanted them to be able to move with some freedom in the VR space. Therefore, this prototype does not use a classic 360 video or similar to create a VR environment. With 360 film you get a photographic visual expression, but without being able to move around in a 3-dimensional environment. One is locked to a point in the universe. Instead, the audience is offered an area of 16 m2 where they can move freely and look around both in front and behind the various objects in the space.
To film the interview subjects that we later encounter in the VR experience, we use a technique called volumetric capture. This is done by using a Kinect camera and software called Holo Cap that combines a video recording with 3D data read by an infrared camera. In this way, we could capture a 3-dimensional figure in the space and not only a regular flat 2D video image. It is only from a partially frontal perspective that it works visually from the point of view of the audience. Still, it is not flat as a video recording, so that the figure's body shape becomes visible in space, making it possible to integrate the figure with other 3-dimensional elements in the space.
The sound recordings used in the VR experience were organized digitally in different virtual places in the VR space in the same way as you would distribute 3D objects. In this way, you can organize sound in the game engine Unity (alternatively the game engine Unreal Engine) so that you experience the sound coming from different directions and at different distances.
Through working with the prototype, we engaged in processes and techniques that are largely about creating virtual places with different opportunities for experiences and actions. The audience's role in this particular VR experience was as a kind of mobile observer without so much opportunity to influence the environment. Still, the audience´s meeting with the interviewed subjects and their personal story inside the VR space seemed to be one of apparent emotional impact. The choice of controlling the experience by the use of a technician tells us that VR not only needs to be a lonely digital experience but can also be conceived as a type of performance where the experience is shaped through a feedback loop like in the theater. After working with this VR-prototype, the overall experience is that focusing on the quality of the story itself still remains important also from the inside of the narrative.
