Module 4
From aural sketches to spatiotemporal design in VR
The last part of the workshop was devoted to the collective production of a VR experience, as a spatial-visual dimension of the spatial-audio composition 'grrawe' that was previously examined.
First, a shared waveform timeline (right) was established, which was negotiated between participants and divided into distinct segments/chapters (eventually nine), according to and based on the individual sketches and impressions produced in the previous module during blindfolded listening. In the mode of working together, the segments were distributed to the participants, with each being responsible for one or two unique segments, and the resulting collaborative timeline consisted of a sequence of subjective, aurally induced spatial entities.
Subsequently, after a short introduction to Unity 3D (a real-time game engine and editor), the participants began the process of translating their segments into virtual spatiotemporal entities. While earlier work was the foundation for this task, notes and sketches were only helpful to some extent. Because many aspects of these environments were elusive to 'cooler' media, further elaboration was necessary in order to describe them more thoroughly and according to the definition afforded by the spatial and also temporal medium of the just-introduced VR. This was achieved through additional listening and sketching, as well as through explorations of the software and what was possible to design with it. The process continued iteratively through design and testing, initially on individual laptops and later using the VR station installed at the workspace (a stationary computer with a VR headset and a tracking area of 4x4 m2).
It needs to be emphasized that these translations were not limited to static, visual 'pictorial' environments, but also included notions of time as well. These aspects emerged naturally in the process of subjectively interpreting the evolution of the aural architecture through the temporal dimension. The three distinct approaches into the temporal dimension observed were: (a) movement and animation of scene objects and the environment, which was observed in most scenes; (b) animation of the viewer-listener's position in virtual space, or 'artificial locomotion', as in scenes 4, 7 and 9; and (c) scripted interaction of environmental objects, with regard to the viewer-listener's position in virtual space, as in scene 6.
Gradually, by the end of workshop day 5, all scenes were finalized, collected, and consolidated, with the help of tutors, into a single project adapted and fine-tuned for VR. The resulting product was a collaborative VR experience of equal time duration to the audio piece, consisting of the nine sequential scenes that the seven participants contributed.
In the next section, a non-VR version of the experience can be downloaded.
Top: Scan of the working document of the shared waveform timeline of 'grrawe' as segmented, numbered, and color coded by the participants. Bottom: Digitized version of the previous – both instances were from before section 5 was consolidated into scene 4. Beyond the Visual, OSSA 2018, Łódź.
Middle: grrawe (10'37" binaural rendering, Gerriet K. Sharma, 2010).
Audio timeline and waveform images were loosely aligned to allow audio-visual comparison. Listening with headphones is advised.
