Each process of artistic production can be subdivided into distinctive development phases. The most substantial amount of work and the biggest costs are often incurred by the last stage, but the beginning is the phase most open to artistic choices. Only during pre-production do we have enough freedom to fully decide about the visual aspects and the style of storytelling. The more progress in production is made, the more difficult and expensive later changes in style become. Making right creative and methodological decisions at the initial stage is exceptionally important with a limited budget. One of the chief tasks of the creative technologist is to keep the balance between the artistic goals and the available resources, and to inform the director if the production approaches the financial limit that can disturb the realization.
During my research, I mostly focused on the stage of pre-production as the most important part of my work in artistic realization. While in the virtual reality project I could follow the realization from the beginning of the work to the very end, in the documentary project I had to adjust to the ongoing production, and I left the team before the production was finalized. Although it was not good for artistic continuity and the proper implementation of all my development, the decisions I made laid the foundations for further production. It also helped me to isolate my experience in the artistic research as a creative technologist focused on the pre-production stage.
The green screen solution, finally abandoned, was carefully planned. I made storyboards, conceptual sketches indicating the placement of the camera and the crew, as well as full descriptions of the scenes. The director wanted to work in a more spontaneous way and to have more freedom on the set. The projection scenes were prepared on more general principles. I made some sketches of a possible technical solution, and a simplified version of this conception was used during the final production. My sketches were a spontaneous illustration of debates with the director and of and issues related to the realization. Shooting and performance relied on my concepts of automatic tracking, a space mouse or a tablet attached to the projector. The description tables from the greenscreen were adapted to the projection scenes. They were a conceptual basis for the realization.
It was impossible to make a storyboard for it because it was hard to predict how the protagonists will behave on set — rough concepts and a number of tests allowed us to work in an improvised way. Usually the result of a creative technologist’s work is more precise, but testing various solutions and experiments with form can also be applicable in different cases. A creative technologist in the first place has to adapt to any specific type of collaboration. Some people need to see the final product, some just need to see sketches, while comprehensive tests are enough for others. In each case, the concepts should leave some space for imagination and creative freedom of other team members. It is natural because the creative technologist’s main task is to deliver the visualization of ideas.
My choice of using an inductive or a deductive method was motivated by a different kind of challenge in either production. Deductive reasoning starts with a premise which leads to other premises and then to the inevitable conclusion. Inductive reasoning goes in the opposite direction. It is based on repeated observations that lead to generalizing repeatedly observed phenomena.
In documentary realization, my inductive reasoning relied on a comparison between the content of people’s stories and the ways of telling their stories. I started with a number of specific instances: animation, Skype stylization, 3D simulation, photorealistic picture extension, green screen. Then my research led to the generalization and the stylistic integrity of the projection scenes. I drew conclusions from observations; the more information about the realization possibilities and protagonists’ stories I had, the closer to the final solution I was. In this aspect, my first visit to Iceland was absolutely necessary to get the required amount of data. At the same time, I made a series of rehearsals. I tried different realization techniques. The process was similar to a puzzle type game, where I had to connect scattered dots to receive a well-defined shape. In this phase of generalization, the elimination of unnecessary elements was crucial. The inductive method is not efficient in most artistic productions because it consumes a lot of resources. However, it was the best solution in this particular case.
In the virtual reality realization, my deductive reasoning was more consistent. Each next step was a modification and improvement of an initially chosen artistic direction. I started with a general premise that we want to show the pipe organs interior. The technology that offered most complete immersion was virtual reality. We wanted to increase the feeling of a presence inside the instrument and thus we opted for the virtual reality technology. Immersion is connected with interactivity. The most interactive realizations in virtual reality are game based. This led to the conclusion that we want to use a game engine for the realization. Most games have levels related to progress. The organs have three storeys and the balcony, so the game was divided into four distinct stages. The target group consisted largely of inexperienced viewers, so I added a tutorial at the beginning. The greatest flaw of the virtual reality organ tour was the feeling of loneliness inside. The most difficult and expensive element in this type of realization was a realistic human animation. We had a limited budget, so I decided to add 360-degree movie sequences with a live character. We needed to limit the area of exploration. The Baroque organ is more important and more spectacular. Therefore, the Romantic organ is treated as a reference in the background. The game progress is connected with completing the tasks. We needed to expose how the organ works, so the viewer will assist the organist to prepare the instrument to the concert. In this way, I started to test theories to reach the final solutions. Deductive reasoning led to a specific instance of a game realization. Conclusion was drawn from the premises.
During the virtual reality endeavor, I encountered several production problems. The project coordinator on the Philharmonic side left the position, and no one was delegated into her place. The contract between Philharmonic and the vendor effectively locked me out of my own work, and the number of correction iterations that we could implement was increasingly limited. The contract did not guarantee access to game sources, so were forced to ask the vendor to introduce every alteration. At the end of production the vendor cut me off of making final adjustments. I wanted to make the changes myself but I was denied to have an access to the blueprint and final asset. No beta tests were provided by the vendor; I had to organize them myself. Unfinished work was accepted against my suggestions, so there was no fourth phase of corrections. Problems of this kind are common in commercial work, but they make the innovative project overcomplicated and result in a discrepancy between the artistic goal and the final result. My artistic concept was not fulfilled because of that, and further project implementation proved it. In such an experimental realization, it is crucial to have a possibility of conducting several tests, spread over time, at the final phase of realization. However, even though I failed to control the work entirely, I still had an impact on the final shape of the presentation, and it was archived as close as possible to my initial idea.
The list of improvements that should be made in the next phase contains bringing the voice sound during the tutorial and during the game to the same level, changing the indicating light into flashing, adding planned special effects to the virtual doors in the tutorial, lowering the light level inside the organs, so that light clues become more visible. To improve assistance, the tutorial should be started after pressing a button, which would allow to set up the equipment for a viewer. The ladder to the third floor should work independently to the progress of the game. Catching the ladder for navigation was not practiced in the tutorial; instead, the ladder should be touched to get to the upper storey. The time counter is visible in the corner of an eye and should be hidden.
Being the director and the creative technologist in the same project is always problematic. During the virtual reality realization, it created a dilemma of priority. When these two roles are divided, the director can focus on a vision, a daydream, and artistic integrity, while the creative technologist supports him/her with original ideas or solutions for the visions, and watches up the resources. When the resources (time or money) are running out, the creative technologist is supposed to signal it and get the project back on track. In my project, the resources ended quite rapidly because of the production problems described above. While it is better for the work and final result when these jobs are split between two people, it is possible to make this type of project with additional support of another team member or to delegate this part of responsibility to the project coordinator or producer. Unfortunately, the Philharmonic did not have such a person, or a budget to hire someone in that place.
If I could improve the process of this realization, I would focus more on the contract and the extension of the final phase of testing and correction. I am also convinced that it is possible to move even more conceptual and prototyping activities to the pre-production stage. The game type realization creates a temptation to work directly on the final meshes, while form simplification facilitates the storytelling exploration. The first testing version was based on the tutorial part, and the most important elements were left until the end. I was focused on immersion breaking points, and the storytelling was secondary to it. However, I should instead treat these two components equally, maintaining balance between them.
I am aware that the new medium requires a new kind of approach. The simulation of 3D space in games was initially regarded as a means of achieving movie aesthetics. Later, however, 3D space came to function as a means by which a greater level of immersion and involvement than cinema could deliver played a part in a development. [1] I tried to find a new language of expression adapted to new rules and, of course, as any artistic language, it requires constant improvement.
No matter how appropriate a platform or an experience may seem to you as an author, your project will be more successful if you design the content respective to the audience’s point of view. Like in solving a two-thousand piece jigsaw puzzle, audiences need the overall image to make sense of each individual piece. [2] After the premiere, I conducted an additional survey among twenty users to verify the result, to check the possibility of the occurrence of immersion breaking moments, or to evaluate the story experience. It was based on five questions related to the “Road to Excellence” realization.
Tutorial. Does it work as a standalone experience?
Most people consider a tutorial as easy to follow, but it does not work as a standalone experience. The respondents often do not listen to the voice of the narrator; they are waiting for the instructions of the assistant. After observing dozens of people of different ages, I have the impression that at the first contact with VR there are so many different stimuli and sensations that the sense of hearing is "switched off." After a few minutes, with the support of someone from the outside, these people become more confident, and then they open up to voice messages from the presentation. The only way to improve seems adding additional visual clues and animations.
The presentation in the first place was planned as an assisted experience, on location. The virtual reality equipment – backpack computer, goggles, and manipulators – are difficult to operate for a beginner. They need guidance. The tutorial requires more visual clues. In addition, my design of virtual doors was implemented only partially. The glass cube with the door in the middle of the room is devoid of visual effects I planned. People are disoriented which door to choose.
Navigation. Is it intuitive and does it preserve immersion?
It does preserve immersion, but for some people without prior VR experience, the teleporting or “jump” is unnaturally fast, which creates a certain feeling of distance from the experience. It is not intuitive what to do and where to go. Experienced gamers are more used to interaction with unexpected conditions. After a few repetitions of the "jump", the user’s movement gets a bit smoother. The common problem is getting up the ladder to the last level. The illumination of the expected direction is also not always understandable. Not everyone wants to go towards the light.
A possible solution for it could be to turn the guidance light into a flashing cycle. This could be implemented easily but it was never done because of the contract. Human mind perceives changing elements of the environment more distinctly. The dissolved light inside the organ is still too strong, in spite of my correction. There is also a need to explain how to hold the manipulators. Most people point the manipulators upwards, while they should bend the wrist and point them downwards to set the “jump” indicator to the right position. As an improvement, I made an additional manual to supplement the guidance of an assistant.
Story. Is it clear and retellable?
In most opinions, the plot is clear and quite simple, although people have a problem to follow the story. They admit that it is nice to see a human character as a guide. Choosing directions or navigating might be confusing, for example it might be difficult to have to go back, to open a door, or to to climb a ladder. Viewers in the VR world focus on what is happening around, on their experience, and listen mostly to the person who supports them in the real world. Sometimes, only after completing the tour and removing the goggles, they ask questions about what they actually saw and where they were. Some people ask for a possibility to play a few sounds on the organ. An element of interaction could appear in the organist's workshop sequence, instead of merely standing and watching (raising the pipe or something like that). Respondents do not always concentrate on the story, because their attention is focused on the way they move. Some people claim that they would follow the instructions if they could repeat the experience.
The narrative in virtual reality does not have the same impact as in flat cinema. Unexperienced users are often stunned with the new kind of experience. The immersion dominates over storytelling significantly. To maintain the balance between the story and the immersion it would be necessary to strengthen the visual clues. Narrator’s guidance is not enough. The option to play the organ would increase the cost of the realization and distract users’ attention. However, signaling such a need indicates that the story is not emphasized enough. People are trying to explore the environment on their own. On the other hand, we cannot force people to follow the storyline. I took into consideration that some of them will want to explore the new environment and will not follow the narrator. That is why I added the time limit to the main version of the presentation (it has three different scenarios). It naturally ends the presentation for people who prefer the individual way of exploration.
Sound. Does it live up to expectations?
There are problems with sound volume during the experience. The narrator's voice is too low; you cannot hear the content. Users have problems to follow the guidance. A person using a hearing aid had could not follow it at all. The sound quality is good, but all that is happening around the viewer and the level of noise in the real environment make a significant impact on the intelligibility of the voice.
The sound was not correctly implemented by the VR sound programmers. They promised to deliver an advanced method of tracking the sound direction based on the headphones angle but it was never fulfilled. I had to improve the quality myself, adding a reverb effect to all audio files to make it more realistic. I asked for an impression of the voice coming from the inside of the instrument, but instead, it was leveled down. In view of the fact that the presentation is often used in a crowdy and loud environment, this poses a serious problem disturbing the storytelling.
Immersion/Realism. Is it believable, does it induce the desired impact?
Users feel totally immersed. The sense of height and depth were convincing enough for one user’s brain to induce a fit of vertigo and we had to break the experience. She understood where she was and felt the sensation of being inside the instrument. Another user, suffering from claustrophobia, was so scared that she had a problem to go inside. She was afraid to fall off the edges. There are opinions from people who have seen the real organs inside that what they see in the presentation is a little bit worse, but they are also aware that it is a digital equivalent of a real instrument. There are also questions from people who have never seen the instrument inside if it really looks like that. They are truly surprised.
The computer rendered parts in 3D work well, but the 2D live-action sequences lose the sense of scale, which is a common problem connected with 360 degree filmmaking. The impact of the presentation on the people shows how strong a virtual reality experience is, but also proves that the immersion meets the expectations and this part of the presentation is well made.
The survey I carried out after the premiere proves that it is absolutely necessary to provide four levels of beta testing for such an advanced, experimental realization. During the first level of testing, I made conformity assessment with the initial idea, script, and compliance with the real instrument; during the second level I focused on general rules of movement and interactivity; the third level was a thorough elimination of game errors. The fourth level could possibly improve the quality of the experience and eradicate the remaining errors.
A creative technologist needs to be constantly vigilant to what is happening around and ready to leap into action quickly. He/she should be up to date with the changes that are constantly introduced during the production as the schedule is modified. The worst scenario is when the budget or the time limit is reached and introducing a next round of corrections is not possible anymore. On the other hand, each artistic project has to have its end, therefore it is better to achieve progress, not perfection. Tight budgets and time limits trigger creativity and provoke us to find new ways to accomplish and to present artworks. The dilemma every artist confronts is when to stick with familiar tools and materials, and when to reach out and embrace those that offer new possibilities. All artists test new means of expression, but in time exploration gives way to expression. [3] It is also related to risk that the finished work is not perfect because each next work could be even better. Against all odds, my job as a creative technologist dynamically contributed to the creation of the film and virtual reality realization.
