Image Stream Investigations
In the following sub-sections I investigate the image stream and some of its forms of presentation. The image stream is important to my research partly because I use it as the main structure for Close, Closer, Closest. It is also the main structure in traditional films. I have decided to use the term ‘image stream’ based on a reflection made in chapter 4, in the sub-section ‘The Screen’. An image stream is created when images overlap each other or are fed onto the screen. It is one of two ways to present information on screen, the other way being through spatial models (explored in chapter 2). According to my observations, the image stream has three presentation forms in digital comics:
- Pages
- Panel delivery
- Cinematic panels
In this list, I place ‘panel delivery’ in-between ‘pages’ and ‘cinematic panels’ because I think it represents a middle stage between the other two presentation forms. I will, despite this order, first address cinematic panels, since this form has been the main focus of my image-stream investigations. I will then address panel delivery. Traditional printed comics are presented on pages, so I do not focus on this form in my artistic research.
Cinematic panels
The term ‘cinematic panels’ denotes comic panels that overlap each other sequentially within one and the same frame. I describe the panels ‘cinematic’ because the comic moves away from its original identity with juxtaposed panels to share the image-stream structure with its sibling, the film. If we look at a digital comic made with full-screen cinematic panels, only one panel from the multi-frame will be visible at any given time, just as is the case when viewing a traditional film (Groensteen 2007: 24).
Since the imagery of a film also changes within the same frame, cinematic panels and the film are closely related presentation forms. As I mentioned in chapter 1, the only difference between the two is that a cinematic-panel comic has sequential images with closure between each panel, whereas a film has full motion with closure between each cut. By ‘closure’, I refer to the reader’s ability to mentally fill gaps in the visual imagery, which is necessary to make sense of the story (McCloud 1993: 67). Because both presentation forms relate to a fixed frame, they make it possible to create seamless shifts between full motion and cinematic panels. There is consequently less contrast between the motion film and the type of comic with cinematic panels.
I first understood the presentation form of cinematic panels when I was able to distinguish it from a presentation form called ‘panel delivery’, which was introduced to me by Daniel M. Goodbrey. Panel delivery, which I will discuss later in the chapter, feeds panels onto a screen instead of onto pages. It shows the changes occurring within the frame of the negative space, whereas with cinematic panels, the changes happen within the frame of the panel (Figure 6).
Although I had previously used cinematic panels when making digital comics (Video 66), I had only done so intuitively. The first time I sketched cinematic panels with their properties in mind was when I sketched my first draft, I Don’t Know Grandpa (2012). This was an experiment with the spatial relationships between the images, where the changes in the panels created a mobile frame that moved from one side of a fjord to the other (Video 67). I also did other experiments, for example by using cinematic panels in a panel layout and by treating the screen as a mix of a page and a split screen (Video 68).
After the premiere of my performance comic Sound of the Aurora in June 2014, I returned to making a digital comic for the reading-tablet platform. Creating a comic for a tablet was one of my aims in this research project because when the tablet capabilities are compared with those of other types of computers, it distinguishes itself as an ideal platform for digital comics. This made the reading tablet highly relevant for comic artists, and I thought it should be a subject for digital comic research. Close, Closer, Closest (2016) was developed for the iPad.
The project I returned to was an earlier manuscript I had developed called I Don’t Know Grandpa. I concluded that it was a bit too long, so I chose a shorter manuscript that became Close, Closer, Closest. With this script, I explored image-stream motion, which is the subject of this chapter.
Following the pattern set in chapter 2, I will describe the framework of the comic before I explore the subject. The framework differs from that of Sound of the Aurora because the work has a different form and is experienced via a different platform. I start with the story development and my choice of format, which is full-screen panels, following up with aspects concerning the tablet platform, namely its user interface, programming and soundtracking. The choice of format was challenging to make, and I started deliberating over it even before I started on Sound of the Aurora. The solution I came up with relates to the image stream, which is why I devote the main part of the chapter to investigating it. I divide the investigations according to the concepts of ‘cinematic panels’, ‘panel delivery’ and ‘automated and interactive full motion’, closing with personal reflections.
In making this digital comic, I worked closely with the programmer Hans Philip Eide, who programmed it in Actionscript for the Apple iPad. I also worked with the musician Stephan Meidell to create the musical score for the comic. I took recourse in my second supervisor, the professor of film Septimiu Moraru, and the comic artist Kim Holm as consultants at the storyboard stage, and to discuss the user interface. The work was first performed at St Mary’s Church in Bergen in March 2016, and finally published on iTunes Store on 8 May 2017. After the launch online, I performed Close, Closer, Closest four times in 2017 (at the book conference Nordisk Barnebokkonferanse at Sølvberget in Stavanger, at the art centre Hordaland Kunstnersenter, at Meridan Space in Beijing, and at the cultural centre Stormen in Bodø). In April 2018, the work won the Gold Award in the Norwegian Visuelt Competition. It won in the category of illustration / books and was the only digital publication represented.
Creating for the tablet
User Interface
In this section I want to describe how the user interface in Close, Closer, Closest was developed. Presenting a single image at the time, as in a slideshow presentation, may seem simple. However, on a touch screen, which is the controller system of a reading tablet, there are several possibilities.
The user interface gives the reader access to all the content of the digital comic. In Close, Closer, Closest, it involves the input controls for navigation, which are visible or invisible buttons or control mechanisms. The user interface also includes input controls that create a more subtle form of interactive participation (Dixon 2007: 20), where a reader can control some reflections in the imagery and the gliding characters in the menu (Video 60).
In a workshop with my secondary supervisor Septimiu Moraru and the Bergen-based comic artist Kim Holm, we established that it was important to facilitate a comfortable and relaxed reading position. I ended up adopting a suggestion from Holm: readers hold the tablet with both hands and only use their thumbs, almost like a game controller (Picture 23). Thumbs are very flexible and can do vertical, horizontal, diagonal and circular motions, so are not that limited at all. However, when I observed people using a reading tablet, they mostly held it with their right hand and used their index finger to tap the screen. Since my user interface differed from this, it became unconventional. I therefore needed to inform readers about the control system with a small tutorial. That the user interface needs to be accompanied by a tutorial does not need to be perceived as negative. Tutorials are common in computer games because user interfaces can vary from publication to publication. Digital comics and computer games share the same premises as new media. The fact of needing a tutorial is more a confirmation of the principle of variability in new media (Manovich 2001: 36). Since user interfaces can vary, not informing readers about the controls can result in negative user experiences. I placed the tutorial for Close, Closer, Closest at the beginning of the app. There is always a possibility that readers will skip over it, but it is there and always available in the main menu.
Soundtracking
I have not addressed audio in my research, partly for sake of limiting the scope, partly also because Goodbrey has addressed this subject in his doctoral research at the University of Hertfordshire. I use his article The Sound of Digital Comics (2015) as a source when I now describe the music made for Close, Closer, Closest.
Music had a central role in Sound of the Aurora, and so also in this comic. In the story, music is one of the links between my grandfather and me. I engaged the musician Stephan Meidell to create the soundtrack for Close, Closer, Closest. He is a contemporary musician and his main instrument is the guitar. His musical interest is somewhere between the ambient and the melodic, and I thought he could make a soundtrack that did not require additional sound effects, since there are so many different noises in his soundscapes. I also wanted original music, with roots from my hometown, Bergen. We equipped Stephan with an iPad so he could keep up with the development of the comic, and we met frequently to implement his sounds into the comic. For these sessions, Eide, Meidell and I worked together in Meidell’s studio at Bergen Kjøtt, giving him the opportunity to create new tracks on site if needed.
Meidell, in this comic, made minimal use of ‘diegetic sound’, meaning sound that is present in fictional space (Goodbrey 2015: 4). An example would be a sequence in which a vinyl record is playing, and we hear the crackling as well as the music when it starts playing. The sounds synchronize with the animated content in the frame. The looped crackling is synchronized with the loop of the vinyl record’s rotation, and when we hear the music, I cut to a flashback sequence, which causes this diegetic music to have a non-diegetic relationship to the image presented in the frame (Video 63). Another example of diegetic sound is the sequence with a torpedo launch. The launching gives a short and instant sound, so there is no conflict between the static image and the time-based sound. Goodbrey refers to these short sounds as ‘spot sounds’. Longer sounds can conflict with the frozen moments in a comic sequence, so it is important to make sure that the sounds are timed to fit the visual context, and vice versa. In Close, Closer Closest, the torpedo sounds relate directly to or are caused by the action we see happening in the illustrations. The music from the vinyl record, on the other hand, gives new information, telling us what kind of music is spoken of in the text. It is therefore a more independent component of the narration. The soundtrack is mainly non-diegetic, acting as a musical frame that creates and communicates moods. It also emphasizes reading rhythm (Goodbrey 2015: 7). It is slow and calm there where we want the reader to take more time, and it is faster there where we want the reader’s pace to increase. The increased pace in the soundtrack also increases the tension in the scene. But I am not letting real audio do all the work in this digital comic. In traditional comics, sound is communicated through visually expressive text and symbols, and this is also the case in Close, Closer, Closest. The dialogues are written text, and animated lines represent or indicate the ringing of a smartphone (Video 64). Meanwhile, the talking animated head of Aunt Tulli represents the ambient sound of speech. The speech bubbles themselves represent non-diegetic carriers (Ibid., p. 5).
Meidell was also asked to participate in the project because he had experience in making modular music. With modular music, I was able to create a ‘responsive soundtrack’ (Goodbrey 2015: 7). ‘Responsive’ in this context means that the music can change or adapt to every input made by the reader when he or she navigates through the contents. The programming activates modules of music, for instance loops and linear compositions, which play and stop at given positions in the comic. One example is the opening scene of the war sailors swimming in the ocean; more tones are added to the soundscape as the reader scrolls forward. When reading that the men are drowning, the visuals do not change, but the music does, and it eventually disappears along with the men, as described in the text. The soundtrack is thus responsive in the scroll-activated sequences. An example is that the volume of a sound effect increases while the reader scrolls through the first word in the main title. The soundtrack adapts to the process of reading.
There is no extra-diegetic sound in my comic, for instance navigation sounds (Goodbrey 2015: 2). The only feedback the reader gets when advancing with the navigation are the visuals and the related diegetic and non-diegetic soundtrack.
Cinematic panels are static, sequential images that overlap each other. Each overlap is a transition to the next image. I find cinematic panels to have two types of transitions:
- Instant transition
- Dissolving images
For the instant transition, one image is directly replaced by another. The dissolving-image transition is more time-consuming, since one panel must dissolve while the next comes into view. It can be done in different ways, but one example is that the second image gradually changes from transparent to opaque.
Instant transitions create time-effective reading. The imagery is perceived instantly by the eye/brain as the reader navigates (Video 73). Nevertheless, I do not think this sort of transition can compete with the rapidity and flow achieved just by browsing juxtaposed panels. Before starting on this artistic research project, I had some finished and some unfinished attempts of making digital comics. I was intuitively delving into cinematic panels and had problems with the instant transitions. These problems resurfaced when I started experimenting with cinematic panels in this artistic research. In particular, I had a problem with the direct replacement and thought the changes were too sudden.
In comics, the temporal relationships between the panels always vary in relation to the context. With an instant transition, the change is sudden, so the temporal shift feels much more apparent than in juxtaposed panels (Video 74). With juxtaposed panels, the reader’s closure seems softer. Maybe this is because I as a reader can browse freely around while figuring out the relationships between the images. I fill in the gaps while I read (McCloud 1993: 63). With instant transitions between cinematic panels, I still fill in the gaps and create closure in my mind, but there are some sequences where sudden changes feel brutal and hard on the eyes, because I can observe the changes. In some transitions I can see that the temporal and spatial relations between the panels get broken. I find this happening in moment-to-moment and action-to-action sequences (Ibid., p. 70), where the point of view is consistent throughout the sequence. Motion in these sequences, even if it concerns moving objects or a moving frame, becomes staccato.6 I see the motion happening, but I also see that footage is missing. I have called this phenomenon ‘broken motion’. It is a negative term because this type of motion initially did not seem right to me and I tried to avoid it. I first tried to soften the transitions by making them dissolve, but this only slowed down the experience and did not solve the problem. Changing the point of view, however, hid the motion or change that happened in-between the panels. This is a trick I have often seen in TV-series with limited animation (Video 6 in chapter 1).
After trying to avoid broken motion when making Sound of the Aurora, I decided to embrace it when making Close, Closer, Closest (Video 74). Instead of seeing broken motion as a flaw, I tried to see it as moving a step closer to film. This presentation form could give me animation and closure at the same time.
The animation occurring with cinematic panels encouraged me to experiment with motion more than I would probably have done had I been making a traditional comic with juxtaposed panels. Perhaps I overdid the animation in Close, Closer, Closest, but it is my first comic in this format, and I allowed myself to get carried away. I found that if I included broken motion in my cinematic panel sequences in a consistent way, a ‘reader contract’ would be formed and readers would start to accept it.
The instant transition applies to more than just low frame-rate animation. It was also observed in the magic lantern technique called ‘the startling change’, which was described by the theorist Lewis Wright. The lantern’s ‘slipping slide’ had the ability to make an instant change between two images, and it was used as an effect to surprise the audience (Wright 1891: 141). The element of surprise in a traditional comic book is usually dependent on turning a page. Thanks to cinematic panels and panel delivery, a surprise can lurk behind every single panel or object in a digital comic. The startling change is a contrast in the graphical relationship between two images. It is a technique used in the American horror comic The Eighth Seal (2013) published by Thrillbent. I also use it in Close, Closer, Closest, but not in a startling way. In one scene where I, as the main protagonist, make a sound recording, I am unsure whether the smartphone has successfully recorded what I intended it to record. Through the visual expression, I show the uncertainty in the situation by giving the lines of the image a sketchy or uncertain quality. The room surrounding the character is not visible, but the reader can get an idea of it because it is drawn on a piece of paper that lies underneath the drawing the reader looks at. When the reader clicks forward, the character is suddenly more finished and the room that was initially only glimpsed under the paper emerges, as a visual metaphor implying that the recording was indeed successful. In this scene, the text and images are symmetrical, so the visual metaphor reinforces the text (Video 75).
In a comic book, the panel layout and panel shapes create rhythm in combination with the comic book’s visual content. In cinematic panels, by contrast, the frame is fixed, so all I have to create rhythm with is the visual content and the transitions. I introduced this section by stating that instant transitions are time-effective. They help when creating rhythm because their instantaneous quality is always consistent and never dependent on variables. The instant transitions made me approach rhythm in relation to the frequency of reader interactivity, that is, every time the reader swipes from one panel to another.
I created contrasts between sequences where the reader flies through the content, contra sequences where the interaction tempo is reduced. In sections with high reader interactivity, I often show animated motion. These sequences have almost no content on which to dwell and are received so fast that the reader moves on instantly (Video 76). When I wanted to slow down I needed to do the opposite. The panels would then contain more content for the reader to receive, and most importantly, to perceive (McCloud 1993: 49). Text, which must be perceived, is the most efficient tool to calm the pace. It requires that the reader pauses to read. I use longer texts when I want the reader to dwell on an image. I find two ways to do this with text and instant transitions.
The first way is to place all the text together with the image (Video 77), thus causing the reader to stop interacting until all of the information is perceived. The second way is to make the reader dwell on an image while still interacting. An example is in the intro of Close, Closer, Closest (Video 78). In this sequence the rhythm is slow and the reader dwells on only one image while reading a longer text. The text is presented in portions, giving the reader medium-paced interaction frequency. The image in the intro could have been static, but I made it into an animated loop. The animated loop creates an ‘eternal moment’ within which the reader can take however much time he or she needs to read. It also creates rhythm, and I will say more about this shortly in the section called ‘Automated and Interactive Full Motion’. Just now, however, I want to point out that cinematic panels have a third way to control rhythm, and this is through ‘dissolving images’, the opposite technique of instant transitions between overlapping panels.
When an image dissolves, it creates a fluid transition to the next image. This was called a ‘dioramic effect’ (Wright 1891: 143) in the context of the magic lantern (which, as already stated, I consider the predecessor to digital comics), and it was made by using two or more lenses. As well as causing one image to dissolve into another, the technique can make fluid changes within one and the same image. Classic magic lantern slides with this effect show transformations from day to night, or from one season to another within one image. I experimented with the technique in my first draft, I Don’t Know Grandpa (Video 79), but instead of making panel transitions, I made subtle transformations or changes in light and shadows that supplemented to the mood. In this way the static images became more vibrant and could be stocked with additional information. I continued using the technique when making Close, Closer, Closest in order to present illustrated material. All the images were photographed with different light settings, so they could dissolve into each other and give the illusion of reflection in the plastic surface (Picture 30).
In contrast to the instant transition, the dissolving image transition is a full motion that consumes time. I must also mention that it is possible to make a dissolving image sequence with an instant transition, but it will not be fluid or able to consist only of two images, as does the dissolving image transition (Video 80).
Most of us are used to experiencing a dissolving image as an automated effect, for example in computer games, on websites and in presentations made with Apple’s Keynote software. But even if its automated form is an established norm, it does not have to be automated. When I reflected on reader control and animation, I understood that the reader could have full control over dissolving images as well as other types of animation. In Close, Closer, Closest, this is executed in one panel transition and in all the light reflections in the Scotch tape (Video 81). I will write more about automated motion in Close, Closer, Closer later in this chapter, and I address reader control in chapter 4. But there is yet another aspect of dissolving images to mention: their semiotics.
What can dissolving images mean or signify? For starters, they can suggest a span of time, as Bordwell and Thomson (2010: 233) point out, or they can signify a dream sequence by shifting from everyday life to an imaginary reality. This is what I do in Close, Closer, Closest when thinking about my grandpa (Video 80). I also use dissolving images in a scene at the end where I levitate. The scene is a visual metaphor for the loss of a parent, a secure haven. This is also an imaginative and surreal happening, which is why the sequence, which starts with hard transitions, is softened by dissolving images towards the end of the scene (Video 82). The time-consuming dissolving effect adds more rhythm to cinematic panels. It is a graphic, temporal and rhythmic contrast to the instant transition. Even if the change is fluid, it harmonizes well with static form.
Panel delivery
I defined the concept of ‘cinematic panels’ in order to differentiate my practice from what is called ‘panel delivery’. I will describe panel delivery as an image stream and a motion graphic technique that feeds or replaces panels, either through automation or interaction/reader navigation. If a given case of panel delivery involves instant and/or dissolving images, it uses an image stream; if the panels fly in and out and move around in full motion, it (the given panel delivery) uses the spatial form of motion graphics. In this chapter I will mainly refer to the image-stream form of panel delivery. If a panel overlaps the previous panel, then this is not a case of panel delivery but of cinematic panels. The difference is that while cinematic panels change the fictional space within a single frame, panel delivery changes the panels in the negative space. Panels can appear, disappear, shrink, grow, change format, cover each other, move around, and so on. Since panel delivery, cinematic panels and pages are all image-stream forms, they can readily be combined with each other. That said, in my study of digital comics, the most common combination seems to be panel delivery with elements of cinematic panels (Video 83). Close, Closer, Closest is mainly made with cinematic panels, however, I have added a stanza of panel delivery to create a rhythmic and temporal shift in the comic (Video 84).
‘Panel delivery’ is not a well-established concept. Comic artists who are new to digital comics might find it confusing because the technique has different names from publisher to publisher. Marvel calls its panel-delivery comics ‘Infinite Comics’, but I see this more as a product tag. Comixology called its panel-delivery comics ‘Native Guided View’ for a while. ‘Native’ could mean a comic that is designed specifically for the digital platform. To combine ‘Native’ and ‘Guided View’, however, I do not think was a good idea. The Guided View mode was, according to my own observations, introduced in the early 2000s. Marvel has also launched digital comics on its website, and members pay a fee to access them online, either reading page-by-page or entering the Guided View mode. In this mode the screen becomes a mobile frame, moving in on the first panel on the page, then moving from panel to panel and adjusting the frame when necessary (Video 85). The mobile frame follows a programmed track with dynamic motion – a ‘dynamic track’ as I call it in chapter 2 (the section on ‘Mobile Framing’). The mobile frame exposes panels by browsing over a comic book page. The screen plays the role of the eyes, navigating on the spatial surface, and it takes control away from the eyes, which makes the term quite accurate. I do, however, see a similarity between Guided View and panel delivery, for both focus on one or a small group of panels at the time. But Guided View is a spatial model using a mobile frame. The screen moves rather like a slow and fixed representation of what the eyes would ideally choose to focus on. In panel delivery, by contrast, panels are fed onto the screen. As such, the image stream affects the layout, building up, moving and continuously developing the onscreen content. Thrillbent, the digital comic publisher which was established by Mark Waid and John Rogers in 2014, is a pioneer in panel delivery because all its publications are based on panel delivery and cinematic panels. However, this company, as far as I know, has not introduced any terms for these two presentation forms. The term ‘panel delivery’ is suggested by Daniel M. Goodbrey, and I find it quite accurate because it describes the actual function of the presentation form.
Comics in general rely on the concept of ‘reading direction’. Reading direction relates to all continuous, spatial arrangements. In a comic, these are the panels, texts and elements of the illustrations. If the reading direction is not taken into consideration when composing a comic, the reader will misunderstand the chronology, and the relationships between the various bits of information will be hard to understand. In the West, this direction goes from left to right and from top to bottom. The panel or object on which the eyes focus represents the present time.
In panel delivery and cinematic panels, however, I find the reading direction to be overruled by what is called the ‘order of appearance’. When a panel appears on screen, regardless of whether it happens through an instant transition, a dissolving image or a spatial change as in motion graphics, I find that it claims the reader’s focal attention and becomes the most relevant information on the screen. It overrules the control of the eyes and represents the present time. Because of this, a panel can appear wherever it likes in panel delivery. The same principle applies to speech bubbles in cinematic panels. Still, the rule of reading direction does not disappear entirely, for it is still active within the composition of the panel or object that appears. Two or more panels can be fed onto the screen at the same time, and they will also need to relate to reading direction.
But a panel-delivery feed is not tied to the reading direction in the same way as is a comic with a pure spatial arrangement. The panels that appear can be built in a direction that moves from left to right or from top to bottom. Either way will create a flow, since the pattern is based on a norm that the reader can predict. Breaking the reading direction can convey contrasts and chaos. It will change the rhythm and probably generate surprize. Two things – the order of appearance and the reading direction – give the digital comic artist more opportunities to create variations in composition and rhythm than would be possible in traditional static comics. In Close, Closer, Closest, I use a split screen in my panel delivery. The three first frames of the panel delivery build the split screen. From then on, the panel grid is fixed and allows for a seamless mix of panel delivery and cinematic panels. The panels in the grid are cinematic panels, but I do not change the same panel consecutively; I jump to the next panel to make a change there, and then to the next as the reader advances. This is panel delivery made through cinematic panels in a split-screen layout. It is a short sequence, but I have tracked the rhythm of the order of appearance in a video (Video 86):
Start>follow>break>follow>break>follow>follow>full-screen
I break the reading direction from the start, jumping back and forth, as if trying to establish rooms in the submarine. Breaking the reading direction creates a staccato sequence that conveys the tenseness of the situation as well as the communication between the rooms the panels represent. Before the captain gives the order to fire, there are two panels that follow the reading direction on a row. They create a short but flowing leap to the picture of the lever that is pulled in the full-screen image. This panel-delivery sequence represents a shift in Close, Closer, Closest’s rhythm and presentation form. It stands in contrast to the more static full-screen cinematic panel sequences that dominate the comic, and it shows one way in which panel delivery and cinematic panels can co-exist and cooperate.
Another aspect of panel delivery is how it relates to time. On a traditional comic page with a panel layout, all the panels that are exposed have the opportunity to represent the past, the present and the future. It is the eyes’ focus that decides the status of the panels: whatever is focused on at any given time represents the present, whereas previously focused-on panels represent the past and upcoming panels represent the future. Previous pages are always the past and future pages are always the future. In the case of cinematic panels, the future panels and the panels of the past are hidden from the reader, just like the pages in a book. In panel delivery, however, panels coexist much as do panels on a page, but at the same time they are different because new panels are always being added and removed. The most recent panel in the feed must relate to the other panels on screen. I have formulated four models to describe how panel delivery deals with time:
A) Future – present – past:
This model of panel delivery has a panel layout in which all panels are visible. They can be presented on pages or a larger canvas. Instead of feeding panels, panels are activated. An early example made with this structure is Marvel’s first The Amazing Spider-Man Cyber Comic, Sandblaster (1998). The panels of the page are shaded and lit up when activated in a form that can remind of model B. This comic mixes cinematic panels and panel delivery (Video 71).
B) Present – past:
This model only shows the panels of the present and the past. The future is always hidden. It is a common panel delivery model in which the layout is built panel by panel (Video 87). Every new panel represents the present, and every previous image represents the past. In some comics, the panels are erased when the screen is filled, then a new series of panels begins and builds a new layout. It can be explained as building page layouts. Other comics of this type create the layout by building and removing panels simultaneously. This last approach, in which the oldest panels give way to new ones, is used by Eric Loyer, who did the programming for Ezra Claytan Daniel’s digital comic Upgrade Soul. Here Loyer uses a model he calls ‘Zero Sum’, meaning that when a new image comes onto the screen, an earlier image must exit the screen. If a large image enters, an equally-large image (or images which in combination are equal in size to the new image) must exit the screen. The result is that the frame size of all the frames in the layout adapt to the new content every time the reader moves forward in the comic (Video 88).
C) Future – present:
This model is the opposite of model B, and I have never seen an example of it. In this model all the panels are visible, but when the reader interacts and navigates, panels disappear as the reader advances. This means the past is never visible to the reader. The reader is like Pac-Man eating up the panels (Video 89). It can symbolize that time is spent, wasted or gone. I would guess this is not a very functional model because the eyes would read faster than the reader would interact, and I will claim that it is more functional to feed new panels than to discard read panels. It can, however, be an interesting model for a hypercomic: if there are multiple pathways and no possibility to go backwards to retrieve erased panels, the choices the reader makes are decisive – perhaps even devastating. Being able to look at future panels can help the reader make a choice.
D) Present:
This model, which only shows the present time, is the one I use in Close, Closer, Closest. The past and the future are hidden just as in cinematic panels. Panels are still fed onto the screen, but all the on-screen panels must represent the present (Video 90). In chapter 1, I wrote about how the split screen in film shared similarities with the magazine in its visual expression. Well, this model of panel delivery can be compared to the split-screen technique in film, where multiple frames show simultaneous events. When all images on screen need to represent the same moment in time, past actions must always be removed or masked away. While making Close, Closer, Closest, I also explored this form in an experiment I made at Eric Loyer’s motion comic workshop at Fumetto International Comic Festival in Lucerne (2015) (as I also mentioned in chapter 1). In this experiment, I worked with this time mode within a fixed panel layout, which can be compared to a fixed split screen. The sequences I made within this fixed panel layout had a combination of cinematic panels and panel delivery (Video 8 in chapter 1). The fixed panel layout is a form that differs from most panel-delivery comics I have seen. The most normal approach is adaptable dynamic-panel layouts, at least layouts that change over time. The small stanza of panel delivery in Close, Closer, Closest builds up the layout, which turns out to be fixed. Basing a whole comic on a fixed panel layout is a form I want to explore further after this artistic research. A slightly different approach than the one I used for my panel delivery sequence in Close, Closer, Closest and my experiment at Fumetto is the one used by Daniel M. Goodbrey for Empty Kingdom (2014). In this game comic, all the panels on screen represent the same time. Every hyperframe forms an open world via fixed panel layouts that the main character (who is controlled by the reader/player) can move around in (Video 3 in chapter 1).
I will close this section with a brief look at terminology used when working with panel delivery and cinematic panels. The Norwegian digital comic pioneers Morten F. Thomsen and Lars Schwed Nygård made a (short-lived) publishing app for digital comics in 2012 called Oxicomics. In making the app, they consulted with Mark Waid from Thrillbent, a publisher known for digital comic publications that use the techniques I call panel delivery and cinematic panels in this artistic research. According to Schwed Nygård, Waid had not developed terms when they worked with the Oxicomics format, so Thomsen and Schwed Nygård made their own concepts to verbalize the process (Schwed Nygård 2017).
A ‘step’, in their terminology, is when a reader advances to the next panel or change through an input action such as a click, tap or swipe. They used the term ‘screen’ for a sequence of steps that constructs a panel layout that forms a page (Video 91). At a time when the medium of digital comics is still finding itself, terms like this will always vary from artist to artist. To collect terms would therefore be an interesting research project in itself. To contribute to this, I will present the terminology I used when working with the programmer Hans Philip Eide and the composer Stephan Meidell. I also used the term ‘steps’, but gave it a different meaning.
I used the term ‘frame’ to describe all the panels, including their steps, and all the frames in the full-screen animations in the comic. I used ‘scene’ as a term to divide the scenarios, as in film, and I gave the panels numbers within each scene. If a panel sequence was represented through several panels with the same point of view, as in a take in film, I used the term ‘steps’ within this sequence. For example, the first panel in Close, Closer, Closest consist of 19 steps (Video 92). Close, Closer, Closest has 12 scenes, as shown in the top menu (Video 93). It has 46 panels which then form the multiframe. However, when counting all the frames with full motion, the multiframe consists of a total of 2,238 frames. There are frames in animated loops too: for instance in the swimming sailors, the record player and the cell-phone video, but these are not counted in the total amount of frames. These animations represent what I call ‘sub-sequences’ within the frames. There are 12 sub-sequences in Close, Closer, Closest (Video 94).
Contents:
Acknowledgements | Introduction | Chapter 1 | Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Reference List
Framework
Story development
Close, Closer, Closest is an autobiographical story with a theme dealing with family history and family relations. It is based on researching my own grandfather and his experience of the German occupation of Norway during World War II. It is a prose text, and my narrator voice is present almost all the time, except in the imagined scene inside a submarine.
The comic starts with a story that was told to me by a man who visited my artistic research-process exhibition in 2014. He told me about a war sailor who experienced being torpedoed three times. This story sets the tone and the theme. The comic continues with an exposition of my research on my grandfather. I present him as a war sailor, and I interview relatives to learn more about him. I find it challenging to catch the persona of my grandfather from what they tell me, and although I do not get any closer to my grandfather, I grow closer to the people I interview.
I describe my mother’s sudden death, which in the given context emphasizes the loss of stories as long as they remain untold. It exemplifies for me that stories from the war could disappear with my generation – if we do not pass them on. The loss of a parent and the loss of a close relationship are universal experiences that many adult children can identify with. At the end of the comic, I make a visual comparison between my loss and the lost situation of the war sailors in the introduction. Their ship has sunk, and they swim alone in the ocean.
This short story portrays a common post-war scenario experienced by people of my generation who want to relate to deceased family members. It is personal, but the theme of relationships is universal.
Format choice
In the beginning of my artistic research, the seemingly endless possibilities of new media were almost paralyzing. I could not see the limitations, so neither were the possibilities clear to me. I did not yet know that two structures – the image stream and the spatial model – were the two possible ways to present a comic on screen. My limited perspective made it difficult to choose which digital comic format to work with.
I could have asked myself whether I should work on a spatial structure or an image stream structure. Instead, I focused on whether I should use a panel layout with a negative space or a full-screen panel without a negative space (Figure 4). Since the full-screen format was the most rarely used, I started there. I sketched the manuscript I Don’t Know Grandpa with a full-screen panel format and made Sound of the Aurora with the same format. However, I was not convinced I should continue with this format for Close, Closer, Closest. Why should I make a digital comic with a full-screen panel format?
A brief conversation with the comics researcher Daniel M. Goodbrey from the University of Hertfordshire incited me to explore the image stream metaphor.5 Goodbrey told me about ‘panel delivery’, which was his term for the format where panels are fed into the screen. He reminded me about ‘spatial models’ and ‘pages’, which are McCloud’s terms for describing the two structures. This made me reflect on whether or not my full-screen panel could be classified as a page. My reflection on this question can be read in chapter 4, and my conclusion was that I would replace the term ‘page’ with ‘image stream’, because I found the page to be only one of three forms within the fixed-window structure. Moreover, ‘page’ was not a good enough descriptor for a main structure in digital comics.
Goodbrey’s influence did not stop there. In our conversation, he told me that he personally would not consider making his own comics by abandoning juxtaposed images. In his opinion, discarding the panel layout was like removing a large portion of the visual language of comics. This statement made me reflect: What aspects or properties would be lost if my comic panels were not juxtaposed?
One aspect that would be lost – McCloud mentions it in Reinventing Comics – is the distinct identity of comics (McCloud 2000: 215). Juxtaposed panels and their layout give the art form a distinct identity that distinguishes it visually from other art forms. Another aspect is the collective visual relationship between panels that are presented together on a page or in a spread. Their composition and expression, both contextual and graphical, affect the overall reading experience. A third aspect that would be lost is speed, because our eyes can read a juxtaposed sequence much faster than when we click through a sequence panel by panel. This also relates to a fourth aspect: by placing panels next to each other, readers are free to create their own reading pace and flow. This freedom would be lost. These four aspects could be considered big losses when seen from the perspective of traditional comics. Nevertheless, I decided not to use juxtaposed panels, arguing that if something was lost, something new would fill the void, and I wanted to find out what that would be.
My goal from the start of the project was to make a user interface that was as simple as possible. During the reading of Close, Closer, Closest, I wanted the screen only to contain content from the comic, without any visual disturbance such as visible buttons or navigational icons. My first user interface had two gestures to navigate through the comic: tapping and swiping. Tapping was for fast-paced navigation through the cinematic panels, while swiping was for ‘scroll activation’, a term for a highly sensitive control scheme that makes it possible to control video sequences and animations. Both the tapping and the swiping were possible to do with one thumb. Nevertheless, when I practised these gestures of the minimalistic and button-free user interface with my programmer Hans Phillip Eide, we found that while tapping was effective, there was a barrier to making readers understand the swipe. Simplicity and consistency are important in a user interface; without them readers become confused. This made my combination of two control schemes challenging. I had two aims: to make a digital comic that experimented with cinematic panels and full motion, and to give readers full control of the reading process most of the time. With these goals in mind, I also had to acknowledge that reader control is lost when a film or animation is automated. (I address this aspect of reader interactivity or control in chapter 4, in the section called ‘Reader Control’.)
Instead of inserting a visual cue to make readers change their navigation gesture during the reading experience, we removed tapping as an option and only used the thumb swipe. The swipe gesture was functional for both instant inputs and highly sensitive inputs, which meant that I could combine two presentation forms in one gesture. The new user interface was thus to swipe upwards for forward navigation and downwards for backward navigation. Tapping was later reintroduced as a gesture that activated the menu, which contained pages and language settings (Video 62).
We had an additional problem with the change between swipes for instant transitions and swipes for scroll-activation. It could be difficult to notice that the control scheme changed from instant transitions to scroll-activation. To solve this, we added a subtle glide mechanism to the scroll to give more feedback from the thumb swipe (Video 61).
Programming
Hans Philip Eide programmed this digital comic with ActionScript, using AIR SDK to cross-compile from ActionScript to a native app (iOS). The new aspect for me in this project was that we always had to take into account file size, and that the compression of files and file management needed to be file-size-friendly. To reduce the size of files, it became important to reuse textures and backgrounds. In sequences where I use steps, some graphics, for instance backgrounds, remain constant, while moving objects or figures change from step to step. These figures must be made as ‘sprites’. Similar to the old cells in classic animation, a sprite is a single object or figure in a digital file; it has a transparent background and is saved in the PNG-24 format. Eide used the software Texturepacker to organize all the graphics in the comic. This software makes sprite sheets, which contain all the graphics the computer needs to load at a given time. How Eide arranged the sprite sheets was part of the logistics for how he wanted the programmed comic to load graphics while running.
Eide did not want the whole comic to be loaded at the beginning, largely because it might cause a delay and the reader might not be willing to wait. The graphics are stored in the memory, and if the user has several applications open at the same time, this memory space is already occupied. To prevent bad performance or to avoid having the application crash, freeze up or be switched off by the reader, Eide loaded the graphics in portions. He hid the loading into fades during some of the transitions. This has resulted in some unwanted fades in my instant transitions, but I consider it better than using a loading screen that interferes, or better than risking that the performance of the comic can be unstable on older iPads. These fades will be more invisible as the next generations of iPads become more powerful, but this is an aspect that can be planned more fully in future works, to take advantage of these fades. I also want to mention that Eide used a software called Adobe Texture Format to do PVR formatting, which helps render the graphics directly on the screen memory.
The idea to sketch cinematic panels that made the panel frame move in fictional space was inspired by the computer game Myst (1993). I played this a lot as a teenager. McCloud (2000: 208) also refers to Myst but does not address the concept undergirding its visual presentation. In the original game, the player navigated through 3-D environments that were different than those in most games of today. These environments were built from a bank of sequential images that let you go from image to image, creating the illusion that you were moving through a 3D space (Video 69). Another more well-known example is Google Street View, yet Google adds an in-between transition that creates the illusion of moving in space (Video 70).
When I became aware of the presentation technique that I chose to call cinematic panels, I began searching through early digital comics to see how the technique was used. The oldest I found was Marvel’s Cyber Comics that runned from 1996–1998 (Video 71). The Frenchman Balak (Yves Bigerel) is one of the modern comic artists who uses cinematic panels in his digital comics (Video 70). He mediates the technique in his digital comic about DIGITAL COMICS (2009), and the presentation form is called ‘turbo media’ on the webpage where his text is published (https://www.catsuka.com/turbomedia/index.php). Then there is Thrillbent, an online publisher established in 2014, which only publishes comics with panel delivery and cinematic panels, but its artists do not formulate concepts in order to verbalize the technique. I have tried to contact Mark Waid, one of the persons behind Thrillbent, but have not succeeded in getting an interview. At one point I registered that the digital comic-book shop Comixology called comics with panel delivery ‘Native Comics’. This was only for a short period, and they do not use that term today. Marvel, on the other hand, has its own production line at Comixology, which it calls ‘Infinity Comics’. These are digital comics with panel delivery and cinematic panels.
‘Cinematic panels’ and ‘panel delivery’ are specific terms for two distinct forms of presentation. In most digital comics I have seen on Comixology and Thrillbent, they are combined. They are two presentation forms amongst other forms such as the page and spatial models. Because they all can mix and vary, finding adequately descriptive terms for the various types of digital comics is challenging. I believe, however, that in the future, some combinations will become more mainstream than others and that terminology will emerge, just as with hypercomics, motion comics and performance comics. It will then be easier to examine each module individually, as I do here. Let us now take a closer look at cinematic panels.
Personal Reflections
There are three turning points that I want to highlight from the process of making Close, Closer, Closest. To begin with, I want to repeat the second turning point that I mentioned in the last section of chapter 2, about presentation forms that communicate well to crowds, and just put it in another light. Close, Closer, Closest is my first digital comic published for a reading tablet. The reading tablet was my medium of choice when I started this artistic research in late 2012. The reason for this was the arrival of the iPad in 2010 – the first widely-available reading tablet in the West. As a portable computer, I think it was, aside from the smartphone, the first platform for reading digital comics since the PC. Furthermore, since it was a new platform, it was highly relevant to explore. Nevertheless, I am grateful that Sound of the Aurora was created before I made this digital comic for the iPad. Without that experience, I would never have thought of performance when I designed it. The result is a digital comic with a double functionality: it can be read on a tablet by a single reader, but also in front of many people in a live performance. I also experienced that there is a lower threshold to performing a comic than to finalizing a digital comic publication. I first performed Close, Closer, Closest in 2016, but it took a whole additional year before I could publish it online. Because of the slow process of making a digital publication ready for release, I assume that I will also in future perform digital comics before they are ready for digital release.
What ended up becoming Close, Closer, Closest in 2017 started with a storyboard of another manuscript in 2012. In this draft, called I Don’t Know Grandpa, I made a sequence that was inspired by the computer game Myst. I made overlapping comic panels to create a mobile frame that moved around in fictional space. Even though I had already been intuitively using overlapping images in digital comics, this was the point when I understood that the overlapping comic panels had properties that approached animation while still retaining the identity of a comic. But the second turning point did not really come before I met Daniel M. Goodbrey at the Visibility Conference at Bergen Academy of Art and Design in late 2014. He introduced me to the concept of ‘panel delivery’. This challenged me to find out whether I was making panel delivery, or something else. My conclusion was that what I was making was different, thus my coining of the term ‘cinematic panels’.
The third turning point that had a huge impact on my artistic research came early in 2013 when I discovered the comic demonstration Soul Reaper from 2011 (Video 29 in chapter 2). HTML5 programming had become more common in web design by this time, and the concept of scroll-activation became a widespread effect in webpage designs the following years. Soul Reaper functioned as an early showcase. It had full motion, but the motion was not automated in the traditional way. The instances of full motion were bound to the action of scrolling, so they were reader-controlled. This was a big game changer. Before this, motion had been automated and stood in contrast to the reader-controlled comic experience. Film, from the start, had been able to show an automated comic. But from now on, the comic could show motion that could be reader controlled. This led to my reflections on reader control and the interactive full motion of Close, Closer, Closest.
Close, Closer, Closest had a different starting point than Sound of the Aurora, and I see this as one of the strengths of its development process: the image stream was defined, I had a prototypical concept I call ‘cinematic panels’, and I knew I wanted to address interactive motion and create a comic in which full reader control was dominant. This made my research focus on the image stream more apparent in Close, Closer, Closest than was my focus on spatial motion in Sound of the Aurora. Panel delivery as a presentation form was already starting to establish itself as a technique, so I chose to focus more on cinematic panels. Despite this, and despite how little I practiced panel delivery, I found out more about it than I had expected to. A weakness in my research (which is also relevant for the research discussed in chapter 2) is that I did not narrow my focus further. I managed to create a fundamental overview of concepts and techniques of motion graphics, and I gained some experience with each technique. But I did not, for example, take the two comic devices and look at text and motion and image and motion in my reflections. Yet just as I have built on the reflections and research of others, I hope my research can contribute to the next steps of research taken by others. I think we need more artistic and academic research that concentrates on panel delivery and cinematic panels and which can narrow its focus to experimenting with smaller modules, for example with text in motion.
The visual expression of Close, Closer, Closest is very different from that of Sound of the Aurora. The biggest difference is that the drawings are simpler and only have solid outlines. This expression can be animated through classic animation techniques without setting up a disruptive contrast to the static comic sequences. I would describe the illustrations as stylized naturalism. I chose a more naturalistic style because the story is autobiographical. I show through my drawings that I can represent reality in a more confident way than when I draw, for example, my aunt’s story in Sound of the Aurora. It is closer to reality. The illustrations are drawn with a thick marker, and I have used glossy Scotch tape as a graphic element. The marker is, for me, similar to sketching with a pencil; it is a tool for making direct and honest drawings, since the process from the mind to the paper is short. The marker is also ruthless in the sense that it cannot be erased. I do not regret mistakes in the lines, and I embrace the imperfection in the drawings. This imperfection is lost in the roto-scope tracing of photographs, but the comic is an interesting mix of both. The marker line is a ‘naked’ expression, like a signature, and it works well in a personal story like this. The paper on which I draw also has a strong presence in the illustrations. This is because I have photographed all the illustrations, so the lighting on the paper surface is included in the final work. The Scotch tape is a material that is only expressive when the illustrations are photographed, in order for the tape’s surface to reflect light. I have been fascinated by real light in drawings since my childhood experiences of watching animation films. The backlighting effects in animated works such as Masters of the Universe amazed me. Real light, represented on a screen, is also a light source. In Close, Closer, Closest, I used the light to make dynamic illustrations. I photographed them with three different light settings. With these light variables, I experimented with dissolving images and gesture controls, where the reader can affect the reflections in the image by tilting the reading tablet.
Close, Closer, Closest is a short digital comic. It dwells on a moment in life and describes the time when my family history became important to me and my own identity. It touches on a problem that many people have, namely, that they do not know their own parents very well. This realization often comes when it is too late to learn to know their parents better. The same is the case with family history and even war history. The stories are there, but they disappear with the people who lived them.
Full-screen panels
In the foregoing paragraph I wrote about the aspects that are lost if the panels of a comic are not juxtaposed. In this section I want to focus on what I gain if I present the panels of my comic in full screen. I want to mention that the full-screen panel structure is not solely found in digital comics. There are examples of comic books with one panel per page, for instance The Cage (1975) by Martin Vaughn-James. The Chinese pocket book comics have this structure too (Video 58), however, it is not very common in the West.
The first aspect of a single full-screen panel could be described as a small benefit. It relates to Scott McCloud’s concept of the ‘infinite canvas’ (McCloud 2000: 200). This concept, which is based on the principle of numerical representation (Manovich 2001: 27), means that the canvas of a digital comic, in theory, has no size limitations. McCloud’s metaphor of a canvas is what I identify as negative space and fictional space in a digital comic. The metaphor of an infinite canvas is mainly referred to when talking about spatial models, in the sense of an unlimited canvas, but the principle of numerical representation is also, I think, relevant for the image stream structure. An image stream in digital comics has no physical limitations. That said, the aspects of there being no size limitations is only a theoretical truth, because the computer has limitations in its hardware technology which affect the possible complexity of graphics and digital-comic programming. These limitations, I think, can be compared to the limitations on the size or length of a physical book. The difference would be that the size of a digital comic can increase as technology develops. Despite the limitations of hardware technology, the aspect of numerical representation is beneficial for full-screen panels in digital comics. I can use the whole presentation surface on one single image at the time, without paying heed to any physical restrictions that would dictate the length of my piece if it were produced on paper. I do not need to compress my sequence to a definite panel layout in order to have it fit into a given space.
The second aspect concerns what visually happens when one full-screen panel is replaced by another. If the images share the same frame, they overlap. This aspect is not only relevant for full-screen panels; it is relevant for all panels that feed a stream of images into the same fixed frame. This means that overlapping panels can be made with all types of panels, from full-screen panels to panels in a layout on a page or on a spatial surface. I want to emphasize that juxtaposed images can present the same information as overlapping images, the only difference being that overlapping images make the changes within the same frame. Juxtaposed panels are a spatial presentation, whereas overlapping panels are an image-stream presentation. When comic panels overlap, the changes in the fictional space are exposed in the transition. The comic sequence approaches the motion picture as an art form, for we can see the changes happening in the imagery. Because overlapping panels share the same image-stream structure as the motion picture, I call this presentation form cinematic panels. I could have called it overlapping panels, but I want to emphasize that the two art forms come closer to each other with this presentation form. The second aspect is therefore the appearance of cinematic panels, and I will return to this subject and investigate it more thoroughly in a dedicated section later in the chapter.
The third aspect of the full-screen panel comic is its ability to control viewers’ focus. To discuss this, I return to my performance comic Sound of the Aurora (discussed in chapter 2), which was my first longer comic with full-screen panels. After Sound of the Aurora became a performance comic, I reflected on my experience of performing a comic live. It prompted me to think back to the time I read comics for my son before he was old enough to read himself. I recall that I always used my index finger to mark where I was on the page, so he could follow the story. This flashback made me understand that a comic with juxtaposed images is designed for individual reading. When I read the comic for my son I had to control the focus. This marks a conceptual and practical difference between comics and film, but also between comics and a good many children’s picture books. I would claim that children’s picture books are mostly designed to communicate with more people than just the reader, because the reader is mostly a performer for others. Furthermore, the pictorial pages usually have large layouts designed to keep the focus on the page or on the double-page spread. A film is not a reader experience, so the focus of its single-image presentation communicates well to crowds. This focus, however, is disturbed if viewers must read a text on the screen; then they will need to revert to an individual reading experience. Some will read slowly, others quickly. It is then hard for the film to control the focus of viewers. When I present the panels of a comic in full-screen mode, I have full control over the focus of the audience. From this, it is clear that full-screen comics are well-suited for communicating to crowds. This third aspect – focal control – is a value that traditional comics with juxtaposed panels do not have.
There are also other presentation forms than full-screen panels to control the focus in digital comics. Panel delivery is a presentation form based on the image stream. This technique feeds panels into the negative space, and even if the panels are juxtaposed, the latest panel to enter the screen claims the focus. Panel delivery therefore functions like my index finger when I read comic books for my son. Another focus-controlling presentation form was mentioned in chapter 2, namely Guided View, which is a mobile frame that focuses on one image at a time in a panel layout.
The aspect of focal control strongly influenced my decision to present the panels of Close, Closer, Closest in full-screen mode, for it would enable me to perform the comic for an audience even if it was made for a reading tablet. But the second aspect – overlapping panels – became the main reason to choose this direction, for it gave me the opportunity to do more research on the image stream and cinematic panels.
Automated and interactive full motion
Most of the motion in Close, Closer, Closest is connected to the cinematic panels, however, I also present full motion through classic animation, and I use spatial motion in much the same way as one finds in motion graphics. The reader will notice that among all these full-motion sequences, few are automated film. They are interactive film. This is because the cinematic panel experience results in a type of animation that gives the reader full control over his or her acquisition. Full reader control is a property that stands strong in physical comics, so I wanted to challenge myself to make a digital comic that allowed as much reading control as possible while still using animated full motion in the presentation. In this section, I will first present the automated full motion in Close, Closer, Closest and then present the interactive full motion.
I start this digital comic with a long automated-animation sequence of sailors swimming in circles. This presentation, however, is not a pure full-motion sequence but a combination of full motion and cinematic panels. The image is automated full motion and the text is a cinematic panel sequence (Video 92). The swimming sailors are made with classic animation and the motion is looped. The text consists of single images set in sequence.
The animated loop is meditative to look at and creates an eternal moment. The sailors are swimming for their lives. They keep swimming to avoid drowning. The automated motion creates tension because the reader cannot control it. I also think the fate of the sailors becomes real to the reader when the sailors are shown in full motion. This is closer to reality than a traditional comic sequence can get. The loop represents a certain moment in time, and the text begins before this moment appears and then passes the moment during the text sequence (Video 92). The text and image create an asymmetric-symmetric-asymmetric sequence, which I never have done before. This could have been executed in a traditional comic sequence as well, but it was the combination of full motion and cinematic panels that led me to create this passage in the way that I did.
There are more automated loops in Close, Closer Closest. With the spinning vinyl record, I attempt to evoke nostalgia in the reader, to create the mood of putting on a record, or at least to describe it for readers who have no experience with this form of recorded music (Video 95). Later in the comic I use an automated loop to show how Aunt Tully talks incessantly (Video 96) while the narrative text moves on. Just as in the opening sequence, the text surfs atop a loop that creates an extended moment.
Another loop distinguishes itself from the others. It is the film on the iPhone, seen towards the end of the comic. This represents an Instagram loop that I actually made from some film material I found in the Norwegian Military Film Archive. I came across it while watching the documentary called Krigsseilerene – Med æren i behold (2008, loosely meaning ‘The War Sailors: Retained Honour’). In this material I found footage that seemed to be of my grandad, because it was identical with a photograph we (my family) have from the same situation. This loop is not an everlasting moment; it is a clear repetition of a series of events that are edited together (Video 97). I used motion graphics and mobile framing when I animated this sequence. The repetition creates a rhythm that differs from that of the fluid and seamless loops seen in the earlier frames.
There are also automated loop animations that indicate sound and vibration. These iconic lines function as fine static, but the threshold to animate them is low, and their presence increases in strength when animated because the motion draws the reader’s attention (Video 98). The last loops I will mention are vibrant speech bubbles. These are automated loops presented within an interactive film, which is also a combination that I have never done before. In this case, if the reader stops to dwell on a speech bubble, the loop perpetuates a frozen moment.
What does an automated loop add to the imagery that a static image could not add? I think a loop communicates an everlasting moment in a more effective way than does a static image. It encapsulates the reader in an ongoing cycle. A long sequence of images that dwells on a single moment can also communicate an everlasting moment, but the loop is more compressed. The possibility to create eternity in one frame is a powerful property. I categorize loop animation in a comic context as ‘passive animation’. By this I mean that the action and motion, rather than pushing the storyline forward, cause the reader to dwell on a moment in time. This makes loop animation easy to blend into traditional static-panel sequences because it does not interfere with the comic sequence that causes progression in the storyline (Video 99).
Close, Closer, Closest is a rather static digital comic because of it full screen presentation. The loops are therefore welcome devices for creating and varying rhythm. Rhythm can control the speed and intensity of the imagery, and in this digital comic, I have used it for calm sequences such as when Tulli is speaking (Video 96), but also for more intense sequences such as the video on the cell phone (Video 97). I also find that loops can create moods when they are used as ambient backdrops. We find ourselves in the presence of repetitive motion all the time, for instance sea waves, traffic, wind through grass, flames in a fireplace, and so forth. These rhythms create moods that we instantly react to and create associations to. Ambient loops can therefore strengthen the emotive power of an image. Despite this, and despite my having worked with looped backdrops earlier (Video 83), I have not used them in Sound of the Aurora or Close, Closer, Closest. Automated loops are also known on the web as gif-animations and are popular on social media such as Facebook. I would also claim that gif-comics have become a type of digital comics that treat gif-animation as a stand-alone effect (Video 99).
Of all the instances of automated full motion in this comic, the loops are the most common. However, I also have a linear automated full motion sequence at the very end of the work. The last scene has a sinking ship, which is an automated moving graphic. It is programmed to sink on a vertical canvas, from the top and all the way down and off screen (Video 100). The frame, however, does not follow the ship, since it is interactive. The reader controls it in a fixed vertical track. When the reader thumb-swipes, the window moves downwards, and the reader can either follow or pass the ship that is sinking to the bottom of the ocean. This makes the reader an observer of the sinking ship, which the reader cannot control. The semiotics in this mix of automated motion in the fiction and interactive mobile frame can symbolize the helplessness of the situation. The scene, for me personally, is a metaphor for the loss of my mother, but it also reconnects the story to the opening scene of the swimming sailors. The reader controls the frame, so is also in control of how long he or she wants to spend in this vertical space. With this mix of automated and interactive motion in mind, I will proceed in describing the cases of more interactive motion in Close, Closer, Closest.
Two linear film sequences in this digital comic are interactive. They are reader controlled by using the technique called ‘scroll-activation’. Scroll-activation is connected to the navigational action of scrolling. The scroll-activation technique became widespread in webpage design in 2012 because of the new code standard HTML5. This has allowed animations and changing graphics to be programmed directly in a page’s source code, rather than through plug-ins such as the Adobe Flash software. This Internet trend made me aware of reader-controlled animation, and it incited me to reflect on reader control in general (the reflection can be read about in chapter 4).
Close, Closer, Closest is not a scroll comic: as I mentioned in this chapter’s section called ‘User interface’, the navigation gesture in this digital comic is a thumb swipe, which is the gesture a reader would use to scroll a webpage on a touch screen. Instead of scrolling down on a vertical space, which is the norm, I make the reader scroll the timeline of my full motion animations. The first interactive full motion is the title screen. In this sequence, the illustration moves closer and closer to us as viewers, until it is as if we travel through the picture, literary doing what the title says. It may look like a mobile frame, but it is a motion graphic sequence, and the programmer Hans Philip Eide has connected the touch control to the animation timeline.
The second interactive sequence was made using classic animation. I did several experiments with time-lapse animation, also called automated drawing or speed drawing. Despite my experiments (Video 101), I only used the technique in a small section of the main title. Here the final word of the title, Closest, is written on the screen as the reader swipes/scrolls (Video 102).
The third interactive full motion animation – the explosion of the merchant ship – is also classic animation, and it is strategically placed just after a sequence where I expect the reader’s navigation frequency to escalate (Video 103). The panel before the explosion is a cinematic panel with ten steps. It shows a clock, and the reader, by navigating, counts seconds. My intention is that the minimal change of information will make the reader increase the pace of swiping. When the next panel appears, the reader will not be able to stop, so the upcoming swipes will activate the animation of the explosion. I can of course only have intentions for the reader; how the reader actually reads is beyond my control. The explosion ends with an interactive dissolving image, the only one of its kind in this digital comic.
The fourth interactive full motion animation is the phone call scene (Video 104). This is also classic animation, made by rotoscoping motion from video footage (Video 105). I had originally planned to show the narrator’s text in the whole animation, but it did not work in practice. It was too easy to scroll past the text, thus a crucial flaw. Another problem was the iPad’s technical limitations. An image sequence is a heavy type of animation, even more so when it includes dialogue, so when making the comic, the image sequence soon became too long for the iPad to handle. As mentioned earlier, we used old iPads to make sure that if a reader had old hardware, it would not crash. This scene therefore did not turn out as I intended, and the compromise was to freeze and create a cinematic panel sequence in the middle, then end with the rest of the interactive animation. The shaking and wiggling qualities of the film are due to the hand-held source material. I wanted to add a ‘nerve’ to the imagery by using a mobile frame, in contrast to the continuously fixed frame that dominates the comic. Despite it being a challenging scene to make and despite not turning out perfectly, it still marks the turning point in the comic, and it differs from the rest with is full motion animation, which was my overall intention.
There are also interactive motions or changes connected to the gyroscope that makes the iPad register how the reader is holding the tablet. The first gesture-controlled motion is in the opening menu, where the portraits of me and my grandfather can slide towards each other. They never end up exactly side by side, which is a visual metaphor of one of the comic’s conclusions. I also used the gyro controls to affect the illustrations. For example, the glossy Scotch tape, which is a graphic element in the illustrations, reflects light differently depending on how the reader holds the tablet (Video 106). This creates subtle vibrant motion in the static images.
The third gyro-controlled motion did not make it into the comic. The time schedule ended and we already had pushed the project financially, so there were no more resources to realize the idea. My intention was to use mobile framing in the panel-delivery sequence showing the inside of a submarine. The idea was that if the reader moved the tablet, the perspective would change a bit, making parallax motion in the images. The motion was intended to be ‘sensitive’ in order to imbue the scene with nervous tension. The graphics for this sequence were made in pieces and layers, and all that is left to do is the programming. This makes it possible to add the gyro-controlled motion later on, if resources allow. This comic is filled with different approaches to making a digital comic, and almost every scene draws on different technical solutions that are chosen based on the context of the story. The fact that I was unable to realize all my ideas is typical for this type of creative process. Another comic that uses the technique of gyro-controlled motion is Upgrade Soul by Ezra Claytan Daniels and Eric Loyer (Video 107), but there it is an overall effect and not used solely to create tension in a limited section, which was my intention.
Video 93: Tapping the top of the screen enabes the menu. Here will the reader find short cuts to all of the 12 scenes of Close, Closer, Closest.
Video by Fredrik Rysjedal.
Picture 20: The app icon of the digital comic Close, Closer, Closest. This digital comic was published on the iTunes App Store May 8th 2017.
Illustration by Fredrik Rysjedal.
Picture 21: Frame 4 from Close, Closer, Closest, which is an auto-biographical digital comic.
Illustration and text by Fredrik Rysjedal.
Figure 4: Comic panels are arranged in the negative space, which means that a digital comic with a full screen panel does not have a negative space.
Figure by Fredrik Rysjedal.
Picture 26: Sprite sheets collect graphics so they can all be loaded at once. This is one of the sprite sheets of Close, Closer, Closest.
Photo by Hans Philip Eide
Picture 28: Here Meidell is Implementing the sounds into Eide's programming, and making adjustments and new recordings along the way.
Photo by Fredrik Rysedal.
Video 76: Frames with little new information will create a higher interaction tempo.
Video by Fredrik Rysjedal.
Picture 30: I shot the photos on my drawing desk, with ordinary lamps as the light source.
Photo by Fredrik Rysedal.
Video 58: An example of a printed chineese pocket comic with one panel each page. This is the first edition of Taoist priest from Lao Mountain (1984), illustrated by Ji Xian Tong and Yong Sheng Zhou. Based on the story Pu, Song Lin (Qing Dynasty), adapted by Chang Ying Hong Xing.
Video by Fredrik Rysjedal.
Video 59: The second aspect is the concepts of motions that appear when images are overlapping in digital comics. The image stream opens up for modern presentation forms compared to the traditional way of making comics.
Video by Fredrik Rysjedal.
Video 64: Animated lines that indicates diegetic sound that is not presented through real audio.
Video by Fredrik Rysjedal.
Figure 6: Panel delivery feeds the panels into the negative space, and it can be fed both with an image stream or spatial as in motion graphics. There are different forms of panel delivery, which I address in this chapter.
Figure by Fredrik Rysjedal.
Video 69: A small clip from a walkthrough of the original Myst. Myst is made by Cyan Worlds.
Video by Filadodesigns:
https://www.youtube.com/watch?v=e-8CFun3nEw
6 When I write about the sequences in cinematic panels, I refer to McCloud’s (1993: 70–72) panel transition scheme and Bordwell and Thomson’s (2010: 225) terminology from film editing. For sake of orientation, here is a list of parameters of such sequences in cinematic panels:
McCloud:
A. Moment to moment transitions
B. Action to action transitions
C. Subject to subject transitions
D. Scene to scene transitions
E. Aspect to aspect transitions
F. Non-sequitur transitions
Bordwell and Thomson:
A. Graphic relation between cut A and B
B. Rhythmic relation between cut A and B
C. Spatial relation between cut A and B
D. Temporal relation between cut A and B
Video 75: Using 'the startling change', not to startle, but to confirm that the recording in the story turned out good.
Video by Fredrik Rysjedal.
Video 79: In my first experiments was made while working on the draft I Don't Know Grandpa. I worked with shadows and subtle dissoving images. This is a video I made during the process, and you can see the actual experimentation from 04:35.
Video by Fredrik Rysjedal.
Video 80: This video show how dissoving images can be presented through instant transitions.
Video by Fredrik Rysjedal.
Video 83: A sequence of panel delivery I made in 2007. I experimented with the presentation form though I was not aware of the concept. Here the negative space is not visible, but I concider the negative space as the level where the panels are presented.
Video by Fredrik Rysjedal.
Video 86: The panel delivery in Close, Closer, Closest is a mix of panel delivery and cinematic panels. This video show how reading direction is overruled by order of appearance.
Video by Fredrik Rysjedal.
Audio 3: An Norwegian interview with Morten F. Thomsen, a Norwegian digital comic pioneer. Published on the Norwegian comic podcast Teikneseriehovudstaden.
Recorded by Fredrik Rysjedal.
Video 91: Ontonauts, Lost Files: So Joined Together (2014), written by Lars Schwed Nygård and art by Jonas Alexander Brårmo Larsen. The first time I have seen a Norwegian comic making use of panel delivery is in this Ontonauts Project. Read more about the project at Schwed Nygård's website:
http://www.larsschwednygard.com/?page_id=386
Video by Fredrik Rysjedal.
Video 104: The phone call sequence in Close, Closer, Closest is rotoscoped from a video reference.
Video by Fredrik Rysjedal
Cinematic panels 1: The first storyboard of Close, Closer, Closest made in 2014.
Made by Fredrik Rysjedal.
Video 109: This is the model I used as reference for the U-boat that turns around in Close, Closer, Closest.
Video by Fredrik Rysjedal.
Audio 2: An interview with Daniel Merlin Goodbrey after the Visibility conference 2014. Published on the Norwegian comic podcast Teikneseriehovudstaden.
Recorded by Fredrik Rysjedal.
Picture 23: Holding the tablet and using the thumbs are a comfortable way to use the iPad, but it is not intuitive so a tutorial is needed. These pictures are from the tutorial sketches.
Illustration by Fredrik Rysjedal.
Video 61: This video show the glide effect that we attached to the thumbswipe. This made it easier to combine swift inputs and more high sensitive inputs in the same presentation. Since most of the interaction are swift swipes, the glide effect will make sure that an interactive animation gets triggered by swift swipes.
Video by Fredrik Rysjedal.
Video 63: Diegetic sound from the vinyl player becomes non-diegetic when I cut to a flashback in Close, Closer, Closest.
Video by Fredrik Rysjedal.
Figure 5: A visualisation of the difference between juxtaposed panels and cinematic panels.
Figure by Fredrik Rysjedal.
Video 71: The Marvel Cybercomics is officially taken off the internet. The Amazing Spider-Man - Sandblasted (1996) is a panel delivery with cinematic panels in each panel. This video is a screen recording from: https://www.youtube.com/watch?v=C7qjtC3Iaw0
Video by Fudgebudger.
Video 74: The broken motions in Close, Closer, Closest are quite gentle I think. This is because there is not much change from the one image to the other. However, notice the difference between turning the phone, and the thumb swipe, where the latter is not broken. This is because it seems like an realistic motion. It represents a rapid motion that the cinematic panels are able to acheive.
Video by Fredrik Rysjedal.
Video 77: Text will make the reader pause navigating, I therfore find text versus no text as a strong tool to control the rhythm of interaction.
Video by Fredrik Rysjedal.
Video 82: A surreal end scene, a visual methaphor that binds the end of the story with the beginning.
Video by Fredrik Rysjedal.
Video 87: This video visualize the four forms of panel delivery that I have observed through my studies and work.
Video by Fredrik Rysjedal.
Video 88: Upgrade Soul (2013) for iPad by Ezra Claytan Daniels, Erik Loyer and Alexis Gideon.
Video by Fredrik Rysjedal.
Video 89: The best comparison to explain model C is Pac Man by BANDAI NAMCO. This is the iOS version of Pac Man for the iPad.
Video by Fredrik Rysjedal.
Video 90: The panel delvivery in Close, Closer, Closest is made with model C, where all panels on screen represent present time.
Video by Fredrik Rysjedal.
Video 95: Putting on a record. The song is a sample of Grex Vocalis, Auferstehen.
Video by Fredrik Rysjedal.
Video 99: My scroll comic, or gif-comic, Ovis Ariesaurus Rex (2015) is a good example of how animated and looped panels blends with the static comic panels.
Video by Fredrik Rysjedal.
Video 101: This is a process video of my experimentations with the draft I Don't Know Grandpa from 2013, here I try out time-lapse as an animation technique.
Video by Fredrik Rysjedal
Video 106: Images dissolving to show a reflection in the tape of the illustrations.
Video by Fredrik Rysjedal.
5 I invited Daniel M. Goodbrey to give a lecture on game comics at the Visibility Conference at Bergen Academy of Art and Design in 2014.
Video 60: Gyro sensors in the iPad open up for gesture controls from the reader.
Video by Fredrik Rysjedal.
Video 85: Here I show Comixology's guided view while reading Marvel's Ms. Marvel - No Normal (2014) on my iPad. This comic is made by G. Willow Wilson and Adrian Alphona.
Video by Fredrik Rysjedal.
Picture 22: Close, Closer, Closest is made with performance in mind, which have given me possibilities to read it at different venues. Here at Norsk Barnebokkonferanse 2017 (The Norwegian Childrensbook Conference)
Photo by Andrea Rocha.
Video 67: The first draft I made in this project was called I Don't Know Grandpa (2012). In This video I go through the storyboard. 5 minutes into the video I show how I plan to use 'cinematic panels' to create a traverse over a fjord.
Video by Fredrik Rysjedal.
Picture 21: Close, Closer, Closest is designed to be read on an iPad.
Illustration by Fredrik Rysjedal.
Video 108: A small clip from my studio in Kong Oskarsgate 62. It took 1,5 year to make Close, Closer, Closest.
Video by Fredrik Rysjedal
Picture 27: The trio behind Close, closer, Closest. From the left Hans Philip Eide (programming), me Fredrik Rysjedal and Stephan Meidell (music composer). Here we work in Meidell's studio.
Photo by Klara Sofie Ludvigsen.
Video 68: In this video from 2014 I show an experiment with an alternative story, where I try to make panel delivery. It is interesting to see this video in retrospect, because what I show here is not panel delivery at all. This is because all the changes happens within the panel frames, so this video shows a digital comic with cinematic panels within its panel layout. It is also interesting listening to how I'm trying to verbalize what I am doing, calling the negative space for the grid as one example. This video is from the period where the concepts I investigated started to take form.
Video by Fredrik Rysjedal.
Video 110: Video documentation of the digital comic Close, Closer, Closest.
Video by Fredrik Rysjedal.
Video 66: My first intuitive approach on cinematic panels in 2007. This is a sketch for a fantasy comic I called Ortorian, however this title was never fully realized.
Video by Fredrik Rysjedal.
Video 65: A 'Kinora' by The British Mutoscope and Biograph Co, from Preus Museum in Horten. It shows in a physical way the concept of the image stream.
Video by Fredrik Rysjedal.
Video 62: This is the tutorial video that plays when the reader opens the app, and it is also available in the main menu.
Video by Fredrik Rysjedal.
Picture 24: A long swipe for interactive animations, or scroll-activated animations as they also are called.
Illustration by Fredrik Rysjedal.
Video 94: This sub.sequence is an animated loop made with motion graphics.
Video by Fredrik Rysjedal.
Video 70: An example of Google Street View by Google anno 2018. The inbetween transitions is at this time almost showing a realistic movement, but in the first editions I recall that the image was scaled and then replaced by the next.
Video by Fredrik Rysjedal.
Video 72: This video shows Balak's turbomedia or digital comic about DIGITAL COMICS (2009).
Video by Fredrik Rysjedal.
Video 78: This example shows a longer text placed upon one looped animation.
Video by Fredrik Rysjedal.
Video 81: Not all, but most of the frames with tape and reflections dissolve when the reader tilt the iPad.
Video by Fredrik Rysjedal.
Video 107: Upgrade Soul (2013) By Ezra Claytan Daniels, Erik Loyer and Alexis Gideon has 2,5D graphics that is connected to the gyro controls of the iPad. If the reader moves the tablet, small vibrant shake effects appear in the fictional space.
Video by Fredrik Rysjedal.
Video 84: The scene from the u-boat is the only scene where I use panel delivery in Close, Closer, Closest.
Video by Fredrik Rysjedal.
