Frozen Moments in Motion

An Artistic Research on Digital Comics by Fredrik Rysjedal

The comic
Defining the comic seems difficult, almost impossible, according to the foremost experts, one of whom is Aaron Meskin, an associate professor of philosophy at the University of Leeds. He concludes his article ‘Defining Comics?’ (2007) by saying that we need to get beyond the definitional project. He questions Greg Hayman and John Pratt’s essay ‘What Are Comics?’, deeming it as an example of a definition project that excludes too much. What is the necessity of defining comics, he asks? The well-regarded Belgian comic theorist Thierry Groensteen, in his book System of Comics (2007), elaborates on how difficult it is to define comics. Defining an artistic medium, he says, creates limitations, reduces possible meanings and excludes artistic expressions and variations of the medium (Groensteen 2007: 14). The Swedish theorist Fredrik Strömberg has made a comprehensive analysis of the concept of comics in his book Vad ër tecknade serier? (Strömberg 2003). He asks a fundamental question: What are comics? His analysis gives a good overview of the Western discourse on the topic. He also concludes that it is impossible to create a correct or precise definition(Ibid., p. 120). Researchers, he says, must be more careful in presenting definitions as universal truths (Ibid., p. 114), and we should be aware that any definition will always mirror the author (Ibid., p. 113).

Meskin claims in his conclusion that there is no pressing need for a definition (Meskin 2007: 376). I interpret him to mean that each individual work of art should be understood on its own terms and in light of its own history. Meskin and Strömberg thus touch on the same subject. I follow Strömberg’s conclusion that defining the comic is difficult and that there is no universal truth, but I am not convinced by Meskin when he asserts that a definition has no purpose. Is there no need to describe the comic art form as a unity? I think there is a need for it if one wants to put a comic in a historical perspective. However, I think it is also important to be aware that the definition of the comic is subjective and will change from culture to culture and from time to time. The comic is a cultural expression(Strömberg 2003: 32).

Why do I need to define the comic? In this chapter, I have already suggested that the digital comic consist of four parameters: the comic, the screen, the computer and interactivity. To understand these parameters, I want to identify the properties of the comic in order to see what happens to it when it gets digitized and presented on a screen. The definition discourse by Meskin, Groensteen and Strömberg has helped me realize that it is impossible to construct an adequate definition. It has also taught me that I must be conscious that I am on a mission, that I actually seek a definition that includes digital comics. This definition, however, does not need to encompass digital comics in their entirety. This is because I realize that at some point, digital comics become something more than just comics. Where is this boundary, and what is beyond it? If I should make a definition that mirrors my intentions, I would want it to be prototypical (Strömberg 2003: 32). This means it would be open enough to include comics that cross the aforementioned boundary. I would also want the definition of the comic to be an aesthetic definition(Ibid., p. 77). This means that the comic would be defined by its aesthetic form and not by its medium. The ‘digital comic’, on the other hand, is a media-specific term; it situates the comic within the realm of the computer. For me, it seems logical that the definition of the comic as an art form should be neutral in terms of media types. I therefore find the aesthetic definition more relevant because it include all types of comics, from comic books and webcomics to performance comics, just to give some examples. By making a montage of existing definitions, I would describe the comic art form like this:

The comic is sequential art (Eisner 1996: 17) that consists of up to two devices, words and images (Eisner 1985: 7). With these devices, or only one of them, the artist forms and arranges content in one or more compositions or panels that are based on the cultural premise of reading direction. The compositions/panels are traditionally arranged in chronological sequences to give a temporal experience (Strömberg 2003: 133) through closure, based on the relationships between the images (McCloud 1993: 63). The multiplicity of compositions/panels can be called the multiframe (Groensteen 2007: 31). How the multiframe is organized depends on its scale and the format of the medium that carries the multiframe. The whole multiframe can be presented on one surface at a time, but it can also be presented in portions, called hyperframes (2007: 30). A hyperframe can contain a single image or panel at a time, or several images/panels at a time.

In creating this definition, I use established concepts from Eisner, and maybe not so well-known concepts from Groensteen and Strömberg. I start the definition with Eisner’s idea that the comic is an art form, and that sequentiality is an essential parameter. I state that the comic consists of images and text and mention them in plural to point out that there are traditionally several texts and several images in a comic. I use the word ‘panels’ instead of frames, because ‘panels’ is an established term in comic design. I add that a multiframe can consist of one or more compositions or panels. It may seem illogical to say that a multiframe can consist of one panel, but in my opinion, one single panel can give a temporal experience. I think it is important to acknowledge that a comic can consist of only one frame, and I say this in relation to Scott McCloud’s definition of a comic in his book Understanding Comics (1993: 9), which excludes the single-frame comic. That said, I think that a comic could consist of one singe panel or composition. If this is the case, what differentiates a single-panel comic from other single images such as an illustration or painting?

Nothing aesthetic, at least. I think it is only the artist’s intention for a single image that can define and differentiate it from everything else, making it a comic panel or a single illustration. This conclusion on intention is an aspect I write more about in chapter 1, in the section ‘The Boundary between Comics and Film’, but I also want to reiterate an acknowledgement I made in chapter 2, namely, that my intention for how to define a work need not coincide with the audience’s definition or categorization of the same work.

There are several comic-related terms that I avoid using in my definition. I do not use ‘juxtaposition’ because I think the preconditions for a multiframe do not require juxtaposition. When I remove a parameter such as juxtapositioning (formerly seen as important), what then differentiates the comic from the picture book? My answer would be the same as for single images. This is where two forms meet. In my eyes, the picture book and the comic are technically the same medium, but they represent or are situated in two different cultures. They can blend and look the same, or the comic can differ from the traditional picture book by having a higher pace between text and images.

Since I do not mention juxtapositioning, neither do I mention the ‘gutter’ between the images. Rather, I use the word ‘relationship’ together with the word ‘closure’. I think closure happens in-between every image that forms a chronology or unity, whether the images are juxtaposed or not. The word ‘chronological’ indicates that the compositions/frames are in a relation with each other, and I stress that I use the word in a traditional sense because there are exceptions to the norm. Drawing on Groensteen’s System of Comics (2007), I use ‘multiframe’ (p. 24) and ‘hyperframe’ (p. 31), but these terms originate from Henri van Lier (multiframe) and Benoît Peters (hyperframe). At first I found these terms a bit too technical. The meaning of ‘multiframe’ can indeed be intuited – ‘many frames’. ‘Hyperframe’, however, is not intuitively understandable. The hyperframe describes a portion of the multiframe that in printed comics would define the format of a single page. Since the page is only one of many possibilities, both physical and digital, the hyperframe is a uniting term. I find it constructive to use because it does not limit the art form to the book, as one example. It is a term that makes it possible for my definition to remain an aesthetic definition.

The computer

The third parameter of digital comics is what makes them digital: the computer. Digitized or computerized media are called ‘new media’ in media science; digital comics are therefore a form of new media. The term ‘new media’ comprises all types of digital media: text, sound, static images, moving images and spatial constructions (Manovich 2001: 19). Diversity was one of the first things I noticed when I tried to find an entry point to digital comics. I wanted to find theory on why digital comics was so limitless and diverse. Manovich describes five principles of new media that I think also can be used to describe the nature of digital comics.

1.     The principle of numerical representation
A digital comic is constructed using a digital code that can be programmed and manipulated through mathematical algorithms. When a drawing is digitized, it is transformed from continuous data to discrete data. ‘Discrete’ in this context means that the data is no longer physically available, but in return it is possible to create random access to this numerical code that can form a comic through computers and digital networks such as the Internet(Ibid., pp. 27–30/52). Discrete data, which has no physical limits, corresponds well with McCloud’s term the ‘infinite canvas’. Because of these non-physical limitations, a digital comic can be both a 2D and a 3D experience. The numerical representation and coding open up for non-linear comics based on hyperlinks, for instance hypercomics and game comics.

2.     The principle of modularity
A digital comic consists of independent units, or modules. The traditional comic consists of two modules – image and text – but in the new media, these are co-existent with other modules such as sound, film, 3D and user-interfaces. These units are built with even smaller units such as letters, pixels and Bassierre-curves, which take us all the way down to the basic unit, namely, the numerical binary code (Ibid., p. 31). The principle of modularity means that the building blocks of digital comics are not restricted to fixed images and text as in print, enabling me to create a digital comic with more modules than would be possible using traditional comic media. It is also these new modules that transform the comic into something more than ‘just a comic’, and I prefer to call it a comic hybrid.

3.     The principle of automation
Automation is the mechanics of the mathematical algorithms used in coding. Algorithms are computer operations programmed to run automatically. They are present in the structures that bind together the modules, but also in the module operations and can be used to perform or manipulate content (Ibid., p. 32). For example, the operation that makes a gif-animation stop or loop is automated, and the film playing at 30 frames per second is another. A more complex example is a digital comic that uses augmented reality. In this case, the camera is programmed to automatically recognize visual data from reality, and, using it as a reference, to mix screen graphics with real-time images. An example of this is the comic book Modern Polaxis (2014) by the Australian comic artist Sutu. An alternative example of automation, one which I use in my own comic Close, Closer, Closest, is the operation of recognizing whether a reader speaks English or Norwegian.

4.     The principle of variability
Variability is a result of the three previous principles. I observe variability in form from comic to comic, but also that a digital comic form can change over time. The differences from comic to comic can be subtle, as for instance the difference between a comic with interactive animation and one with automated animation. The differences can also be more distinct, as for example when comparing a linear reading experience of my own Close, Closer, Closest with Daniel M. Goodbrey’s game comic The Empty Kingdom (2015), which includes exploration as part of the experience (Video 3 in chapter 1). Another aspect of variability is that a digital comic can be changed at any time with an update, or exist in potentially unlimited editions (Ibid., p. 36). There are re-edits, quality upgrades, error corrections and sometimes even entire changes of content. Nothing is written in stone.

While performing my comic Sound of the Aurora, I was able to experience this principle myself. In-between sessions I changed it by adding content. I also edited the comic live, during the performance, which created a subtle form of variation. Rather than being fixed, the comic is still evolving after the premiere. It is as if it only exists when I perform it. But I understand that variability is not always positive, and the negative aspects raise ethical questions about the making of digital comics. I will not address this issue in my artistic research, but the principle of variability makes me wary about the differences between correcting, maintaining and actually changing already-published and purchased material. At the same time, I think the variability in my live comic performance Sound of the Aurora makes every session special and non-repetitive for audiences who see it multiple times, even though the story is the same.

5.     The principle of transcoding
The fifth principle, transcoding, is the communication between the different coding languages inside a computer. There are two levels of computer language: the language the computer uses internally and to communicate with other computers, and the language the computer uses to communicate with the user (Ibid., p. 46). Transcoding is like translation. As a visual artist, I am not in direct contact with these kinds of computer activities, but my programmer uses different coding languages when he constructs my comics and transcodes them to fit the iOS system of an Apple iPad.

The computer is the physical framework for virtual phenomena, and it is what makes screen-based digital comics a new medium. But this medium is not fixed because publications and productions can change, and the variability in form can be huge from comic to comic. Such modularity lowers the threshold for the already-established comic art form to be mixed with other art forms such as film, music, games and so forth. So when does a comic stop being a comic? I think the answer is when the comic gains a module beyond text and images, as for example sound or animation. As such, it is no longer ‘pure’ but becomes a hybrid. I think the art form of digital comics hosts both pure comics and hybrids. Even if the boundaries between them can be subtle, the hybrid will still represent an alternative and experimental direction, and the pure digital comic will continue as the mainstream. But there is still great variability in the pure digital comic, and I see the hybrid forms as expansions of the pure digital comic.

Interaction

In the first section of this chapter, ‘Fundamental Parameters’, I suggested that interaction – that is, the reader’s interaction with the artwork, also called interactivity – was the fourth essential parameter of the digital comic. Interactivity is also seen as one of the properties of the modern screen (Manovich 2001: 102). In my two comic projects Close, Closer, Closest and Sound of the Aurora, I approach interactivity in different ways. For any digital comic, regardless of whether or not it is a reading experience, the creator must take into consideration interactivity and choose the level of interactivity the production is going to have.

All art is an interaction between the viewer and the artwork…

(Dixon 2007: 559)

Steve Dixon, in his effort to structure a discussion on the wide field of theatre, dance, performance art and installation, suggest four levels of interaction in such works (Ibid., p. 563):

1.     Navigation

2.     Participation

3.     Conversation

4.     Collaboration

One of Dixon’s conclusions is that the concept of play (as in games) is not a distinct level of interaction because it uses combinations of these four levels (Ibid, p. 597). He also points out that it is not a perfect model, that the boundaries between the levels can be diffuse, and that the levels are mostly combined in interactive works. Despite this, he finds it useful to think of them as anchor points when discussing interaction. The levels of interaction also have increased complexity, with the first level, navigation, simply concerning finding one’s way through the content. When reading a comic, regardless of whether it consists of pages, panel delivery, cinematic panels or a spatial scroll, the reader needs to navigate through the content to read it at all. I would therefore say that navigation is an activity related to reading, even if it is in a book or on a screen. Such reading requires activating buttons, going back or forth, deciding the path. This is what Dixon calls the simplest form of interaction (Ibid., p. 566).

The second level of interaction is participation, and it allows a user to affect the outcome. Typical models for this kind of interaction are the voting structures found in dialogue in role-play games. The user chooses options that affect the progress by activating alternative endings to the game. In comics, the traditional path is a linear story through which the reader navigates. A non-linear story gives the reader the possibility to follow multiple paths, or, if the story is an open map of panels on a large canvas, the reader/viewer can freely roam through it. Comics that have nonlinear formats are called hypercomics (Goodbrey 2017: 87), and comics that contain game elements which allow the reader to explore or solve puzzles are called game comics(Ibid., p. 123). Both of these digital types of comics require participation. How the story turns out is in the hands of the reader. In the game comic, the reader must explore and solve puzzles to progress, as in Goodbrey’s The Empty Kingdom. Another good example is the computer game Mass Effect 2 (2010, by EA Games). This computer game accommodates new players by starting with a comic that recaps the storyline of Mass Effect 1 (2007). During the recap, new players make choices which affect the story when playing ME2. Those who have already played ME1, however, do not need the recap comic, because the settings and data recorded from their playing the first game are transferred to the sequel ME2. The sequel thus functions as a substitute-ME1 for newcomers, while at the same time allowing experienced players to launch right in and make choices based on their previously recorded data.

Conversation and collaboration are rarer types of interaction in comics. Conversation would require a live performance or a complex artificial-intelligence programming. Collaboration, however, is a type of interaction which I myself have observed. The artist Kim Holm from Bergen had a webcomic project called Diary of a Space Monkey (http://spacemonkey.no), in which he encouraged readers to suggest what would happen next. Each week he chose some of the suggestions and made a voting poll. This digital comic used collaboration when Holm invited readers to act as co-authors, also enabling them to participate by voting on suggestions. Holm, meanwhile, retained editorial power by controlling the content of the voting poll and of how a suggested direction was to be executed.

Dixon’s levels of interaction help me see how the audience can get involved with my comic art. Examples are the reader navigation in Close, Closer Closest, and participation when the reader can control the animation (Video 103 in chapter 3). I also use collaboration when I work with musicians to perform Sound of the Aurora. They improvise while I do live editing. Making Sound of the Aurora and during the process of its first draft, called I Don’t Know Grandpa, I thought a lot about navigation and the reader’s experience. I developed a perspective on the status of the reader when reading a digital comic, which I present in the next section.

Personal Reflections

There were four turning points in my process of exploring the digital comic, and I would like to write about them in this final section. This chapter is the result of a need to base my research on a foundation that can explain what a digital comic is. McCloud (2000) has contributed to this foundation, but only the two last chapters of his book address the actual form of digital comics. He presents aspects very briefly, which means that the aspects can function as stepping stones for further research, which is what I have done. A turning point in this orientation happened when I understood that I had to reflect on McCloud’s two ways of presenting a comic and information on screen. The way he presents them in the book did not give me a foundation that had room for my field of interest, motion. By reflecting on the concepts and rephrasing his theory by adjusting and augmenting it a bit, it became a solid base for my approach to motion in comics.

Another turning point was when I read Manovich’s (2011) theories, as they helped me understand digital comics on a level that made me see the properties and therefore more of the limitations of the medium/art form. By adding his perspective on the screen, the computer and new media, I had more control over what I could achieve. Before this point, I felt I was in a fog and had great difficulty navigating and seeing how a digital comic was put together.

A third turning point was my research on the magic lantern, an entertainment form predating the computer. It gives the term ‘screen-based comics’ added meaning for me. The screen turns out to be a more important parameter than I initially imagined. In formats where the screen is indispensable, it is also uniting. The conclusion that comics on screen can be both digital and analogue (e.g., a drawing on acetate that is projected onto a screen via an overhead projector), and that ‘screen-based comics’ can serve as an overall term, contributes to defining the landscape around the digital comic.

The final turning point I will highlight is Friedberg’s (2006) presentations of sub-screens. This inspired me to create a sub-screen perspective on digital comics, which became a tool that made it simpler to write about my own digital comics. It helped me locate and assign effects and motions to the fictional space, the negative space and even our real space. Since the sub-screen perspective is not limited to digital comics, it can be used when describing all types of comics. It is an especially practical perspective to refer to when discussing digital comics because the different levels of screens can interact with each other.

This chapter has been challenging to incorporate into my artistic research project because it takes a step aside from the primary research and my digital comics to do some fundamental reflection. Nevertheless, I find my conclusions relevant for my research focus, which is to explore concepts of motion in digital comics. Since the chapter does not directly concern the two digital comics I have created, I decided to position it after my presentations of the two digital comics. I see it as a strength of this project that I took time to investigate and define the digital comic; if I had not done so, my perspective and presentation of the concepts of motion would have been very different. Before developing the ideas and reflections included in this chapter, I had used mapping to reveal the different types of motion and presentational forms (Picture 36). My focus was on modulation and how various techniques could be mixed together. This approach turned out to be inadequate for gaining a clear perspective of the digital comic, and I discarded it medio 2014. The drawback of writing this chapter was that it was quite time consuming to do these reflections and to come to the conclusions. If I had had this base from the start, I believe my research on motion in my digital comics could have gone a bit further.

Chapter 4: The Digital Comic

In this chapter I explore a question: What are the fundamental parameters of the digital comic? This is a step aside from my main research question, but the reason for it is that I could not find any discussion of motion’s place in existing digital comic theory. I concluded that I had to investigate this topic myself to be sure that my theoretical understanding and conception of motion in digital comics could be adequate for helping me develop my artistic practice.Reinventing Comics (2000) by Scott McCloud is my main source for theory on digital comics. McCloud describes many aspects of digital comics, but the broadness of his scope also results in brief descriptions that do not offer deeper analysis and insight. He mentions motion, for example, but does not explore it extensively. 

The reflection presented in this chapter has an objective: to find a perspective that makes more room for the kind of digital comics that expand on the traditional formula of static text and images. I want to find the position of motion in the digital comic’s basic structure.

I identify the fundamental parameters of the digital comic and explore them one by one, making references to theory and reflections by Fredrik Strömberg, Thierry Groensteen, Aaron Meskin, Lev Manovic, Daniel M. Goodbrey and Steve Dixon. Towards the end of the chapter I refer to my own experience of making the digital comics Sound of the Aurora (2014) and Close Closer Closest (2016).

Fundamental Parameters

What are the basic parameters of the digital comic? These would be elements that define it; they would be components of such importance that if one is removed, it would cause the digital comic to turn into something else with a different definition. One parameter must be the medium itself, the comic. The comic, however, has to be digitized in its final form to be a digital comic. This brings the computer in as a necessary component. To communicate through the computer, the screen is necessary as well. Until technology reaches the point where computers can communicate directly with our brains, computers will always need a screen of some kind to communicate visual content. This makes the screen an elementary part of the digital comic. The fact that the comic is contained by the computer and shown through a screen facilitates the need of a fourth parameter – access, resulting in interaction. If a digital comic consists of more than a single image (it could also be called a page or frame), a reader would need some control mechanisms to access the rest of the content. Such mechanisms could include a keyboard, a button, a mouse or a touch screen, just to give a few examples. As stated, I call this parameter interaction. I also see a fifth parameter, at least when I study my own digital comics, namely audio. In comics, however, sound is also communicated through text and images shown via the screen. The audial component is therefore not essential or fundamental for the digital comic’s existence. It can be seen as an additional parameter.⁷ With this in mind, are the other four parameters really indispensable?

To begin with, the comic itself is indispensable in the digital comic; it is the art form of our expression. Then we have the screen: if we remove it we will be unable to see and read the comic. If we remove the computer, we will be left with the screen and no container for the digitized comic. Can a comic and a screen work together without a computer? Yes, but it would not be a digital comic. The screen is a much older technology than the computer. The computer has largely replaced other technical devices from the past, as for example the View-Master, the overhead projector, the cinematograph and the magic lantern. The latter is a 400-year -old technology that represents the first projectors to show slideshows with visual narratives. This is sequential art that I believe qualifies as pre-modern comics. The magic lantern tradition shows us that it is possible to communicate as a comic, with a light source, a lens, illustrated slides, foils and silhouettes. The screen’s importance, also that the comic can be presented with other technical solutions than the computer, indicate that digital comics are part of something bigger: I suggest that the screen-based comic, could function as an overall term, since it includes all digital and analogue formats, even a slide for a magic lantern, a carousel projector, a View-Master, the film in a cinematograph or foils laid on an overhead projector. The screen-based comic, however, is not a well-established term. I used it in my master’s thesis Den skjembasete teikneserien (2008) to describe a comic designed for screen viewing, but I did not question why I should use it as a term instead of ‘digital comics’. With this reflection, I now conclude that the term screen-based comics could be used as an umbrella term, of which digital comics is a part.

The fourth parameter, interaction, remains to be challenged. Is interaction just as essential for making a digital comic as the comic itself, the screen and the computer? On one hand, the answer seems to be no. I can imagine a short comic that only needs one image, or one screen (or frame or page) to be read. As such, interaction, in my sense of the term, would be unnecessary. However, reading is a form of interaction too, given how the reader’s eyes navigate through the visual content. With this aspect in mind, I must conclude that interaction is essential. If the comic exceeds a single screen in size, then controls and a user interface are necessary in a digital comic. In my search for a basic understanding of the digital comic as a medium, I will now elaborate further on these four main parameters: the comic, the screen, the computer and interaction. 

Reader control

Mark Waid, in a talk in 2013 called ‘Tools of Change for Publishing’, said that his ‘north star’ when working with new digital formats was reader control. Reader control is related to reading and the reader’s control over the acquisition. Making a parallel to Dixon’s theories in the last section, reading is classified as navigating with eyes and hands. With a traditional printed comic, the reader has full reader control. Nevertheless, when adding time-based media into the mix of components in digital comics, reader control is disturbed or challenged.

This could mean that reader control is not a matter of course in a digital comic. How, then, can it best be understood? I made a scheme after performing Sound of the Aurora, and concluded that three concepts of reader control are relevant for the digital comic:

1.     Full reader control (full interaction)

2.     Partial reader control (partly interactive and automated)

3.     No reader control (full automation)

1.     Full reader control
Full reader control concerns comics where the reader is in full control of navigating through the presented narrative. The reader’s eyes browse the panels, receive the images and perceive the text; the composition of motifs and the relationship of the images create closure in the reader’s mind. The panels, compositions and text create the rhythm and the pace, but the reader controls the tempo and advances at will. This category cannot contain automated time-based media, since such media remove the full control of the reading.

Despite this, I have experienced that programming allows a comic artist to include motion in this category through scroll-activation. I choose to call it ‘interactive animation’ since it can also be used without the action of scrolling. Interactive animation is not automated and played by a video player, but is activated by the reader’s navigational interaction. This is possible thanks to highly sensitive navigation controls such as touch screens and pads, joysticks, scroll-buttons and gyro-technology or sensors that allow motion controls. However, it is also possible to introduce a sense of animations in sequences that use traditional click-and-activate controls. By using cinematic panels, the overlapping images are experienced as ‘in motion’ when the images overlap each other. Cinematic panels contain closure and do not show full motion. (You can read more about this presentation form in chapter 3.)

Examples of digital comics that differ in form but that all have full reader control:
The Long Journey (2013) by Boulet
The Eighth Seal (2013) by Tynion IV and Rock
Batman Year One (publication date unknown) by Miller and Massucchelli  
Hvorfor Ananas heter Ananas (2014) by Jenny Jordahl

2.     Partial reader control
The category of partial reader control includes comic hybrids that mix full reader control and no reader control. These comics use automated animation in the presentation, but they only appear as fragments, or portions, and the reader still must interact to make the story progress. Examples of digital comics that differ in form but that all have partial reader control:

Outside the Box (2002) by Brendan Cahill
Nico and the Sword of Light (2013) by Studio NX and Imaginism Studios
Ovis Ariesaurus Rex (2015) by Fredrik Rysjedal
Metal Gear Solid – Digital Graphic Novel (2006) by Konami
Close, Closer, Closest (2017) by Fredrik Rysjedal

3.     No reader control
No reader control concerns digital comics that are presented in an automated film format, or comics that are performed in front of an audience. Having no control makes the reader a spectator. Film is the first true multimedia (Manovich 2001: 50, 51) because, from its beginning, it has had the ability to present a time-based presentation of a static sequential narrative, for instance a comic. This is why we have the digital comic type called motion comics. In a performance comic, by contrast, a performer plays the role of the reader and portrays the comic to the audience. This scenario actually uses two sets of reader controls. The first performer, the reader, navigates through a comic that can possess all three types of reader controls. Members of the audience have no reader control; they are spectators, not readers. This is the case until the performer invites the spectators to interact. Then they can receive and perceive the information on the screen just as in the case of full reader control, but they still lack control over progress, time and tempo. Since the performer is reading and controlling the digital comic, the audience will most likely associate the experience with a film, because their role is similar to that of screening a film. Many who talk about my performance comic Sound of the Aurora refer to it as a film. Just as with the motion comic, I think it is fine that it is hard to define the work and to experience ambiguity over whether it is a comic or a film. It operates in a liminal zone between film and comics, and I think it may be up to the creator to define whether the intention is for it to be a film or a comic. In light of the comic performance, I now view the traditional reader as a performer as well, given that reading could be seen as an individual personal performance. Examples of digital comics that differ in form but which all lack reader control:

Watchmen the Motion Comic (2008) by Warner Bros
Sound of the Aurora (2014) by Fredrik Rysjedal

In conclusion, I must say that the most important aspect of my reader-control system is the acknowledgement that a digital comic can be automated and have no reader control, but that makers can also concoct various combinations of automation and reader control. Since the norm of comics and digital comics is that the reader should have full control of the acquisition, I think this is an important point, for it adds force to the notion that digital comics can be hybrids as well as pure comics.

Contents: 

Acknowledgements | Introduction | Chapter 1 | Chapter 2 | Chapter 3 | Chapter 4 | Chapter 5 | Reference List

The screen
The second parameter for digital comics is the screen. McCloud (2000: 222) refers to the screen as a window. This is an old metaphor, first used by the Renaissance painter Leon Battista Alberti in his treatise On Painting (1435, book 1 section 19). A window is transparent, and it frames and shows the world on the other side. The frame of the moving image, however, is different from the architectural window and the frame of the painting. The spectators of a film remain immobile, but their point of view may change according to the content of the moving images (Friedberg 2006: 6).

McCloud presents two ways to treat the screen when making digital comics: as a window or as a page (2000: 222). McCloud’s window is actually a mobile window that can scan a larger canvas or move through virtual space exposing the comic panels in all kinds of arrangements. This virtual space he describes as an ‘infinite canvas’ – infinite because it lacks physical limits. He also presents two ways of organizing digital information: by hypertext and spatial models. Hypertext (jumping from one piece of information to another) can link pages and panels, whereas in spatial models, the screen moves either to expose the panels or the fictional world. But this prompts me to ask: Is this theory still relevant today?

I suggest that the page and the window are incorrect metaphors for digital comics. Instead of the basic screen-window metaphor, it is logical for me to think in terms of a ‘fixed window’ and a ‘mobile window’. The page, as I see it, is only one variation of what a fixed window can present. If I were to describe the page metaphor, I would say it is a flat two-dimensional (henceforth 2D) surface with a fixed format. It is presented in a fixed window or as defined surfaces within a mobile window. A page can contain a whole comic, the whole multiframe, or only part of a comic, a hyperframe. With the fixed window, and by using hypertext, I can deploy the panel-delivery presentation technique, where I feed panels into the fixed window, creating mutable panel layouts. I can also change the content within the frame using a technique I call ‘cinematic panels’. This technique changes the content within the panel frame. (I write more about these techniques in chapters 2 and 3.) There are thus three ways of making comics within a fixed window:

1.     The page: a fixed frame with panel layouts, image/images and text/texts.

2.     Panel delivery: animage stream that makes panel after panel appear on the screen. The term is suggested by Daniel M. Goodbrey (ref. interview).

3.     Cinematic panels: a single, fixed comic panel is replaced with overlapping panels. This term for describing an image-stream structure points to the same phenomenon as we find in the moving image/film, thus the word ‘cinematic’. Cinematic panels are as close as the comic can come to the film without using automated animation. Such panels can appear in panel delivery and probably on a page (I have not observed this), but can also function in full screen. ‘Cinematic panels’ is a term I myself have suggested.

Since I see these three ways of making comics within a fixed window, the multiplicity can stand as an argument for not treating ‘the page’ as a general metaphor for how to conceive of the screen in digital comics.

I also want to address McCloud’s theorization of the two ways in which digital information is organized (2000: 231). He uses the term ‘spatial models’ to describe navigation on spatial surfaces, and ‘hypertext’ and ‘hyperlink’ to describe jumping from one piece of information to another. ‘Hyperlink’ was a term which initially seemed irrelevant for my project, because I associated it mostly with databases and labyrinths of non-linear structures. Once I eventually realized how important the image stream was in the digital comic’s structure, I concluded that it was precisely in McCloud’s theory that the image stream belonged. Perhaps not in the way we organize information, as he writes, but in the way we present information. I therefore do not claim there is anything wrong with McCloud’s theory, but that I create a parallel theorization in order to describe phenomena related to visual presentations.

‘Image stream’ is a basic term for describing film and moving images. In an image stream, the images overlap each other, or the transition between them is so fast that they seem to overlap. I would claim that several pages in a digital comic constitute an image stream, but in contrast to film, they have a lower frame rate and have closure in-between the images (this was also mentioned in chapter 1, in the section ‘The Differences between Comics and Film’). The digital page, in contrast to the physical page, never uses the reverse side of the surface. In contrast to physical comics that require a page-turn to read the next page, in an image stream you just jump to the next page. The concept of the image stream also covers what happens in panel delivery and cinematic panels, both in digital and analogue screen-based comics. It is not that ‘hyperlink’ is incorrect terminology, but ‘image stream’ does a better job of describing what happens in a visual and spatial sense.

As a conclusion, there is no doubt that McCloud’s theory is still relevant, but my contribution is to say that it is also important to point out the two forms for presenting visual information on screen, and to see them as equal in status to how we organize other types of screen-based information. Nevertheless, in light of how I observe digital comics today, the umbrella term ‘screen-based comics’ covers both the analogue format and the digital format. I also suggest rephrasing the terminology in point 1, and add an alternative perspective in point 2:

1) The screen in digital comics could be described as ‘a fixed window’ and as a ‘mobile window’.

2) The ways of presenting information on screen could be described as ‘an image stream’ and as ‘spatial models’.

In the first statement, I switch ‘page’ with ‘window’ and focus on the fixed and the mobile properties. According to Alberti’s window metaphor, the screen would be a window in both categories. The second statement is not a re-phrasing but a supplement to McCloud’s way of organizing information. Statements 1) and 2) stand in a conceptual relation: the image stream is related to the fixed window, and the spatial models to the mobile window. These are not fixed forms, however, so can be mixed together, an aspect I will address later in this chapter in the section called ‘The Computer’.

Ann Friedman’s book The Virtual Window (2006) has made me look at comics in an alternative way. Whereas a physical window frames our perceptual view, the window metaphor in a computer’s graphical user-interface refers to sub-screens within the screen of the computer (Ibid., p. 2). The main screen of the computer is traditionally associated with a workbench or desktop, as it is called in the most common operative systems (Microsoft’s Windows and Apple’s iOS). Such a desktop is the space where sub-screens or windows appear, and these sub-screens enable us to access folders, documents, text, images and so on. This structure with multiple windows within windows is also a fitting description of the structure of most comics, whether printed or screen-based. If I see the digital comic from a sub-screen perspective, what would it look like?

When I count the spaces I observe in-between the windows in a digital comic, I find in total four levels of spaces, and three levels of windows. By ‘total’, I mean that different comics can have different sets of levels. The levels I present under are those I find most common:

·       The real world, our space

·       The digital desktop, virtual space (most relevant for webcomics)

·       The negative space, virtual space (where panels are arranged)

·       The fictional space, virtual space

McCloud (2000) introduces the metaphor of the infinite canvas. Where would this concept fit in this picture of levels? The infinite canvas builds on the principle of a numerical representation of images and space, which I will address in the section on the computer. This digital non-physicality opens up for unlimited content. It is often illustrated with a never-ending comic scroll, or an open surface that never ends. But I interpret McCloud as also including networks of hyperlinks, which can include fixed window presentations in addition to mobile window presentations. I also see the infinite canvas as having the ability to be both 2D and 3D. I have previously always thought of the infinite canvas as the negative space with a mobile window, but based on this analysis, I think it is more plausible to think of the infinite canvas as a metaphor for the virtual space in new media. The virtual space will, in my sub-window perspective, include both the negative space and the fictional space.

‘Negative space’ is an established term in the fields of visual art and graphic design, and I find no argument for changing it. The negative space in a digital comic is not just an empty surface; it can also be an empty three-dimensional space. The size of the negative space and its limitations are decisive for what format the digital comic has, just as the size of a printed page in a traditional comic decides the books format.

The four spaces that I have highlighted in this section are divided by sub-screens. Despite the separation, the spaces can interact with each other. The barrier between these sub-screens can be called ‘the fourth wall’, a term from theatre, which describes the concept of the invisible wall that separates the stage from the audience (Scavenius 2007: 277). An example of how the term is ordinarily used is in the idiomatic saying ‘breaking the fourth wall’, which means that fictional reality and our reality interact. A classic example of this is when an actor turns to the camera and speaks directly to the audience. This term is also relevant for comics, where the most common way to break the fourth wall is when characters or objects pop out of the panels. The negative space actually represents an extra space in-between the comic and the reader/audience, and it is something that film and theatre traditionally lack.  

In digital comics, the fourth wall can also be broken on a technical level. An example is scroll activation, where the action of scrolling down a page, a motion running through the panel space, affects and activates motions in the fictional space (Video 29 in chapter 2). Motion controls can also break the fourth wall, as I do in my comic Close, Closer, Closest. There I use motion controls to affects the illustrations and to connect the real world to the fictional space (Video 114). Another idea for how to break the fourth wall, one which I have thus far not found a reason to use, would be to have the comic collect information from Internet or other real-time sources such as GPS. With this strategy, the comic could have the same weather as the weather where the reader is. If it rains, it rains in the comic, if it snows it snows. Real-time information is an aspect Lev Manovich introduced me to through his book The Language of New Media (2001), and I will refer to it again shortly.  

In Sound of the Aurora, I used real-time information in the real space that affected the fictional space. I did this by using a filter. The term ‘filter’ denotes an overlay effect that can be added on top of any layer, also on the screen of a digital comic. These visual effects are used in film and animation for colour manipulation and effects and may also be used in digital comics for the same purpose. I used a filter in Sound of the Aurora when I projected the comic onto a textile; when wind blew across the textile, it created waves in the image.

At this point it is worth mentioning another system for addressing levels in a visual narrative. The concept ‘diegesis’ is well-established in film and narration theory and points to where the narrated events and situations occur (Prince 2003: 20). There are several diegetic levels, but the most relevant in this context are the diegetic, non-diegetic and extra-diegetic levels. In film theory, the non-diegetic level is a level where narrative content that is not present in the fictional reality can exist. This can be sound, text and image. Non-diegetic sound can be a narrator’s voice or a film soundtrack. Examples of non-diegetic text are expositions, titles, and credits. An extra-diegetic level would pertain to content that is not part of any diegesis (Ibid., p. 20). To give an example, in my own comic Close, Closer, Closest, a menu that can pop up whenever the reader likes could be described as extra-diegetic (Video 115).

Should I use the established diegesis rather than the sub-screen map when conceptualizing digital comics? I think both are useful. The concept of diegesis offers a neutral-level system that relates to fictional storytelling. The sub-screen map, on the other hand, is more specific to digital comics and makes it easier to identify, locate and plan the vast operations, properties and formats in a digital comic. The sub-screen map shows the role and position of the infinite canvas in the structure of the digital comic. It defines a space between the fictional reality and our reality, which is the negative space. With these levels defined and the notions of filters and the diegesis, we end up with a terminology fitted to describe my comics on a theoretical level.

Another theorist who has changed my view on digital comics is Lev Manovich. I think his theory in The Language of New Media (2001) continues where McCloud’s Reinventing Comics (2000) stops. Manovich (2001: 95–99) describes four types of screens:

1.     The classic screen

2.     The dynamic screen

3.     The realtime screen

4.     The interactive screen

The classic screen is the traditional static picture. Manovick defines it thus: 

…the existence of another virtual space, another three-dimensional world enclosed by a frame and situated inside our normal space. The frame separates two absolutely different spaces that somehow coexist. (Ibid., p. 95)

This definition corresponds well with the window metaphor and the levels of the digital comic that I described earlier. The second type, the dynamic screen, which can show an image that changes over time, can show motion and spatial movement as we know from film and animation. The third is the interactive screen: you gain access to the content on the screen by using some kind of control mechanism such as a keyboard, mouse, joystick, touch screen or virtual-reality goggles that let you move around in a virtual world and interact with graphical objects. The fourth screen is the realtime screen, and like a live web cam or radar, it captures images that only exist in the present.

I interpret the types of screens Manovich describes as abilities of the modern screens we know today. And since the art form of digital comics is screen-based, I would like to suggest that the four screen types show four possibilities of the visual expression of digital comics:

1.     Static visuals

2.     Dynamic visuals

3.     Interactive visuals

4.     Real time visuals

As I will mention in the next section on the computer, the principles of new media such as modulation make it possible to mix and intertwine these abilities. At the same time, each ability can also mark a main direction in digital comics:

1.     Static visuals: digital comic books such as those sold at Comixology (Video 116).

2.     Dynamic visuals: motion comics (Video 117)

3.     Interactive visuals: hypercomics and game comics (Video 118)

4.     Real time visuals: augmented reality comics (Video 4 in chapter 1)

The variations within each of these and how you combine them lead to diversity in digital comics. I focus on dynamic imagery within digital comics in this artistic research, but all the abilities and directions are relevant and must be considered when I make a digital comic.

Picture 32: What is a digital comic? The limited litterature made it nessesary for me to reflect on this topic to establish a base for my research.
Illustration by Fredrik Rysjedal.

Picture 33: A comic that can be digitized is one of the parameters of the digital comic. This frame is from Close, Closer, Closest.
Illustration by Fredrik Rysjedal.

Video 112: At an art residence at Tegnerforbundet in Oslo in 2015 I experimented with interactive animation, where the controls where the touch screen. This was experimentation in advance of the work on Close, Closer, Closest.
Video by Fredrik Rysjedal.

Figure 7: In this sub-screen map the levels of the digital comic are arranged. The amount of levels can vary from comic to comic, however this visual example is relevant for webcomics.
Figure made by Fredrik Rysjedal.

Picture 34: When performing Sound of the Aurora, I, the performer, have reader control. The audience on the other side have no reader control. This photo is from my performance in Mariakirken i Bergen (St. Mary's Church) 2016. From left: Sigbjørn Apeland, Nils Økland, Fredrik Rysjedal and Aslak Helgesen.
Photo by Ben Speck.

Video 111: A View-Master is a screen formed as a goggle, a stereoscope that can present 3D images. The slide shows of a View-Master can remind of a comic sequence, but could it be defined as a comic? The slide show in this video is based on Transformers the Movie (1986) and even got juxtaposed panels.
Video by Fredrik Rysjedal.

Video 114: The reader affect the graphics. 
Video by Fredrik Rysjedal.

Video 115: The menu in Close, Closer, Closest could be described as extra-diegetic. 
Video by Fredrik Rysjedal.

Video 116: This video show a digital comic with static visuals. Satania (2017) is made by Fabien Vehlmann and Kerascoet, and published as a digital comic book by NBM at Comixology. In the settings of Comixology's reading app it is possible to activate animations that makes the pages slide in and out. 
Video by Fredrik Rysjedal.

Picture 33: A frame from the digital comic Sound of the Aurora the 2D edition. 
Illustration made by Fredrik Rysjedal.

Picture 35: In the period 2013–2014, I used mapping as a method and wrote down aspects and observations from my experiments. This research guided me in my search for relevant litterature.
Photo by Fredrik Rysjedal.

Video 113: The screen of Sound of the Aurora is originally a projection on a dynamic canvas that can make waves in the image. 
Video by Fredrik Rysjedal.