Software Aesthetics and Representations

Figures 19 and 20 might also be considered worthy of analysis in themselves; a well-designed software interface has beauty, particularly when enhanced with contextual information. Figure 19 reveals a mass of information including the duration of the audiovisual data file, the quality of the scan, and clues to the analysis. It also offers an additional surveillance gaze as the current viewer may note the time and date of the analysis; the operating system and software can be determined and assumptions can be made about the type of computer being used. Presenting OTE’s video stills within screenshots from the Tobii Pro Glasses 2 software system further complicates the data recursions and patterns of gazes. Figure 20 highlights this: a viewer can see three faces in each of the embedded screens; the performer in Telephone, as well as the reflection of the ‘participant’ watching the video, and the reflection of Svetlana in the background watching the participant’s viewing behaviour.The inclusion of the software imagery tells us that this whole drama is seen by an unknown analyst, creating an additional gaze and illustrating the recursive nature of data capture by reflecting back the gazer’s gaze in complex and unpredictable ways.

The re-enactment video and its associated stills constitute the artist’s performance, which mimics the consenting participants’ viewing behaviour. But how authentically does the re-enactment reproduce the participant experience? The re-enactment video, and to a lesser extent its photographic stills, aesthetically reflect the first-person perspective of a head-mounted camera such as a GoPro. Recent developments in citizen journalism that have led to the dissemination of eyewitness footage of events, and the proliferation of affordable wearable cameras that can be used at home to capture personal adventures, add to photographic conventions that suggest immediacy, authenticity, and intimacy. In most image-based media, the gaze of the camera stands in for the viewer; in the case of OTE’s re-enactment video, a viewer is literally put in the place of the participant, while a voice from an unseen mouth speaks for the participant, as if we are really there, experiencing Svetlana’s performance first-hand. The re-enactment video therefore achieves a sense of intimacy and authenticity by employing the conventions of reality effects, such as apparent proximity to the action, and unsteady footage that appears to be caught in the moment rather than carefully staged.

The live art installation of OTE might have instead asked participants to use a wearable camera while watching Telephone in order to capture audiovisual material that felt authentic and immediate. Such a work, however, would not have achieved the impact of OTE’s performance of science, in which we see Svetlana’s authority partly residing in her eye-tracking glasses, a sophisticated technology decidedly not for at home use.[18] OTE employs the superficial power of the appearance of complex technology to achieve its effects, and the complex para/social interactions between participants and Svetlana largely reveal an implicit trust in science and technology as neutral and beneficial. On the other hand, however, OTE aims to reveal the construction of image media and, consequently, subvert the power of eye-tracking technology. 

In OTE, there is no objective way to measure how well the re-enactment reproduces the participant experience in this project — a loss of power that reflects the theme of the work. We can also link this loss to Berger’s argument in Ways of Seeing, which is itself based on a reading of Benjamin’s ‘The Work of Art in the Age of Mechanical Reproduction’ (1936). Both scholars argue that the aura of an authentic original image is diminished through its reproduction, and any subsequent attempt to again reach that original is entirely futile.[19] In OTE, as in the online world, there is no original; the work was repeatedly performed for audiences of one or two in an art gallery setting. OTE was also performed when the artist re-enacted the visitor experience, who used the eye-tracking glasses as a camera. The experience of watching the re-enactment video includes Svetlana’s self-conscious performance, in addition to viewing Telephone via the head-mounted eye-tracking camera. OTE offers a ‘language of images’, in Berger’s terms, or a recursive drama of looks.[20] Viewers of this video and its stills have no choice but to trust that the artist is reproducing the participant experience accurately. 

But authenticity is hardly the point. The audiovisual documents that resulted from the re-enactment are qualitatively different from those that would have resulted from using a wearable camera. A wearable camera only shows the general direction of a wearer’s head, while data-rich eye-tracking systems show what the wearer is actually attending to. The fact that the Tobii Pro Glasses 2 system is not something that one can use at home leads to different viewing behaviour and different representations of viewing behaviour. Further, eye-tracking glasses are also different from their cousin-technology, desktop eye tracking, in which the software both presents the image and tracks the eyes. As such, there is a direct connection between the image on the screen and the location of the eyes. But eye-tracking glasses record the location of the eyes relative to the glasses themselves while also recording the field of view from a scene camera on the front of the device. The glasses are, in a sense, unaware of what the wearer is looking at. 

In order to create heat maps and gaze plots, a representative image is captured from the scene camera — called a snapshot — onto which the data is mapped. The software generates a new coordinate system and attempts to line up the eye-tracking data with the snapshot based on height, length and depth information captured during recording and the feed from the camera. Figure 15 shows a snapshot with data heat-mapped onto it in the conventional colours; Figure 16 is the same snapshot with the heat map appearing in arbitrarily different colours. Depending on the scene, multiple snapshots may be needed to capture the events. In a heat map, areas where the gaze is less frequent eventually disappear, and this disappearing act can be seen when a gaze plot and a heat map are viewed side by side, as in Figure 17.  Figure 18 returns to heat map visualisation but with less concentration, meaning that the heat map acts as a gaze plot. The images are not raw or simple snapshots of the action, but complex constructs.