Unlike hex editing or live camera datamoshing apps, we would be dealing with a set of unknowns as we attempted to datamosh the incoming digital broadcast transmission; for example, we would not have control over how the digital broadcast was being encoded/compressed on the transmission side, nor any knowledge of the codec or software language utilised by the converter box.^[1] With the various methods of early video artists and contemporary datamoshing artists in mind, we decided to explore three diverse techniques to disrupt the digital broadcast signal before it reached the decoder: software hacking the box’s built-in code; circuit-bending or hardware hacking the circuitry of the box; and external interruption of the electrical components that were hardwired to the box. We also considered that it might take a combination of all three methods to achieve datamoshing. We experimented with each technique to different degrees of success.

In the following, we outline these processes and the key findings discovered through our research. Central to our process was understanding that we are not computer software engineers or technologists. Instead, we approached datamoshing as a process of artistic discovery; inspired by a specific visual or aesthetic result, we would identify and categorise it, reflect on how it was achieved technically, and experiment with other tools and methods that might offer further enhancement. We then applied our knowledge as media practitioners to both the hardware and software components of the digital-to-analog converter box to achieve datamoshing of the broadcast signal in real time.

BOX DISASSEMBLY

Our first step was to disassemble the assorted converter boxes we had collected. We began by examining three different boxes manufactured by Digital Stream, RCA, and Zenith. One of the first things that we noticed was that the components of each box were soldered to their respective circuit boards, making the replacement of any one part a virtual impossibility. Using serial numbers and, when available, the manufacturer’s name, we searched in vain for service manuals or other information that could shed more light on the various components. We did, however, discover that all three boxes utilised a Thompson TV tuner, the electrical component that receives digital broadcast signals from an external antenna before they move through box to be decoded and converted to analog signals. Our initial external manipulations of the converter box’s electrical components would focus on the TV tuner.

EXTERNAL INTERRUPTION OF THE ELECTRICAL COMPONETS

We began our investigation by employing a series of simple interruptions to TV Tuner of the Digital Stream converter box. We found that datamoshing of the incoming signal occurred when we simply tapped the TV tuner, causing obvious characteristic effects, namely, the painterly-like smearing of the image and pixel rupture.

This led us to theorize that the application of low frequency interruptions to the TV tuner would cause datamoshing. Frequency is the rate at which a vibration occurs and is most often measured in hertz units (abbreviated Hz). Tapping is a low-frequency vibration that does not carry much data. Conversely, the incoming DTV signal runs through the TV tuner at an extremely high-frequency rate, carrying multiple channels of data.^[2] High- frequency waveforms have a faster rise and fall rate than their low-frequency counterparts. Because low frequencies have a slower rise and fall rate, when they are introduced to the high-frequency digital television stream they offset its timing and gaps in video frames emerge before the timing system can find them – producing an effect that is akin to the dropping of I-Frames in hex editing. When we introduced low-frequency signals by tapping, we offset the timing signatures of individual video frames as they were entering the TV tuner and effectively forced the decoder to resolve the image onto a dataset that did not match, resulting in a datamoshing effect in real-time.

To test this theory, we applied consistent vibrations to the TV tuner using a simple battery-powered rotating-head toothbrush. Rotating-head toothbrushes typically generate between 1,300 to 8,800 oscillations (or rotations) per minute, producing a low frequency  that ranges between 21.67 and 146.67 Hertz. The low frequency vibrations produced by the toothbrush produced significant datamoshing; bright over-saturated colors, smearing, pixel rupture and tearing of the image were obvious as the incoming high frequency broadcast signal was interrupted and frames of video collided. 

That datamoshing of the digital broadcast would take place without hacking or modifying of the converter box’s software was a significant discovery; in other words, datamoshing occurred at the level of the signal rather than the code. Low frequency has a long history of being utilized by artists to interrupt the electromagnetic waves that carry the broadcast signal. For example, the fans that Nam June Paik utilized to create interference with the analog broadcast signal introduced low frequency disruption. In analog video synthesizers, low frequencies (between 1–1200 Hz) also traditionally cause the most significant image modulations. Nam June Paik’s Wobbulator, for example, produces the most obvious abstract and geometric patterns and distortions at frequencies ranging between 60 and 90 Hz (Miller Hocking, Brewster, Wright “Raster Manipulation Unit: Operation and Construction”). Our investigation into the interruption of the electrical components via vibrations revealed that frequency remains a unique behavior and material for manipulation within the digital broadcast transmission. 

However, once low frequencies were introduced to the TV tuner and datamoshing occurred, the converter box would read the incoming signal as weak, issue a weak signal warning on-screen and shut-down. While frequency was the key to datamoshing the incoming broadcast in real-time, the code still controlled how the incoming signal interacted with the box. We needed to find a way to override shutdown to enable real-time datamoshing to take place in a sustained and continuous fashion that we could exercise control over.

Via online searches, we found that the Digital Stream converter box utilized MicroController (MicroC/OS, or µC/OS) Real-Time Operating System (RTOS) which defines functions in C, each of which can be executed as an independent thread or task. Using a terminal-style program that provided us with text-based access to the television tuner’s MicroC/OS system, we examined the various commands that the box would execute when strong and weak signals were being received. A string of code— “Periodic Signal Check” —would be listed when “good video,” or a strong signal, was being received. We also noticed that when we disrupted the signal too much that a “bad video” notification would appear, and both audio and image would turn-off and attempt to resync. This led us to speculate that if we could turn off the converter’s ability to resync “bad video” that we would be able to receive a continuous image no matter how much we disrupted the signal.

To override the resync, we hypothesised that the SigMon (Signal Monitor) command might be key. When we sent a ‘Stop SigMon’ command, the whole device shut down. This led us to examine a series of sub-tasks and discover another piece of code — ‘App_Pause_SMTask’ — that, when typed into the terminal, paused signal monitoring rather than shutting it down, which kept the converter from trying to resync ‘bad video’. We could now disrupt the incoming broadcast and not lose signal. This was a significant breakthrough; once the weak signal warning was paused, the signal became a material that we could freely explore and manipulate.

DATAMOSHING THE JOY OF PAINTING

To test our new software hack, we decided to datamosh an incoming television program in real time using our rotating-head toothbrush. A local PBS station was airing The Joy of Painting, allowing for a certain synchronicity with the painterly modifications that datamoshing applied to the incoming broadcast image. This datamoshing experiment was, in turn, live streamed via Facebook Live so that we could share our initial results with a larger audience and, in the spirit of live video synthesising projects like Nam June Paik’s Video Commune (Beatles from Beginning to End), which was broadcast on WGBH Boston public television in 1970, explore the potential of a modified converter box as an instrument for live performance.

CIRCUIT-BENDING: CREATING A RESPONSIVE TOOL

While the vibrations created by tapping and the rotating toothbrush were key to understanding how low-frequency interruptions could datamosh the incoming digital broadcast signal, they did not allow for any control over the resulting effects. We wanted to create a more variable and responsive system that would allow artists to bring in a range of frequencies and effects over time, rather than just apply vibrations in an on-and-off sequence. Our goal was to create a more variable system that would enable artists to control the amount of frequency being applied to the image — to increase or decrease datamoshing effects — and apply datamoshing not just as a visual effect but as an aesthetic tool that could generate meaning through responsive manipulation of the image in real time.

Once we had resolved the weak signal issue, we moved on to more advanced hardware hacking, or circuit bending, experimenting with resistors, capacitors, and potentiometers to create a variable system. Resistors and capacitors, much like the electronic toothbrush, could create low-frequency interruptions in the digital video stream. Resistors are electronic components that limit the flow of voltage in an electric current. Capacitors hold a direct current charge and allow an alternating current to pass, creating a low-frequency pulse. Potentiometers, also known as slide pots or slider pots, are variable resistors that can sweep between a range of values. We tested the effects of various resistors, capacitors, and potentiometers on the 5-pin antenna strip on the circuit board of the Digital Stream converter box. The 5-pin input strip splits the various components of an incoming broadcast signal into manageable parts as it enters the converter box: video, audio, voltage, and ground loops.

Circuit-bending is often described by its practitioners as a random or chance-based process. As our own experimentation illustrates, it is also a highly-aestheticised practice that can produce a wide range of visual modifications. As we applied the various electronic components to the antenna strip, we attempted to describe, categorise, and catalogue the visual effects we were achieving. We identified a range of visual ephemera including colour cycling, pixel shifting, and colour saturation, some effects identified with datamoshing and I-frame distortions, others introduced by low-frequency disruption of the signal. As we applied various electronic components to the antenna input strip, we found that the most datamoshing-like effects — pixel rupture, smearing of the image, or colour saturation — occurred when a 16k resistor was placed into a 50k potentiometer range. The potentiometer essentially worked like a volume-control knob, allowing us to bring the frequency up or down and control the application of the datamoshing effect, thus creating a variable system with a high measure of control.

During the circuit-bending process, we were also able to explore datamoshing at a deeper level than either hex editing or automated apps could allow for. For example, we found that datamoshing of the image would occur when we applied resistors to the audio pin located on antennae input strip. Because audio and video signals are split into separate components as they enter the converter box, we were able to control and modulate audio separately from video — an impossibility in hex editing. Applying low-frequency resistors to the audio pin allowed for the same ripping, saturation, and pixel rupture of the image as video datamoshing. The ability to disturb the incoming sound and utilise it for datamoshing was an exciting, if unexpected, development.

MORE SOFTWARE HACKS

Since we weren’t versed in MicroC/OS, we decided to focus our research on datamoshing effects that could be achieved through hardware hacking and circuit bending. However, we also were able to identify some visual modifications that could be created through software hacking, including one that applied what can best be a defined as a stereoscopic effect to the broadcast image (see video). Software hacking of the MicroC/OS is an area rich for future investigation. Our research points to the artistic potential of software modifications to visually alter and modify incoming broadcast signals using a range of effects in either in concert with, or independent from, datamoshing.