Interrupts and Intervention

By Bjarni Gunnarsson

Computing today is increasingly oriented towards real-time interaction, networks, distributed systems and adaptive algorithms. Using generative computer processes when creating music raises questions of how to model compositional procedures that allow for openness, interaction and malleable organisation of time. Instead of employing predefined algorithms during composition, processes should be designed to adapt to or evolve within such a situation. The framing of decision-making is essential to the design of such processes.

An important consideration is how to represent a generative algorithm as distributed (open), with multiple entry points instead of being closed and result-oriented only. How to introduce dynamic modifications of goals (or heuristics) as a creative interaction mode or how the attitude of evolving criteria (variable intentions) can be made audible through the characteristics of the corresponding sound processes. The main idea is to separate the scheduling and producing parts of generative algorithms. To balance the influence of processes that take a certain time to manifest themselves and their activation. This involves making the algorithms block-based so that different areas of their evolution can be accessed freely. It also makes use of adaptive heuristics and interrupt-based scheduling.

This exposition forms part of my research on the positioning of algorithms for composing computer music made at the The Royal Conservatoire and Institute of Sonology in The Hague from late 2017 to early 2019. Further research on the topic can be found in my work on the context and scope of generative algorithms within compositional environments.

Presented below is a software framework, an interactive, container-based, process scheduler which will be abbreviated in this text as CPSI. It provides an integrated approach to interaction and scheduling of generative processes. CPSI has three main parts: A scheduler, an interaction module and a process container. CPSI can be programmed to run without any user intervention but also to operate within a real-time situation.

The CPSI framework offers several important functionalities:

• Dynamic containers that encapsulate algorithms, data and temporal states
• Block-division of input processes that facilitate interactive scheduling
• Direct and adaptive interactions algorithms for gradual and immediate causation
• Real-time reconfiguration of heuristics and process scheduling
• Parameter transfer from automated procedures to manual intervention
• Interrupt-based scheduling of partial algorithms with scheduling actions
• Parallel schedulers with scripting facilities and interoperability support
• This text will proceed by presenting ideas of container-based algorithms and adaptive heuristics. A discussion of interaction and interrupts will follow and finally, the software framework will be presented and the research results evaluated.

Algorithms

Goal orientation

Generative composition programs represent a mediation between predetermined ideas and practical realizations. During the conversion from intention to execution, a translation takes place that can result in certain choices being established in order to express musical processes. This fixation takes place through algorithms, descriptive processes that are usually expressed with enumerated steps that precisely notate how to advance from one state to the next. Algorithms are usually measured in terms of efficiency, clarity, abstraction, and reproducibility. Their capacity is often obtained by computing how they respond to the use of time and space in terms of resources and duration. In fact, if an algorithm performs strongly with regards to these criteria, the details of exactly how it proceeds is considered less important (given the fact that it will produce a satisfactory result).

An important factor is how the quality of a result has to fit an original intention, a factor that has influenced other choices made during design and implementation. Algorithms are therefore usually goal-oriented. Designed to deliver a certain output and expected “to render something unambiguous” [1]. Algorithms represent abstract solutions to well-defined problems. They are ideal, finite, pure and isolated from any environmental complexity. However, abstractions often tend to neglect details or residues that can in subtle ways be considered important to the domain being modelled. The framing (or implementation) of a generative process to a working program involves decision making and alignment of interest that has a strong influence on their technical affordance [2]. Instead of focusing on fixed metrics, algorithmic modelling should be something that takes shape during the compositional process but not prior to it.

Black Boxes (choosing to ignore)

Algorithms and abstractions exclude certain implementation details when exposing their functionality. What matters is the input and output of a given process “a device whose functionality we understand, but whose inner workings we don’t, or choose to ignore.” [3]. In such a working mode, inputs are given to a black-box, it runs and returns results. How a black box operates is an unknown mystery and only accessible within the box itself. The structure of the system does not change while it runs or at any moment once it is being used. Neglecting implementation details can introduce important problems regarding the relationship between a concretized process and the external factors that have initiated it. How can mutual influence, dialogue or exchange take place only through the input and output phases of a running process? How can a change of behavior be introduced within such a model?

Discussing his work programs for the IBM 7090 computer, Iannis Xenakis mentions the idea of “la structure abstraite” [4]. A program that consists of formulas and reasonings but that does not change in itself. Xenakis mentions strategies for varying the input to a black box, how the structure of a composition can be changed (not the stochastic program) and the importance of manual modification of the output. He also mentions how such a static program can be used to generate families of pieces. However, the “boîte noire” stays static and all other operations are based on this fact. It takes a certain time for a process to manifest itself. To express its time-varying qualities.

How a process is configured in relation to other structural functions could be understood as simply being composition itself. An example of this can be found in the electronic works of Roland Kayn where the focus is on the creation of interconnected electronic modules and the sonic relationships that emerge from a certain system design. The control is mostly delegated to the system itself bringing forth a question of acceptance: “The composer is entirely divested of his original function. He can merely decide whether to intervene, guide, or direct, or whether he is prepared to accept what emerges as an auto-generative procedure.” [5] Such distances introduce an attitude of letting go, of engaging in something that unfolds or “in which autonomous computational processes or models are set in motion, generally free of interaction.” [6]. This point of view creates a clear distinction between the process, that is to be respected, and the environment from which it emerges. Internal laws and causation condition the becoming and the development of pre-planned actions.

Transparent modes of production emphasize a certain clarity but also an abstraction that remains fixed, or de-coupled with regards to a dynamic sensitivity of process discourse and creative exchange. What is interesting in these examples is how the fact of not being able to interact with certain aspects of a generative process is considered essential to the final result. That the technical limitations are considered important for the artistic outcome. A questioning takes place regarding the ‘acceptance’ of what a machine gives. This introduces a certain kind of ‘letting-go’, where modes of interaction vary between command and autonomy [7].

Rules and heuristics

An alternative to static rules and fixed structure can be found in the usage of heuristics. Instead of progressing towards a known goal, heuristic methods explore uncertainty and the search for objects whose contents are unknown but where a certain criteria is given [8]. Situating algorithmic processes as being shaped within a creative process, heuristics allow for real-time evaluation of a chosen direction. Actions are performed that are then “neither blind nor completely determined, but more like educated guesses” [9].

An adaptive heuristic is usually effective in a dynamic setting where a simple “rule of thumbs” heuristic can be applied that leads to movement (change) in seemingly plausible directions. The adaptivity occurs if heuristics can react in relation to what is taking place in the setting at any given moment [10]. The term originates from the Greek word heuriskein [11] meaning to find or discover. It could be understood as the process of trying by example rather than following a preconceived plan. Heuristic functions always result in approximations, “good enough” solutions [12] that are subject to change. Such lively qualities fit well the permanently fluctuating sound processes implemented within CPSI as discussed below.

Interaction

Influence

Observing evolving, generative processes from an outside perspective introduce a certain distance. Tendencies of hierarchy, imposed control or interactions that reflect, comment or advise on something that already has an acquired form. In CPSI, a model of algorithmic containers is introduced that addresses running algorithms and their modes of interaction. The focus is on the behavioral dynamics of such methods, allowing for two categories of operation.

External influence concerns direct intervention, guidance, the taking over of a certain control, splitting it or simply disturbing an active algorithmic process. Such interaction is usually one-directional where the system reacts to a provided input. Broadening such a connection, it is possible to extend the idea towards internal influence where processes react, adapt, respond or engage. This occurs in a conversational and cooperative exchange where the output is based on cumulative interaction and not a singular one [13].

In CPSI, processes are assigned a certain direction, a heuristic towards which they aim to direct themselves. This may be assessed by parametric balance, proportions or known goals. By keeping the computability of such a function accessible while a process runs, another option for interaction is introduced by evolving endpoints.

Dynamic goal setting reflects a compositional process based on discovery. It also emphasizes a two-way, responsive relation where feedback and change occur between the system and its interacting agent, a coupling through interaction that results in a mutual influence. Through such exchange, it should be clear that the purpose of the system is not simply to act on a composer's intention but also to contribute to its inception. Responses occurring as a result of interaction can have multiple causal effects in a transformative situation. For example an immediate, direct response but also a developing or evolving effect that contributes more to behavioral changes in an ongoing process.

The duality of the direct and the ongoing is a central element in CPSI’s scheduling schematics where direct, micro-processes are coupled with longer, ongoing ones to create different situations of response. Special attention is given to external influences of processes that evolve over time. How does interaction occur within a movement that requires a substantial duration to manifest itself? Instead of extensive control, strong influence can be more engaging [14]. This can be achieved through a network of weighted influence-parameters but also, by adaptive interaction based on gradual behavioral changes as implemented by the CPSI interaction structures.

Taking Over

In CPSI, algorithms are distributed to several parts, each of which has its own modes of interaction. When selecting process sections or partial specifications [15] for further engagement, one can design a formative frame of reference, a contextual view of agency. The separation of data and process and the focus of interaction and reconfiguration through time allows for a reiteration of algorithmic parts in real-time. Inputs can be modified and rules altered while a process runs. This also emphasizes the temporal side of interacting and evolving processes. Perhaps a truly interactive algorithm is developed in dialogue through accreditation where the lack of a clear starting state or default pathway [16].

Andy Lippman understands interactivity as a mutual and simultaneous activity where agents might have similar goals. He determines important factors to include a ‘limited look ahead’ and the absence of any default programmed routes [17]. Systems that improve or redefine themselves through interaction and adaptive heuristics. An attitude of tuning computational processes in real-time against any pre-computing of potential paths but also through interrupts or turn-taking as points of departure. In CPSI, autonomy can be halted through a shared agency and by the taking-over of process part. This introduces a methodology situated “between the formal and the craftsmanship” [18] where processes start and stop and their isolated parts can be further treated and coupled with external intervention.

Interrupts and Scheduling

Algorithms that instantiate processes, cause artefacts and commit to linear (perhaps predictable) progress are often plausible as subjects of radical causality such as the rapid cutting (in and out) of their original context. Given an input or initial condition, an algorithmic process can be set to motion, interrupted, forced, blocked or disturbed. It is possible to understand machines as systems of interrupting flows, where the cutting of one ensures the continuity of another [19]. An interrupt can translate to a very direct causation or disturbance of an ongoing movement. It reflects a well-known model of allocating computation to running processes where it is used to enter a process during the time it is running. “The interrupt not only creates a break in the temporal step-by-step processing of an algorithm but also creates an opening in its ‘operational space.’“ [20]

Applying interrupts to generative algorithms introduces complexity in a causal chain where the inner program is not sufficient in itself but must be coupled with external influence that decides how and if it should be allowed to run. The interrupt is in a certain sense a condition of cooperation. It makes way for the many instead of the one and holds a special position towards algorithms and their completeness since it possesses the ability to force them to halt.

What value offers an algorithm that stops before completing, at the expense of another activated as consequence? Could such modes of interaction provide ways of engaging with unfolding musical processes? In CPSI we encounter a conflict of methodologies for operating sound process. On the one hand, we have the open, partial algorithm made accessible through various ways of interaction and that by design should be refined as it evolves. On the other hand, we treat such processes through external forces, through interruptions that have a direct and straight effect based on a well-established hierarchy. Processes are started, completely stopped, resumed, halted to a point or treated through interaction and the taking-over of a given algorithmic state.

Interrupts offer a dynamic way of balancing autonomy and command through the interaction with generative processes. Interrupting a flow of events also introduces anticipation patterns, hierarchy (what can be interrupted and how the interrupt takes place) as well as questions of how interrupts affect (or break) silence [21].

System

CPSI is implemented in the SuperCollider environment [22]. The system consists of distributed parts that relate to a central component, the scheduler responsible for the arrangement of all the container-bounded algorithms. The architecture has been constructed to support an interactive working-mode focusing on scheduling partial processes with gradual and direct interaction.

Partial Processes

Processes treated by the CPSI scheduler are implemented as SuperCollider patterns. Patterns offer higher-level representations (or blueprints) of musical tasks and can be understood as “abstract representations of sequences independent of any specific performance.” [23]. The processes follow a partial construction pattern that allows them to be treated as separate blocks. The methods are segmented by design in order to facilitate scheduling and to support multiple access points. This imposes constraints on the algorithms supported within CPSI but also enhances certain behaviors. Processes in motion, supporting dynamic, uncertain behaviors, full of details but without any fixed goal are suitable in this context.

Each block represents a certain state of an algorithmic process that can in itself be further developed. These processes combine both sound and control and in many cases, the two overlap to a great extent.

Various methods of event generation have been implemented using stochastic functions, iterative equations, and state machines. Synthesis methods (mostly time-domain based) include pulsar synthesis, waveset distortions, noise and chaos functions as well as various types of granular synthesis. These will not be discussed further, what is most important in relation to the scheduling are their process-based qualities. To always produce data and continuously evolve, to contain different states while being capable of running on their own (and forever) until being subject to a scheduling action.

Adaptive Heuristics

Each partial process in CPSI produces data according to a given heuristic function. The processes exhibit a tendency towards satisfying this heuristic and such guidance can be altered while a process is running. In CPSI, a library of evaluation function has been implemented that can be applied to various aspects of an algorithm such as the evaluation of pitch, duration or dynamics. Functions exist that favor scarcity, availability, primacy or recency [24] of values, each with their special effect on a running process. Instead of being domain-specific or context-dependent, the CPSI heuristics are designed as general and important by the fact that they can be applied on the fly and adapt while the generative aspects continue.

A given algorithm is realised by sequentially scheduling each of the blocks it consists of. However, once activated a certain block can also be repeated, extended or transformed for further development. The block sequence can be altered and the output increasingly treated through filtering, mapping or repetition of generated events. This ‘taking over’ of a running process or algorithmic state enables an interaction that zooms into the temporal windows of operation.

Direct Actions and Gradual Influences

Two main categories of interaction are supported in CPSI. Direct actions and gradual influences. These reflect a certain attitude towards generative processes where a direct action (10 - 200ms) causes an immediate (and often forceful) change but a gradual mode (> 1000ms) proposes more of a conversational mode, employing a higher focus on behavioral changes.

Gradual influence has been implemented here through process changes such as the responsiveness and adaptation of given variables but also by behavioral parameters such as entropy or density. While a process is executing, its data or heuristics can also be modified with important creative consequences.

Scheduler

Besides providing a framework for accessing generative algorithms through interaction, CPSI contains a scheduler for activating them in various ways. The scheduler encapsulates important behavioural aspects for an algorithm that is executed, scheduled, stopped and subjected to ongoing and direct influences. The model consists of three main components, the process queue, the scheduler and the scheduling actions.

Once activated, the scheduler handles items from the process queue. Those are usually blocks from a larger algorithm that can be treated individually or in sequence. Once a block has completed, the scheduler proceeds by handling the preceding item in the queue.

At any point, an interrupt can be made. The interrupt halts the ongoing process and either activates another block or silently suspends. During an interrupt, special actions can be put in motion. These can be of any kind but are usually short sequences of sounds intended to articulate a certain interruption or change of events. Moreover, the interrupt actions can be chained to further extend the causality effects of an intervention. An interrupt can also be connected to post-processing that affects the block that is interrupted. For example, halting a rhythmic pattern can be executed by sharply applying a filter removing a substantial part of the frequency spectrum. These transformations evoke imprints of interference where “disturbance intercepts and reshapes the signal during interaction” [25]. Executing actions and post-processing procedures enables creative musical performance using the scheduler.

The scheduler can also be programmed through an event-list and as such execute algorithmic processes. Various modes of extensions are possible beyond the basic scheduling. It is possible, for example, to use the adaptive heuristics belonging to a certain process to determine their order in a process queue. This allows for mixing methods in creative ways based on a given heuristic function. Another option is to run multiple schedulers at the same time each with a slightly different mode of functionality or interaction. Finally, the scheduler implements the same interface as the algorithmic containers. This means a nested approach is possible that allows one scheduler to activate another one, forming more complex causality chains.

An example of a scheduling situation might be as follows: Let the scheduler S contain a set of blocks B = {B1,...,Bn} where a single block can be expressed as a bundle B _i : < p^b ,d^b ,i^b ,h^b ,a^b ,n^b > with the attributes:

● p^b, pattern to be played
● d^b, block duration
● i^b, interaction interface
● h^b, heuristic function
● a^b, interrupt action
● n^b, unique identifier

S supports four main scheduling operations:
● schedule(S, b, i) schedule block by index
● reorder(S, b, n) change block order to n
● order(S) change order according to heuristic
● cancel(S, b) cancel a block

S supports six main intervention methods:
● interrupt(S, b) activate interruption
● interrupta(S, b, a) interrupt with action
● interruptp(S, b, p) processing interrupt
● interact(S, b) transfer process to interaction
● heuristic(S,b) alter heuristic function
● write(S,b) alter parameter data while processing

Scheduling and intervention activities can take place in parallel and/or in sequence. These combine in forming an output stream that can be both automated but, through intervention, also be fully manual. Two main algorithms run while the scheduler operates.
1. The processing of the queue items or partial processes (blocks) that have been added to the queue.
2. The conversion of the block data to sounds streams by processing the queue items.
These are listed in a simplified format here below.

              Algorithm 1: PROCESS QUEUE (S) 
Input:  S
while  S is not empty:
 while  items in input (S, b) do:  set  job j = s  chedule (S, b, i)  if   process_halted (j):
interrupt(S, b)
 if   process_transfer (j):
interact(S, b)  wait (i.duration)  order (S)
i = i + 1  end while wait(S)  end while
            

              Algorithm 2: PROCESS ITEM (b) 
Input:  (S,b) while  b.duration
for each  pattern  p  in b:  set  e =  new_event ()
set  h =  heuristic (b)  while  has_values (p) do:
compose_event (e, h, p.value)  process_event (e)  wait (e.duration)
end while  end while
            

Conclusions

This article has presented interrupts as a compositional and performative tool in the context of generative algorithms and event scheduling. The interrupt operates on partial, dynamic processes that use adaptive heuristics while running. Direct action, process take-over, and gradual influence were presented as possible modes of interaction. A software framework, CPSI, was introduced that implements the concepts that have been covered and its software architecture was demonstrated. The CPSI scheduler, its algorithmic containers and object ecosystem offer a new perspective for representing and interacting with generative processes in an open and direct manner.

CPSI has been used by the author for the creation of a live performance (Vicinity Pulse, 2017) and the composition of 2 x 8-channel fixed media piece (Prisme, 2018 and Epicycle, 2017). In both cases, the scheduling effects are dominant where different processes get ‘activated’ and ‘interrupted’ contributing to a progression of the music. The framework offers many original viewpoints in terms of addressing algorithms and dealing with evolving criteria and uncertainty.