{JR, 180921}

Thanks HH, for inviting me into your configuration and into your thinking processes. :-)

Where to start.. well, as I told you over our last voip conversation, I'm at an inflection point in my artistic practice. I was teaching at Leuphana University for nearly three years, helping to set up the practice-based part of a new international bachelors study in digital media. This was an undoubtably enriching experience, where I had the opportunity to meet a number of brilliant colleagues (now friends) and truly engaged students. It was also a physically and mentally exhausting period, commuting between The Hague and Hamburg almost weekly, compiling class plans in airports and at 4am in the AirBnB of the week. Not to mention this being my first full-time university job, which comes with its own strange politics in the German University system. Needless to say, my artistic practice was on the backburner for those years, while my theoretical and conceptual chops grew considerably.

Last year I received a year-long talent development prize/stipend from the Dutch Creative Industries Fund to take the time to establish a studio, and explore new methods, plant research seeds for future work.. and generally interrogate what kind of artist I've become since adding lecturer/pedagogue/media theorist to my list of career subheadings.

The ALMAT residency/configuration comes smack in the early-middle part of this process. So it's a very dear time for me. A time to reflect on my old practice while thinking forward towards the next work.

{keywords: [_, travel, organization]}

Initial Conversation

{kind: title}

{JR, 180921}

Thanks for sharing the lovely texts from the Transpositions collection. I especially enjoyed your focus on 'writing', that the algorithm and individual co-write each other. Especially with interpreted and live-coding systems ~ I like to think about literary terminology ~ co-writing, ghost-writing, epistolary writing maybe ~ as there is a context to the conversation which both the interpreter and the artist-programmer keep 'in the back of their minds', so to speak.

Lately I have been thinking, and reading, about oral music traditions in early America, a music scene I was a part of when I lived in North Florida about a decade ago. This is also a tradition of writing, in the sense that the transmission and transmogrification of musical ideas depends on inscription into human memory, and written expression through the recombinative processes of recall. Today most of these songs are collected in gigantic archives like the Smithsonian. There's an interesting tension in what the archive does to the potential future morphologies of the tradition. 

How do the stories we tell in performing become transformed by the writing systems and dynamics of said systems' memory - how do these systems co-construct the process of storytelling? Individually? Collectively? 

{keywords: [_, writing, memory, storytelling]}

{hhr, 05-Sep-2018}

If you have a look at https://almat.iem.at/methodology.html , you can see that our original plan was to somehow "permutate" the configurations in each iteration. We did not strictly adhere to it, in order to react properly to each particular situation, but I think it's still useful as a guideline. In that sense, the roles would be that I work more closely with you in the sense of the practical work on sound, and David would go a bit more into the position of observer and facilitator. (We can still adjust that as we go along). One thing that interests me is what happens when you try to bridge systems; for example, I have been building this computer music environment, Mellite, which I would like to "inject" somehow into the iteration if it allows it. In the beginning of the first iteration with Ron, we were talking quite a bit about it, as there is some common technology involved, SuperCollider, and since you also use, I think, SuperCollider, it may be another opportunity (for me) to use Mellite as a vehicle to understand the processes. The system is quite idiosyncratic, simply because there are not many users except myself, but it has always been thought of as something that could be used by other artists. The process to learn it can be quite involved, as it is based on another programming language, Scala; however, the aim should not be that you need to learn to program this system, unless you are interested to do so, but rather that I can try to work on things that could be connected to your work, even if you use SuperCollider or a different platform in the end. For example, with Ron the situation was that he strongly relies on the pattern system in SuperCollider, something that I had never used, and so for me to understand his composition process, it was interesting to try to implement some of his sketches in my system, and on the way, to formulate my own version of the patterns system. In the end, this became a vehicle for me to understand what happens to a strategy or operational context if it gets translated or transposed into a different environment. What elements can still be "identified", how does the system exhibit resistance, how does the process take specific turns. Another opportunity would be to see, if our working process permits that we simply try to exchange data between two otherwise opaque systems, as audio or OSC signals.

{keywords: [_, pattern, SuperCollider, Mellite, translation, transposition]}

{hhr, 03-Sep-2018}

Although already two years or so in the pipeline, the recently published Transpositions book edited by Michael Schwab contains a chapter by each of us, which could serve as a fast-forwarded into our thought processes on algorithms: https://lup.be/products/108941

{kind: reference, keywords: [transposition, algorithms]}

{hhr, 24-Sep-2018}

aka shapeshifting?

{JR, 180921}

You got me, I'm a SuperCollider junkie. Once Marije Baalman came to Steim in 2011 she converted all of us, and once I became increasingly involved in livecoding that was the nail in the coffin.

One thing I'm interested in developing in Graz is a toolkit for live coding both for performance and composing. I think you, like me, bounce around between discourses and disciplines. So you've probably felt that gut feeling that it's been too long since you've focused on musical craft. It might be a nice exercise to compare/contrast how we create toolkits for ourselves ~ what are our priorities? For me flow, movement and simplicity are important. I would like to experiment more with live storytelling, so it's also a question of what kind of linguistic tools support narrative arcs and accompaniment. ↗

The concept of memory that I mentioned earlier, in the context of storytelling and the oral traditions of old time American music, keeps rearing its head. And I'm tickling the notion of how to integrate different forms of writing/memory media into such a performance. e.g. using reel-to-reel tape machines, hard-drives-as-speakers, machine learning models... with respect to the latter, I'm finally catching up with the machine learning literacy that seems like required knowledge for artist-programmers in the 21st century. ;-) And thinking about user interventions into machine learning systems. I've been taking an online course in Tensorflow to bone up on that. Google's Wavenet speech synthesis & raw audio analysis algorithms look really interesting in terms of their potential to inscribe and speak. There's also a wonderful phenomenon in connectionist systems, called "Catastrophic Forgetting". Besides being one of the great examples of computer science's flair for poetic terminologies, seems also like an interesting entrypoint for user intervention into machine learning processes. 

I'd be interested in exploring ways of making hooks for live coding in SC to connect with some real-time-ish machine learning processes. Could this also be something for us to investigate together? 

{keywords: [_, flow, movement, language, tools, memory, machine learning]}

{hhr, 24-Sep-2018}

"Flow and movement" are two terms that also resonate with me, perhaps in one way because I'm still interested in understanding Deleuze/Guattari's concept of machines, and in other way since movement seems a good replacement for otherwise cybernetic terms which have too much historic baggage and tend to direct the discourse in unproductive directions (e.g. talking in terms of "movement" instead of "control" seems more fruitful to me). Flow and movement also inherently stress a process-over-product perspective.

Funnily, simplicity does not immediately capture me, although David has worked with the term 'radicalisation' which may relate to it. Somehow I tend to make things that eventually are complicated, messy, buggy and slow, and I somehow then "accommodate" those properties, even if they seem to make life unnecessarily burdensome. So simplicity could be an interesting "frictional term" in our discussion.

Likewise, storytelling and narrativity are two terms that cause some "repelling force" in me, so it could be interesting to discuss our understanding of these as well. Perhaps I should lay out where that preoccupation comes from. I think it comes from the "spatial" property that for me characterises installation art. In my adolesence I was very much fascinated by the collection of the only contemporary art museum within proximity which was the New Museum Weserburg in Bremen, which had (has?) a focus on 1970s and 80s, at least as I remember. That building for me was one giant breathing space, so calm and not "telling" me how to move, where to go, how long to stay, what to read. From the gray blurry photo paintings of Richter to the objects of Beuys, a metal swing by Horn, an inconspicuous sound installation by Julius; and next door GAK, the society for current art, with more recent works by less famous people. Somehow, sound installation for me presented itself as "against music" in the sense of "against stage", "against dramaturgy", "against narrativity and linearity". For me the term narrativity is always connotated with consumption, e.g. the successful sale of a movie, the "success story" of a company/product, the elevator pitch, the TED talk, etc. that is the "formatting" of something into things easily digestable and "communicable" to an anonymous mass audience. Also, the fallacy of distinguishing form and content ("authoring a VR scene", "scripting an experience", ...)

That does not mean that narrativity, from my perspective, cannot be a useful concept, but then the question becomes how we define narrativity, how is it constructed (and by whom). I am very much for "broken narratives", in a way that can only be understood in the plural, and that originates from the person that experiences a piece of art (or any aesthetic encounter).

Notwithstanding, I am very much interested in text and language processing. Of course, NLP is an exciting field to explore, although I am kind of wary with respect to the imperative "required knowledge for artist-programmers". Since a while, I am sensing a strange sort of force exerted on artists working in elecronic media that urges them to navigate in this new landscape of open source neural networks, in the sense that it seems to be required to be have some python, tensorflow, you-name-it, on their CV, no matter what they are actually doing with these systems, just to prove a certain "literacy" in these technologies. I will be honest: I have tried some of these things, and I have a fairly good understanding how these work in principle, but I would never describe myself as any sort of expert on neural networks or machine learning systems. I'm deeply sceptical when someone comes and presents themselves as working at a "forefront", because in 2018 the "best" technology is to encode things with Generative Adverserial Networks, possibly attached to some body fluid and a blockchain cryptocurrency, just to throw in a few buzzwords. I'm deliberately sarcastic here, but the point I want to make is to say that I think as artists we should question the pressure of technological "novelty" to subscribe to; we shouldn't be afraid to pick any sort of machinic approaches, whether they were invented in the past, or in the past six months. We had a funny little off-discussion with Frieder Nake at the last xCoAx, who was very critical (I think rightfully so) after a keynote that basically consisted of a romantisation of "deep" convolutional image processing. I am not against these processes, but I think we should ask ourselves first what it is that makes them interesting, which properties, and then how to appropriate them as artists, with no pressure to "correctly implement" them or to use the fancy toolboxes instead of coding up our own clunky partial solutions.

To come back to NLP, I think it is very interesting to probe the collision between sound and text/language, the former resisting the encoding in speech and language attributes. What happens when the irreducibility of sound comes together with the transcoding into phonemes and formants, intonation, words, sentence structures, then cross-linking between the linguistic properties, etc. If that can result in "broken narratives", I am very much interested in that. I have only rudimentary experience with NLP, I mostly use old fashioned signal processing to work on the acoustical level of spoken language, and I use simple (ha!) processes such as look up, permutation, chunking and re-sequencing, to work with text, and I definitely would like to learn more. For me the challenge here is to break up the (supposed?) discrete nature of language. I have often said, it would be fantastic, if we could write a reverberator for text. Like you enter a text fragment, and the machine "reverberates" that, creating echoes and decays, blurring, ... These utopic kind of machines I find fascinating.

Perhaps David wants to add another perspective on these questions? I know he is very interested in the design of a new programming language.

{keywords: [_, movement, flow, simplicity, storytelling, language, code, writing, text, technology, computer, machine]}