Diegert: In your article from 1991, As if by Magic, you talked about ‘handles’ as a way of gaining a physical grip on the abstract computer.

Ryan:     Yeah I remember that. Physical handles on a ‘ghostly’ instrument.

Diegert: Yeah, you used the word “phantom models”. We really keyed in on this section of the article because of this sentence here, “handles are just as useful for the development or discovery of the piece as for the performance itself.” (Ryan 1991, p. 5)

Ryan:     Yeah, well we have to recognize that it's part of the compositional process as well. It spreads over the whole realm. We make these distinctions: instrument making, composition and performance, but it's all one thing.

Diegert: That's certainly the way I think of it. Since your article was in 1991, and here we are 30 years later, would you say that anything has changed? Would you amend that now that the computers are faster? Does software nowadays already come with built-in ‘handles’, interfaces, solving the problem for us?

Ryan:     I don't think we have made a lot of progress. The materiality of musical instruments and the kinesthetics of music practice are still not taken seriously. STEIM was the only studio I knew of that gave creative musicians their full attention. Certainly from an engineering perspective great progress has been made, but the framework of digital technology is not yet integrated with the desire to make music. Many people still believe interfaces are instruments or vice-versa in case it makes more sense that way: that instruments are interfaces. My quibble is with the flatness of the computer science concept of interface compared with the wet (sweat or spit) complexity of musical instruments. Computation has assimilated sound; the formalisms of composition find a good fit with coding, but music practice, hmm? Digital technology confronts us with its spellbinding difference – a modality of being made of numbers and code, i.e. something real that seems to be made of ideas.

New technologies always challenge the limits of reality, and the history of instrument making shows musicians as early adopters, always eager to try out new technologies. Popular musics have been more successful at integrating the digital medium with their practice. Hip-Hop, reggaetón, EDM, to name a few we are more familiar with – popular genres all over the world cannot be characterized as alienated by new media, new technology. But conservatory and jazz musicians and composers? These are often reluctant to open their heritage to possible electro-contamination. Perhaps their education dis-enables synthesis and intermarriage? Actually, I can sympathise with them; digital tools and instruments are still rough works-in-progress. Why should players surrender painfully acquired virtuosity or their idealism? But the history of eMusic reveals a host of genre-eluding artists who contrived most of the new instrumentation and modes of presentation. Many of these new territories were vital and remain so even if underpopulated and uncolonized.

I used to frame this as, ‘Emusic is the first new music to evolve without a performance practice’. Digital technology brings with it a radical abstraction that is comparable only to the introduction of written language. Of course, interfacing is essential for digital instrument making, but it doesn't yet take in the intimacy of contact that music involves. Every technology presents us with something new which extends our world.

To come to grips with this, I like to imagine what must have gone on in other moments of technology-driven change like the Bronze Age. Musicians must have been amazed when they first encountered the sounds of these new hardened alloys. Bronze making was extremely high technology at the time, involving great communal expense to realize, but musicians were there touching and listening and tuning pieces of the stuff. Bronze offered a completely new sound source and one with its own voice: just knock and it talks back to you. In China, a millennium of music-making included orchestras of hanging pieces of metal which must have produced truly awesome music. This went out of style in China 2000 years ago but is still alive in Indonesia and other parts of south Asia where both the technology and a music practice are sustained. Later on, the design possibilities of metals showed the way to so many new instruments. Our cymbals and gongs are living relics of that early encounter and playing them has evolved extraordinarily rich kinds of touch (Philly Joe Jones, Elvin Jones, Tony Williams, etc).

Diegert: What kind of change in perspective do you think we need to adopt when it comes to working with computer-based musical instruments?

Ryan:     With digital, we have an extraordinary new sound source, but one in which all access is mediated, the bridge to touch has to be constructed. Traditional instruments could be seen as amplifiers of a musician's touch. It seems essential that computer instruments become something you can embrace physically.

I get links every week to new writing about instruments, so it is a hot topic in the academic info sphere. But there's a minority interest in the materiality, the physical side. Physical modeling was popular 20 years ago. It's difficult to do but the results were impressive. The problem is what makes the modeling work is not what makes using it interesting. A completely different mentality is required to connect the synthesis to music making.

I don't know if there are new words for this (you have to refresh interest in hard problems), but ‘mapping’ was the name of this problem. It is so much easier to deal with it as if it's a problem of MIDI messages. Well of course it's a problem of mapping, but the mapping can be solved physically and solved mathematically. You actually need both. I'm writing something now called The Inside Out Trombone - a tricky title to make a special point. But the trombone is a perfect example of what looks like an interface. It is just a slider, but in fact it's not because it's plugged into your body, into your incredibly elaborate ‘over determined’ embrace. Control is just stupid word now, because it simplifies the relationship of playing an instrument. It's not a relationship of control. You don't control the saxophone. That would sound like, ‘Please, sir. You should control your child. Something's wrong, he's all over the place’. You don't say that about your children. Other people say that ‘he doesn't control his children.’ We understand it's a complex situation, but the irritated guy is thinking about it as if it's an issue of ethics.

I think that's the problem, that ‘control’ is primarily an ethical or political concept about power. But from a musician's point of view, it's always a sexier relationship; passionate and sensual, not a power relationship. Even when you get to the point where you are angry at your fingers or your lip or something, it's something you have to get over. I didn’t have what you would call a satisfying experience with instrument playing; I studied classical guitar. So it was probably my frustrations that led me into electronic music … and then back towards instruments, because I was so attracted to what real musicians know. First was George Lewis in Paris; in a great moment of synchrony, we were at work with very similar tech in a strange town. Through George, I met so many people like Evan Parker ... a long list ... musicians who listened and touched their instruments in really original ways. It certainly didn't seem to be about ‘control’.

Diegert: Clearly you're thinking about the ‘handles’ as something more than just this idea of control or mapping and the idea of an interface. Do you think that any technological progress in the last 30 years makes any substantial contribution to that?

Ryan:     I think there has been a huge amount of progress. I don't know if it's musical progress, but a huge amount of technological progress. You know, like back 10 or so years ago, when things like Leap Motion appeared. Most, of course, lacking much effortful physical contact, but they at least take the gestural part seriously. I do see the word ‘gesture’ come up; I get links to new articles all the time, but they tend to think of gesture from a pretty reductive point of view. They don't go very deeply into what might attract a creative musician and they're hardly aware that the issues are different in different musical cultures. So they end up analyzing it ‘engineeringly’, or something like that.

Diegert: When we spoke last time, you talked about your collaboration with Evan Parker. I'm really interested in some details about how you work together, how you have done electronics with him. You mentioned to me before that he was improvising into a microphone and you are manipulating his sound in a certain way. 

Ryan:     We had time to get to know the range of possibilities before going public. I'm always worried that a musician might not enjoy what I do, since it overrides their usual sound world. My approach is diverse and depends on the nature of a player's sound, but in brief it's like giving them a new instrument but one where all their chops still apply. But I am definitely tampering with their practice and I need to let them discover this without the pressure of performing for an audience. For Evan. I actually made a recording processing an excerpt from one of his solo CD’s and played it for him and Barry Guy when they visited STEIM. This got him intrigued and we took some time to play in the studio. (Barry is also great to play with, he has this wild alter ego inside his precision baroque musicianship.) I've insisted with each new encounter that we have at least one rehearsal because it turns out that there's a good minority of musicians who can't deal with the decentering of their instrument.

Diegert: Could you describe that process a little bit? Like, what would you do in those rehearsals? And how would you approach it? Would you come with material prepared?

Ryan:     It’s probably got two elements; one is that I tried to give them insight into what I'm doing. So it isn't a totally black box. I give them examples, talk to them a little bit as we go through my fall collection. And it's fairly obvious when you play: there are a lot of ways to displace and expand time, likewise intonation, then an odd variety of more diffusive, coloristic kinds of processes. It only requires that they be good listeners able to hear how they can act to master the mix. Evan was a fabulous listener.

It turns out that one of the most difficult categories for me was the violin. And it's not because of the instrument, it was because of the attitude of some violinists. I worked with a young Moscow Conservatory player. And it's not that he couldn't understand our musical premise, but he wouldn't understand it because it was inconceivable that his violin should change. And since I was screwing with his instrument, he more or less ignored it. He was not listening to what was filling the hall from the speakers, from his playing, he was listening to his inner violin.

But I really can't fault him for this. I'm changing the physics of the violin, changing its size, its elasticity, all the dynamic characteristics of his instrument. But his whole development as a musician was about keeping the physics under control so that an ideal violin could become real. I'm undermining this stability, putting the idea of stability in doubt.

There are, of course, many different ways I can approach an acoustic instrument; I work differently with different performers. But there has to be a willingness to get involved. I guess it's just like anybody's jam sessions: you get together and see if there's any chemistry. Romantic talk like that is appropriate here.

Rehearsal is another issue in the many territories of jazz and improv. Certainly, it is an economic reality but also an ideological one. In hard core Euro Free Improv, there is no sense to preparing for a new encounter and nothing to recover. There is just improvisation and the moment; everything else is cheating.

Diegert: That's interesting how that relates with live electronics, though. If you were a traditional DJ, spinning records, or scratching or doing something else, producing your own sounds, you could imagine that being treated just as a typical instrument. But when your instrument is actually extending or manipulating or distorting the sounds that are being played live, that seems to me like a substantively different setup.

Ryan:     Yeah, it is. I was lucky my first test situation was with a really great improvising violinist, Malcolm Goldstein. He was recommended by John Cage when I asked him who he thought was an interesting composer ‘these days’. Malcolm, it turned out, was often in Holland to work with dance people. When my first 16 bit DSP instrument was ready (based on the first 56000 audio coprocessor from Digidesign), I invited Malcolm to STEIM. Only much later I realized how fortuitous this was. Malcolm is a challenging player but a great listener. He is always listening to what his violin is doing (believe me this isn't the default case). He was not watching to see how the violin was succeeding or failing at being The Violin. He allows his bow and violin a certain freedom and in listening he discovers his journey. He listens to watch for that something special to happen physically, then he jumps on instantly to ride it: sonic surfing. His sense of play was perfect for me as I was having the first experiences of my algorithms, and I was listening and surfing them too. This is a feat of mutual listening, both of us are effecting 'what happens'. This one instrument, two players balance characterizes all that I have done musically since.

Diegert: If I could just ask one more follow-up question on the collaboration topic. You said you'll do a minimum of one rehearsal. I'm wondering about the other end of the pendulum. Do you have collaborations where you've worked with somebody for a longer period of time, where you have the time to get to know each other really well and build a piece out of that working relationship?

Ryan:    The most extreme case of collaboration was when I worked in ballet where we performed the same piece for five years. 70+ times the same piece, which had a strong improvisational core. The music was performed by a violin, three trombones, and my signal treatment instruments. It's called Eidos:Telos in Greek. So ‘Eidos’ is image or idea, and ‘Telos’ is the purpose, meaning. It's a really famous piece in the history of modern ballet, and, it turns out, a very powerful, emotional piece. Three acts and lasts a full evening. This is more than a band, it was a company of about 50 people including all the tech support tonmeisters, lighting, dance masters, dramaturge, etc.

It took six weeks to ‘create’ the piece, not including the development time which was a year or two. And because it was Ballet Frankfurt, we had all the facilities of Frankfurt Opera house. We worked in several studios, preparing the digital instruments and experimenting with the violinist and trombones, and seeing the work that Forsythe and the dancers and the star Dana Casperson were coming up with. Finally, everyone went to the opera stage itself two weeks before the previews. Everybody: dancers, choreographer, music director Thom Willems, tech crews, and this band were on stage 10+ hours a day for two weeks pulling the pieces together. This was a revelation to be able to compose with all forces present within the actual space of performance. We were creating our work on one of the world's largest opera stages, 27 meters wide, with a fabulous acoustical space for amplified music. Its sheer scale provided a ‘space’ in which our sounds made sense. We could go from tiny unamplified ambiences to diabolically huge crescendos and it made sense. I don't know where other than in music theater you can find these conditions, certainly not for new work. Add to this conscious cross media collaboration!

This started with the vision of Dutch composer Thom Willems who had been music director at Ballet Frankfurt most of his musical life. During a Holland Festival, he introduced Bill Forsythe to me at STEIM and we hit it off right away. I flew with the company to Tel Aviv and got to watch them in Loss of Small Detail half a dozen times. I signed up to work in the next premiere. The new piece was to include live instrumentalists (relatively rare in modern dance), transmuted with the digital instruments that I had been working on. But I had to seriously reconsider them to fit this instrumentation and theatrical situation. You could view this as composition or instrument programming, but in the end, it's making music.

We started from Thom's idea of a choir of eight trombones. We auditioned eleven players and then realized that maybe we didn't really need eight! Given what I was planning to do to their playing, a trio turned out to be perfect; for density, harmony, layering, breathing, improvisation, etc. We could make huge Wagnerian chords and fierce sonic transport with just three trombones. The players had to be helped to feel confident with the techniques that ‘went with’ each instrument scene by scene. We had to do this for the violin as well, new instruments but also new playing techniques. Though the young player was a virtuoso, he, unlike the trombone players, had to be coached (even tricked) out of his comfort zone. But we had lots of time, 6 days a week, more than a month to design and hone our practice. It’s so rare, and valuable, to have such an abundance of time to work all together: ‘rehearsal’ isn't the right word.

I wish that people could see the value of letting artists and musicians collide to create new work. It works in film and music theater but is rare in pure music. The excitement of seeing how differently dancers, actors, or writers respond to you is the greatest stimulant and should be the standard model. Instead of hiring studio musicians or orchestra players to drop in for one day – that's for classics and genre pieces. What works for invention is long tours, months, a year of playing together, and discovering how to create something new.

Diegert: Could you go into more detail about the working process? Was the composer, Thom Willems, writing music or giving more general direction? What kind of interaction/collaboration did you have over that period of time?

Ryan:     Well, I think Thom Willems and I were sharing the same stage and trying our best with whatever means we each had to produce a great piece. It was great to work with a composer as open and creative. Thom had composed a violin score inspired by a passage in Stravinsky's Apollon Musagete for an existing Frankfurt piece with the great and revealing title, Self Made to Govern. An amazing synthesis, forceful choreography illuminated by improvisation. Each night, there was a change in the cast, sometimes a whole new group, sometimes a single shuffle, but always a resetting of the piece. I never tired of watching it. For Eidos, we added the three trombones and my treatments. When we moved to the big stage, the first thing we realized was that the trombones were going to have to be pushed back as far from the audience as possible – because trombones are so fucking loud. This is a mystery I have to negotiate every night I join with acoustic musicians in a new hall. How do I find a balance between the familiar acoustic and stylistics of their instrument and the expanded sound world I hope to project? This is not a technical decision, it depends on individual players, the performance space and the moment, yet it can only be achieved by learning how to tune the tech. Even with the huge sound system we had for Eidos and the help of expert tonmeisters, this was a process that was renewed in every new theater and on every evening of every performance.

Finding the place for the trombones also helped define their choreographic relation to the piece. The audience could see them way at the back of the stage on the right. I strongly believe this visibility was essential evidence that this was live music. Audiences are used to recorded sound; somehow they accept that a disincarnate recording shares in the actual of live performance. I think this unexamined economic pragmatism has greatly hampered dance, theater, and music itself. In Eidos the liveliness of the music helped the audience enter the ecstasy of the performance.

Tommy eventually wrote collections of harmonies and little motifs, almost like lead sheets for jazz. There were headphones on all the musicians, and Tommy was cueing them yelling ‘four’ or ‘F-sharp’. There were the little numbered motifs, and it worked. Every night Thom Willems and Bill Forsythe were there in the first balcony with microphones connected to the stage. This dedication to the live moment really impressed me; it set a standard for me that I always strive for. But, and probably this was a little rude, I refused to listen in. I couldn’t focus wearing headphones. I was obsessively balancing a precarious acoustic bubble; moreover, this was very fast moving music with all sorts of appointments to make with dancers and the scenario. To keep up, I needed to listen and respond second by second. So they hired a student of mine as ‘understudy’ to listen and (gently) clue me in to pertinent exchanges between Thom and Bill with the trombone players and technicians.

This Frankfurt company was not a band of tin soldiers acting out the machinery of a dance; one where you can predict what's going to happen and then watch how gracefully it plays out. Maybe that's what some people come for: predictable/perfection. It wasn't that way in Frankfurt. William Forsythe is from New York City, his desire was for a kind of classicism but with an extraordinary vivid presence, related to un-dictability. Somehow he was able to synthesize something that could be accepted as ballet but folded together with spontaneity that came from crypto improvisation. He was able to do this with a heterogeneous company of modern and classical, even self-taught, dancers, but not without great effort. So many classically trained dancers, who have incredible technique, but for whom presence is the discrete signals they make in advance of their moves as of, ta da!, they were in the circus. This provides very little sense of contact or dialogue with real people. He wanted a real dialogue between the dancers and thus real improvisation, a way to choreograph dancers for a Self Made to Govern.

For this piece, the poster simply listed the names of all the people who worked on the piece alphabetically. It was acknowledging the collaboration; wonderful because so much performing art is collaborative but not recognized as such. This is a big issue here in Europe, because the idea of the composer is now more important than the music itself.

Diegert: In musical collaborations, we usually just come together at the last minute and throw on the piece. I find it interesting the idea of developing longer-term collaborations and having a much deeper kind of co-authorial method of creating music. Since you've mentioned the dancers in that ballet as co-collaborators: when you're dealing with dance, visuals, and staging, in the theater, there are multiple disciplines that have to come together to create the final thing. I also have a particular interest in collaboration across media. To what extent does the music impact the dance in real time? Did your music impact the choreography, or was it an existing thing from the beginning?

Ryan:    Generally, dance doesn't have live music except traditional ballet pieces where there's an orchestra. But in this case, there really was not just live music in the nominal sense, but ‘scary’ live; we were pushing things. But it wasn't like the dancers couldn't guess if it's going to turn quiet three minutes from here. They could depend on important pivots, but they couldn't know exactly what would happen between them. So they could listen to the inflections in the music just like they watch the other dancers, which these dancers were doing all the time (I remember Forsythe at a premiere so upset that, invited to choreograph another national company, he ‘couldn't persuade them even to make eye contact with each other’). Even with the precision, these dancers were always projecting they were watching what the other dancers were doing: how that could be joined, how to make a relation to gestures or spatial alignments, or the special presence someone was creating.

Dance in the late 20^th century was obsessed with getting a notation system going, the Laban thing, how to memorize choreography. This took a unique turn with Ballet Frankfurt, but the arrival of video tape largely filled the memory gap. I always saw small groups of dancers sitting on the floor in front of monitors watching videos of old performances, either to recover a piece or to bring a new dancer up to speed. By this time, there was a special position filled by a former dancer, recording everything and keeping the library. Video is essential to making dance, but this wasn't present for the audience. It was part of the technology of performance.

Forsythe did collaborate with video artists in that period, but the practice of video art wasn't ready for performance. They would simply not come to the rehearsals; they were studio types and only comfortable alone with their gear. And when they finally brought something, Forsythe had to say, ‘Sorry, it's nice but it has nothing to do with what we're doing, you know, now! It was a nice idea, I liked talking to you. We enjoyed the process back then, six months ago, but we are someplace else now.’ The night of one premiere, he just turned all the monitors around (this was the 90s) facing the dancers. As far as he was concerned, there was no reason to show them to the audience. It turned out he kept these monitors throughout the next five years as they worked to inform and coordinate the dancers’ improvisations.

How do you get 30 people on the stage to do something besides mimicry, doing exactly the same thing in parallel? How do you get them to do something which is somehow readable as coherent? The coherence could come from the music – we were doing really intricate things that had gestural coherence and could be easily read as movement. Eidos was striking, because it was a chaos that was readable. It was very successful because of that, and it only was possible because we had the kind of rehearsal time. How much rehearsal do you get if you have a new piece with a big orchestra? They get like two days, one day of attention, and that's it. And I think your project is fabulous because that's exactly where we are now, we need this time. We need this expanded time. And it makes more sense in this collaborative thing to have that be rehearsal collaborations.

By the time we came to make this piece, Forsythe had assembled a language which later he called Improvisation Technologies. It was an ample (the CD-ROM tutorial had 60 chapters) alphabet of isometries or transformations that could be applied by a dancer to movements and postures. It was not stylistic, but a way of looking at and reading a dancer, yourself, or a partner, or even an object across the room, and a set of tools to transform what you saw. This is not mimicry or impressionism but a systematic way to see that the dancer learns in practice with the alphabet. A dancer notices an interesting relationship and has a gamut of methods to demonstrate this relation. If two or three of them do this at the same time, forms will be reproduced.

The goal was that both naïve audience members and improv experts could recognize new forms. Some of the terms were just versions of geometric transforms: translation, dilation, contraction, and rotation, others were specific to dance practice like ‘avoidance’ or ‘floor reorientation.’ The alphabet consists of embodied mnemonics that come from many frameworks of dance and choreography, enabling agents who though they might not be equipped to speak about geometry, were all masters of mobility in three dimensions.

There was one new dancer, Nick Hafner, who was put on earth to make video. He offered to make ‘bar tapes’ for the alphabet – if you went into a club or bar in the 90s, there were always video collages playing. A cinema lover, he rented his favorite old movies and searched for scenes that evoked one of the mnemonics. These, in fact, are what the video monitors ran during the performances of Eidos. If at any point a dancer felt a need for inspiration (improv was a huge step for many classically trained members of the company), they could look at one of the monitors and discover in what they saw one of the transforms, that they could apply to their own movements. Paradoxically, improvisation added compositional coherence as parallel readings could be made by several dancers, and their transforms could quickly ripple across a stage full of dancers. I have seen and heard many improvisation systems in music. In general, they seem more like authors' extensions: machine surrogates supplying instruction for the composer. Few seem interested in enabling independent groups musicians at a depth comparable to this dance technology.

After studying Physics and Philosophy in California at Pomona and UCSD, Joel Ryan went on to develop idiomatic digital signal processing instruments for electronic music, which he has since used in performances with a long list of artists, including George Lewis, Steina Vasulka, Evan Parker, William Forsythe, Malcolm Goldstein, FM Uitti, Agustí Fernandes, Peter Evans, Keir Neuringer, Najib Cherradi, and Sainkho Namtchylak. He came to STEIM via Ircam around 1985, where he coded new instruments, evangelized and helped nurture a live performance practice for electronic and computer based instrumental music in Europe.

Ryan was gracious enough to agree not only to an interview on June 28th, 2021, but also to follow up with additional written material. This discussion is available here, including discussions about the nature of digital musical instrument design, collaboration in the arts and across disciplines, and a detailed account of his experience working on the famous ballet, Eidos:Telos.