Sonification Strategies

Erin explains some strategies she adopted to translate heart rate, respiration and skin conductivity signals into sound.

[EG]
The patch has 3 synths and I discovered that, if you have 3 synths going at the same time, with different envelopes, this is actually what makes the sound a little more alive. This could be rather elementary but it was a sort of "light bulb moment" for me. I’m wondering if I could do something to modify these envelopes actively, something that would generate some interesting subtlety that makes sense with the data. One idea that I had is that I could simply use the heartbeat itself as an envelope, because look: the envelopes I'm using alreay look like an heartbeat!

The multipliers have some slight additions and changes based off a different low-pass filtering of the BPM. So the BPM would affect the color in the long- and short term. The amplitude of the signal is modulating the sounds that come out of the synth. The heartbeat is activating the envelopes. The actual pitch itself is a function of the skin conductivity level, which changes very slightly. I multiplied it by a power, because what I found really irritating is when it’s moving between two values, like moving in this stepwise motion. I find it too linear. Throwing it into some exponential operator, sometimes it produces some leaps, and it’s a little bit more interesting.

Then there is this extra cycle, which is modulating absolutely everything. I was thinking of coupling this with the respiration, because it would be nice if you could use your respiration in more direct way. Because it’s easier to control your respiration than to control your heartbeat, for example. And finally the most evocative signal, and the one I think the most people can feel directly, actually is this GSR change - or skin conductivity response is the proper name for it -. This is actually controlling the overall amplitude of the sound. There is a lot of silence, because the SCR just changes every once in a while. But I think what’s interesting about this is that, if you are not used to think about your SCR, it is something you can kind of "feel". If you are really focused and there’s a jump in your SCR you can actually materially feel it in this weird way in your body. I think you feel your body qualitatively different when your GSR move up and down.

 {function: description, origin: conversation transcript, place: Experimentalstudio, date: 180528, keywords: [_, body, synth, envelopes, sound heart beat]}

Embodiment and Sonification Strategies

[EG] I’m interested in taking signals of respiration, heart rate and skin conductivity and, instead of categorizing them into little buckets, to kind of just low pass them and have different time scales of all the signals layered over top of each other, and allow you to ear all of them at the same time.

I think that something that’s really frustrating with me is that I wasn’t really convinced by how interesting it was by itself, but I realized that listening to it I was mostly excited because I could actually feel the relationship to my body. And I think it’s that embodiment, that it’s really difficult to capture in the documentation, that it’s really fundamental to what I’m doing.

{event: Signale Soiree, date: 180528, origin: conversation transcript, keywords: [embodiment]}