Mediating elements. Semantic matching
My initial idea to match music and food in this research trajectory was that I could try, through the use of adjectives, metaphors or similes which one might apply to either the food or the music, to bring one of the latter closer to a semantic field in which characteristics of the other might be defined. This is what I had, more-or-less serendipitously, done in my first performance combining food and music, A Year in Four Bites. There, for instance, I chose music with timbres and harmonies that I defined as 'icy' and 'fragile' to match food that had been made with ingredients that represented the winter season. I found it very hard, however, to define exactly what it was that I wanted to do with these descriptive words and phrases, and how that actually had anything to do with either the food or the music. I discarded the idea as I started to read about crossmodal correspondences, which seemed to offer something more solid to base the pairings on.
I was surprised to discover, in one of the review articles about crossmodal correspondences I was reading, the concept of ‘semantic matching’. The authors mentioned that “semantically mediated correspondences may develop if the same terms or concepts are used to characterize sensations arriving from different sensory modalities. […] [T]hese interactions between the senses may take place at a late stage of information processing– after information arriving in different senses has been encoded into a common, abstract (possibly semantic or even verbal) representation” (Knöferle & Spence, 2012, p. 999). This sounded a lot like what I had been looking for when I began my investigation. A link between taste and hearing that could be established through meaning, meaning encoded in words. I saved this idea for later use.
Semantic matching through the use of words describing texture
While analysing the results of the first experiment, I realised that I wanted to use texture as the central element of the pairing that would take place in the second experiment (more information as to why I took this decision can be found in the section Mediating elements. Crossmodal correspondences). When reflecting on how to approach this endeavour, I remembered the idea of semantic matching as defined above. Using the same term to characterise impressions stemming from different senses might, it seemed, mediate a correspondence between those impressions.
I started to work from the table I had created featuring the different extended techniques for each instrument (this table is shown in the chapter The music, and further discussed in the section Mediating elements during the creative process), and added an extra column and row with words that I thought could describe the musical texture of each of the techniques for each instrument. I limited myself, however, to words that could also describe culinary texture, and therefore left many of the techniques undescribed, for which I could find no adequate adjective that could also be applied to the texture of food. This modified table is shown below.
Using this table as my point of departure, I defined three timbral combinations, which I would describe as follows:
1.- Air sound on the traverso. Harmonic on the G string of the violin, played sul tasto. The combination of these two elements would result in a musical texture that could be described as light and airy.
2.- Pizzicato with double tonguing on the traverso. Col legno battuto ricochet on the violin. The combination of these two elements would result in a musical texture that could be described as crispy.
3.- Flatterzunge on the traverso. Sul ponticello on a double stop on the violin. The combination of these two elements would result in a musical texture that could be described as coarse and flaky.
I already had some ideas as to what kind of food could work with these specific sounds. The first timbral combination could be matched with a foam, which might also be described as light and airy. The second, with a crispy chip of some sort. The third, with a paratha, a type of Indian flatbread made with wholegrain flour and layered with fat.
In the next step, I refined both the music and food so the match would be as accurate as possible. I tried multiple types of foam (gelatine-, egg white-, and agar-based), and finally settled for the gelatine type. I experimented with different wholegrain flours and decided to use a stoneground spelt flour containing relatively large pieces of bran, which would enhance the coarseness of the paratha. I also realised that, since I would serve everything cold or at room temperature, the paratha would considerably more chewy than if warm, so I adjusted the musical material to match this (how the chewiness of each culinary element was considered in Experiment 2 is described in more depth in the section Mediating elements. Structure). Below you can find audio examples of the different musical textures, the food they corresponded to, and the words used for the semantic matching.
In the feedback form the participants filled in during Experiment 2, they were asked to describe each dish in three words, and to do the same with the music corresponding to it. Several participants used the exact same word to describe both the first dish and the music corresponding to it. These words were "light", "airy", "soft", and "smooth" ("airy" was the most repeated of these). There were two participants who also described the second dish and the music corresponding to it with the exact same words, namely "crispy" and "crunchy". Only one participant used the exact same word to describe the third dish and the music corresponding to it, and that word was "firm". These results are not particularly surprising, as the first and second dishes had a very clear, simple, and easy-to-describe culinary texture, whereas the third one had a more complex and less easily definable texture. Moreover, the words "light", "soft", and "smooth" are commonly used to describe music, and the word "airy" would have been the most obvious one to describe the type of sound the flute was making in this section. By contrast, the word "crispy", which was very consistently used to describe the beetroot chip, is less commonly used when describing music, and therefore was not used as often when describing the music corresponding to the second dish.
However, there were also several participants who did not use exactly the same words to describe both elements of the matching, but words that could be said to be synonymous, or that at least share a part of their meaning. One participant, for instance, described the first dish as "light" and "foamy", and used the words "airy", "soft", and "smooth" to describe the music corresponding to it. Another participant described the second dish as "intense", the music corresponding to it as "aggressive", while another one thought that the former was "playful", the latter "exciting". The last dish was described by one participant as "smooth", its music "soothing", while another one through the dish was "rich", "dense", and "hearty", and the music corresponding to it "full" and "round". (Note that not all of these words describe texture, more on this below.)
The fact that a substantial part of the audience used the same or similar terms to refer to both the music and the food points to the possibility that their perception of the match might have been affected by these semantic parallels. There are a few other examples from the other two performances that were carried out as part of this research that seem to point to the fact that a correlation of some sort does indeed exist between the words chosen to describe the elements of the match and whether this match was perceived as successful or not. After Experiment 1, one of the participants mentioned that the last dish was "tender and tasty", and matched with the "delicate and beautiful" music. Another participant, however, defined the same dish as "dense", whereas they described the music corresponding to it as "light", and therefore found that there had been no match between the two. An example taken from Four Bites of Autumn would be that one guest found that the "best point" of the performance was "the sweet viola da gamba playing" that accompanied the (also sweet) Glühwein that was served, while another guest found that the matching was successful except for precisely the Glühwein, which was "too aggressive" when compared to the music, which would have required something "more soothing". It seems that the constrasting impressions of these pairs of listeners might have been influenced by the words they used to describe both the food and the music: when similar or compatible words were chosen to describe both elements of the pairing, it was perceived as successful; however, when the words used to describe music and food were not compatible with each other, the match was perceived as unsuccessful.
Semantic matching through the use of words describing affect. Linking narrative and semantic matching
After having experimented with semantic matching in Experiment 2, I found myself designing the performance Four Bites of Autumn, which would also feature 18th-century pieces, and not only music composed by myself. This meant that, if I wanted to use semantic matching, texture words might not be the right choice to base it on, because the musical textures of the pieces we were going to play could not be defined as easily as the isolated combinations of timbres which I had composed for the experiment. Instead, I decided to focus on 'affect' or 'passion', a concept essential in 18th-century arts, which could be defined as the emotion or feeling that a piece of music or section thereof is meant to stir in the listener. Some of the words participants in Experiment 2 had used to describe the music and food, such as "aggressive", "playful", "exciting", or "soothing", could be said to roughly belong to this category.
I chose the 18th-century repertoire in the performance by deciding the type of affect I wanted each of the dishes to convey. The first, I decided, would need to give the guests a feeling of 'welcoming joy', which could also describe the spicy pumpkin and carrot soup that they would be served. The second was meant to convey a feeling of 'painful sorrow', for reasons explained below. The third would carry an affect of something like 'generous triumph', matched with a piece of pecan bread with pears and parsnips. Lastly, the fourth piece would evoque a 'loving tenderness' in the guest, and would be paired with a rose cake.
The links between these words I had chosen to describe the affect of the different pieces and the food were obviously much less evident than those the texture-describing words I used in Experiment 2 established. For this reason, I decided to employ narrative as a device to steer the listeners into the sort of affect I was envisioning for the different dishes. As is described in the section Mediating elements. Narrative, I told the sad life story of the composer Albertus Groneman as an introduction to the second dish. The perception of the music we played, though it might have conveyed that 'painful sorrow' I was looking for anyway, was undoubtedly influenced by the tragic outcome of the story. The piece of rye bread with apple syrup that was eaten while it sounded did in all likelihood not contain any 'painful sorrow' in itself. However, the fact that the audience knew that it was similar to the food that the unfortunate composer might have eaten during the times he spent in a madhouse towards the end of his life did probably have an influence on how the bread was received.
The other three dishes were introduced by somewhat less elaborate stories, but these were nevertheless intended to steer the listeners into the affective realm of the next dish. Sometimes, I would even explicitly integrate some of the words I had used for the semantic matching in the spoken introduction to the piece.
Conclusions to the use of semantic matching
I found using semantic matching fascinating, both when thinking in terms of culinary and musical texture, and when thinking about affect and relating it to the narrative surrounding the performance. The fact of choosing a particular set of words allows for a certain ambiguity that is tremendously productive from a creative point of view. Thinking about texture was also easily relatable to the type of timbral thinking that characterises my work as a composer. In that sense, it was a lot more comfortable for me to compose the music for the second experiment than for the first.
It also seems to be a rather intuitive way to interpret the pairing, as can be seen in the comments from the participants that have been cited above. I might add to this that several of the participants mentioned in the feedback form that they had guessed that the pairing had taken texture as the starting point of the matching in Experiment 2. This was somewhat more obvious to them than the crossmodal correspondences I had presented in Experiment 1 (perhaps even too obvious from an artistic point of view). It nevertheless might be related to the overall impression the participants had of the match being acceptably or perfectly succesful, more so than in Experiment 1.
In a sense, it felt like, by using the same words to describe both food and music, each of them worked like a definition or an analysis of the other. The foam seemed to explain the airy music, the paratha's complex texture was broken down by the music that was designed to match it. What was happening, I think, is that the fact of having an obvious focus on constrasting textures in both music and food made it very clear that this was what the audience should be most attentive to. It occurs to me that this might be a very interesting way of presenting a challenging piece of contemporary classical music, for instance. One could focus on its changes in musical texture, and give the audience a series of tactile samples that would show equivalent changes in texture, mediated by semantic matching. These samples might be different bites of food that the audience could eat, but could instead be pieces of fabric or something else that they could explore with their hands. Or one might focus on the changes in rhythm in the music, and show the audience paintings or photographs with a similar, semantically matched, visual rhythm.
Finally, I would like to once again stress the importance of the links that might be established between narrative and semantic matching. Two audience members that have built a very different narrative around a certain performance are unlikely to establish the same semantic links between the elements of said performance. In my opinion and experience, these two mediating elements should go hand in hand, and should be taken into account together. In fact, they might be two manifestations of the same phenomenon, namely our semantic and very often verbal perception of the world that surrounds us.