Crossmodal correspondences refer to the phenomenon where stimuli in one sensory modality (such as sound) are systematically associated with stimuli in another sensory modality (such as colour). In the case of sound and colour, several crossmodal correspondences have been identified and studied. A very delicate point must be stressed. Among the various experiments and studies that have been conducted (Marks, 1974; Wicker 1968) there are no conclusive/coherent results regarding a direct mapping between colour and sound. What is of importance and seems systematic is the correlation of pitch with colour (hue), which will lead us to the first type of crossmodal correspondence.
In the following sections we shall not analyse the methods in which the experiments took place but focus on their results. Should the reader be interested in the "technical" way in which the studies are realised is advised to look up the references provided at the end of the text.
There is ongoing research with regards to crossmodal correspondences. It contains a very wide range of different combinations since the matching of distinct features and/or dimensions happens across different sensory modalities.
The Bouba-Kiki effect is probably the most famous example. Without any context: in what order would you assign the names Kiki and Buba to these 2 shapes (no explanation/justification needed).
If you answered that the round shape is Bouba and the spiky one Kiki, congrats! Most of us do this association. What is even more striking is that apparently there is no indication of cultural background influencing this response. That is why there is still ongoing research on this research topic.
The first type of sound/colour crossmodal correspondence is that of pitch-luminance association. It is the oldest and most common crossmodal correspondence. The property of describing how “sharp” (high) or “flat” (low) a tonal sound appears to be is associated with the luminance of a colour. Ontologically this is of particular interest. Unlike loudness, pitch is typically considered a metathetic rather than a prothetic dimension. This means that higher pitch isn't inherently "larger" or "greater" than lower pitch but is qualitatively distinct. Consequently, it's commonly believed that pitch is mapped onto sensory dimensions in other modalities, such as luminance, based on qualitative correspondences between them. However, some associations between pitch and color may have been influenced by changes in loudness. Human hearing sensitivity is frequency-dependent, and within the tested range, the subjective loudness of pure tones generally increases with frequency. Therefore, using stimuli normalized for peak or root mean square amplitude may not be enough since the higher-pitched sound might still be perceived as louder, introducing a confounding factor. Nonetheless, evidence suggests that associations between pitch and luminance, saturation, and hue persist even when loudness is held constant, indicating that cross-modal correspondences involving pitch aren't solely mediated by loudness.
While there's extensive research on color-loudness and color-pitch associations, there's less experimental evidence on how color relates to spectral characteristics like formants—frequency bands amplified by the vocal tract, which create different vowel sounds. In a comprehensive review of sound-color synesthesia spanning centuries of reports, the associations between color and spectral characteristics remain relatively understudied.⁜
So even though it would make more “sense” to compare 2 variables/dimensions that are qualitative (louder with brighter) this doesn’t give consistent results.⁜
Saturation +Timbre
A definition for the crossmodal metaphors (examining the timbre semantics in this case when trying to describe music) is : "a particular concept, an abstract yet structured idea anchored to and allowing one to make sense of a particular perceptual representation".
Another dimension to consider is instrumental timbre. Timbre, often defined by what it is not, refers to the distinct quality of musical sounds once factors like pitch, loudness, and duration are standardized. Sometimes referred to as 'tone color' (derived from the German term "klangfarbe" introduced by Helmholtz in 1885), timbre is frequently described using color-related metaphors in English (for instance, a 'bright' tone versus a 'dark' tone). However, there has been limited research exploring the connections between timbre and color or how different aspects of timbre might correspond to various dimensions of color perception.⁜
The reason I am referring to this type of crossmodal correspondence is because of my PIA (Professional Integration Activity, as part of the Master Project). I want to share what I think might indeed be an indication of the potential relationship of colour (especially it’s saturation) with timbre. You can find more information about the project on this page. Basically, we chose six -contrasting- pieces and asked for people to associate each one with a colour. People then voted. The colour with the most votes would win and we would switch on a torch of that colour. Only one of the pieces was full of Pizzicato (for all the other pieces we used our bows to perform). In the 16 times that we performed, every time the majority of the people (10-30 persons would sit in the audience each time) would associate the pizzicato piece with yellow. The audience was diverse (different age groups), and we performed in many different spaces with different acoustics. Nevertheless, the winning answer was always the same. I find this very interesting since hearing pizzicato and a bowed sound is a clear and huge difference in timbre.
Initial intuition based on similar patterns prompted an investigation into precise natural and physical connections between color and musical notes. However, findings have unveiled a link between visual brightness and pitch, and more recently, apparent associations between hue and timbre. Despite some believing in a natural, physical connection rooted in perceived similarities between corresponding stimuli in perceptual continua, many scholars over the centuries have argued for emotional mediation instead, often termed as affect, mood, or as 'inner harmony' by Kandinsky (1977). Recently, psychologists and practitioners have shown increased interest in emotional parallels between music (or paintings) and color, known as emotionally mediated correspondences. While emotional mediation offers another way to translate sensory impressions, it may lack the rigor some desire. Simpson et al. (1956) previously hinted at emotional mediation as an "indirect" association, suggesting a common mediating variable such as mood. The emphasis on emotional mediation aligns with Werner's (1934) view of early perceptual experiences as syncretic and imbued with expression and affective properties. Exploring why mappings between musical sound pitch/timbre and hue/brightness have captivated our interest over centuries, despite the existence of other crossmodal correspondences, reveals diverse motivations. These include structural and perceptual analogies, artistic curiosity, technological advancements, and reports from synesthetes. The predominance of visual perception and color's significant role in human behavior may also contribute. Ultimately, the affinity between musical sounds and colors appears to be explained by affect, mood, or emotion, regardless of stimulus complexity. This suggests that pitch/timbre-hue correspondences are not inherently special.⁜
Research has shown that usually minor slow movements are associated with more desaturated, cool colours (grey) whereas major fast movements are associated with more warm colours. They performed more experiments to make sure that this data was not the result of a correlation between the variables. They studied what emotions paired with colour associations. This outcome was evident in the responses we had during the series of concerts with my string quartet. That is why the selection of the pieces was crucial.
Crossmodal Correspondence Studies
Lawrence E.Marks, "On Associations of Light and Sound: The Mediation of Brightness, Pitch, and Loudness", American Journal of Psychology 1974, Vol.87, No. 1-2, pp. 173-188
Charles Spence, "Exploring Group Differences in the Crossmodal Correspondences", Multisensory Research 35 (2022) 495–536,DOI:10.1163/22134808-bja10079, 2022
Hauck P, von Castell C, Hecht H. "Crossmodal Correspondence between Music and Ambient Color Is Mediated by Emotion." Multisens Res. 2022 Jun 8;35(5):407-446. doi: 10.1163/22134808-bja10077. PMID: 35985652.
Ward J, Huckstep B, Tsakanikos E. Sound-colour synaesthesia: to what extent does it use cross-modal mechanisms common to us all? Cortex. 2006 Feb;42(2):264-80. doi: 10.1016/s0010-9452(08)70352-6. PMID: 16683501.