Demonstration of the full version of the Translator showing the effects of various controls being adjusted (see details below)
This is a non-specialist overview, with additional information in the appendix. The device was produced in three different versions, but the general principles of each were the same, and these are described first.
2.1 Principles
The broad principle of the device is that a sound signal (in this case, birdsong) is received, either via a stored file or directly from a microphone, and then analysed to determine its varying pitch and output as a stream of numbers representing musical notes (known as MIDI values). These values are recorded, and each note is mapped in turn to one of the seven notes in a single octave of the C major scale. These notes are then grouped into sets of four. Each four-note group represents a word in the Solresol language, and each English word corresponding to each four-note group is found using a dictionary look-up table (see details in appendix).
The Translator’s software is written in the Purr Data real-time graphical programming environment, which is a Pure Data extension. This is open-source software that can be run on a variety of hardware and operating systems. This environment utilises patches which are small pieces of computer code with defined inputs and outputs that can be linked together to build sophisticated, powerful systems.
The pitch-tracking patch 'Sigmund', created by Miller Puckett, is at the heart of the software. In response to an input sound signal, this produces a MIDI note value. There are several parameters that can be set to control this analysis, including the overall volume and duration of the signal required to register a new note. Following experimentation with various birdsong files, the default values for these parameters were determined. In some versions of the device, the user can change these in real time to optimise the translation. Because the Solresol language includes some single, double, and triple note words, the user can set a parameter that specifies how long to wait before collecting the next set of notes. Two quick notes of birdsong, for example, followed by a pause and then two more notes can be treated as two two-note words or one four-note word.
The Sigmund pitch tracking patch generates microtonal values, which means that the MIDI values generated have a fractional component; the values are continuous and not limited to fixed notes of traditional Western music, such as those on a piano keyboard. To map onto the Solresol language, the notes are rounded to the nearest whole number (i.e. mapped to piano keys) and then further restricted to only the C Major scale notes (i.e. only the white notes on the keyboard). This is accomplished by simply replacing notes that are not in C Major with the nearest note; for example, F sharp is replaced by F (there are some user settable parameters to determine the details of this process).
The translated words can be displayed on the screen in real-time and saved to a file for later off-line use.
The translation device generates colours in addition to translating birdsong to words. The colours’ hue is fixed in accordance with Sudre’s definition – one rainbow colour for each of the seven notes of the C Major scale. As a result, the four-note Solresol ‘words’ are output as a set of four colour blocks, as well as the looked-up English translation.
The device’s output of musical notes varies depending on the version as described below.
2.2 Software and hardware versions
Three different versions were developed:
-
Full -
Lite -
Pi
The Full version is best suited for a desktop computer and was tested on a Windows 10 PC. There is a separate user control panel and graphical display. The Gem package (Graphical Environment for Multimedia) is required for the display and a MIDI player is required to play music (we used a virtual player). On a Windows 10 laptop, the Lite version was used. The Pi version ran on a Raspberry Pi (we made two – one using version 3B+ with a 3.5-inch screen and one with version 4 and a 7-inch screen).
The Full version was created first, and it was gradually simplified after experimentation to produce the other versions. This also reflected a shift in thinking as the project moved away from complex desk-based systems that used recorded sound and toward simpler, more portable systems that could be used in real time ‘in the wild’.
2.2.1 Full version
The Full version generates music as well as a separate graphical output of text and coloured squares using Gem. This version allows one to change the brightness and saturation of the colours in proportion to the volume of the sound.
In addition to the notes in a single octave of C Major, the MIDI stream is used for music generation at an early stage. This stream is not limited to a single octave, and the notes can be changed to map to different keys (or modes), as well as the overall pitch level. As an accompaniment, several drones (continuous notes) can be played. The MIDI note stream can be routed to a virtual MIDI player, which can emulate a variety of musical instruments selected by the user. The MIDI stream can be saved for later use.
The user interface is depicted here in a labelled screenshot, and it can be seen in action at the top of the page. Opening any of the patches to reveal the underlying code will give more control (see example - on this page to the right).
Essentially, the signal flows through the system in the order depicted in the screenshot, with user control available at several points:
-
Input: choose a sound file or live mode to input from a microphone.
- Optional signal pre-processing: high or low frequency cut-off and signal delay. Combine the left and right channels or choose one. Play the input signal with adjustable volume before or after pre-processing.
- Choose between continuous and ‘call and respond’ mode.
- The analyser module. This is the Sigmund object, which includes graphical controls for parameters that govern how the input sound signal is converted to notes.
- Module for modifying the analyser’s note stream output to a specific scale or mode. This also sends the C Major notes required for the Translator.
- Optionally play the C Major Translator notes. This can be done in ‘just intonation‘, which is a purer set of notes than is usually played (‘equal temperament’).
- Turn MIDI on/off and control volume.
- Save MIDI to file.
- Choose MIDI instrument.
- Choose drone accompaniment.
- Graphical display set up.
- Control the graphical display text. Can select font, including Echonian.
- Control the saturation and colour lightness. This is linked to the volume of the sound. Control the position of the displayed squares.
- Show the C Major notes as colours.
- This is the module for grouping notes into words. The amount of time to wait for the next note can be set, but it is reset after four notes are received. If this time expires, a word made up of less than four notes is formed.
- Options to save text to file.
- Solresol words output.
2.2.2 Lite and Pi versions
The Lite version and Pi versions are similar. Essentially, these are pared-down versions of the Full version that focus on the translation device’s key purpose. These versions lack MIDI output as well as separate graphical output using Gem (instead, only graphical elements from Pure Data are used). Otherwise, the sound analysis and translation are performed using the same underlying patches as in the Full version. To control parameters, the user must first open the patches. There is no way to change font, colour saturation, or brightness (see example screenshots below).
2.3 Use of device
Experimentation was conducted to optimise hardware and software parameters for off-line and live recording (see appendix).
The following farmland and garden birds were recorded: Wren, Blackbird, Song Thrush, Mistle Thrush, Sparrow, and Robin. Additional recordings were made at a seabird colony (St Abb's Head, Scotland). Other recordings were downloaded from xeno-canto (specialist site for birdsong) or freesound.
2.4 Producing outputs
The video at the top of the screen shows how the various controls in the Full version are used.
Several birdsong recordings were used to create videos with varying music outputs and colour schemes. Some used English words, while others used Enochian (Asprem 2012).
The text was saved to a file and then used to create poetry. We made only minor changes to accomplish this, adding no more than two lines, changing tense sparingly, adjusting line breaks, deleting some words, and repeating some phrases. On the outputs page, examples include using a speed-reading app, a text-to-speech app (see appendix for details), and a performance reading. A few tricks were used to adjust the output when using the apps, such as judicious use of punctuation, line breaks, and repeated words.
Presentation of the work at Allenheads Contemporary Arts (UK) as part of their 2019 Continuum season of “inspiring art, science and speculative fiction”.
The artist demonstrating the use of the device outdoors. Sound is being recorded with the Telinga parabola and stereo mic and fed via the ZOOM f4 to the Pi version of the translator.
Upper video
Version 3B+ with 3.5 inch screen
Lower video
Version 4 with 7 inch screen (note: video has no sound)
