Informative Sound Assists Timing in a Simple Visual Decision-Making Task

Keith Nesbitt

In this study, we examined the design of informative sound to assist timing in a simple, multimodal, game task. The game was initially designed as a series of eight-second, visual decision-making tasks. Players could respond quickly, with more risk, or wait longer for more information, thus reducing their risk of being incorrect. The player was scored by counting the number of correct attempts they completed in a five-minute interval. Waiting longer made the task easier but reduced the number of opportunities they had to repeat the task within the time limits of a game. In general terms, this game was designed to examine the way a player balances risk and reward. Unfortunately, the visual version of the game introduced the unintentional risk of a “safe” player waiting too long, timing out and failing to respond at all. This caused the risk-reward structure of the game to be out of balance. This study reports on the subsequent design and evaluation of a simple informative sound that was added to the game. The intention was to address this time-out issue without impacting other aspects of player performance. A within-subject experiment with 48 participants measured the response time, number of timeouts, and success rate of each player in a sequence of tasks. We measured player performance under three conditions: no sound (visual-only), constant (non-informative) sound, and increasing amplitude (informative) sound. We found that the increasing sound display significantly reduced timeouts when compared with the visual-only and constant sound versions of the task. Importantly, this reduction in timeouts did not impair the players’ performances in terms of their success rate or response time.

1. Introduction 5. Discussion

2. A Simple Decision-Making Task 6. Conclusion

3. Method Biographies

4. Results References

3. Method

3.1. Hypothesis

Our hypothesis was that, compared to the visual-only task and the non-informative constant sound, players would have reduced timeouts in the informative sound display. This is the display where sound increased in amplitude over the 8 second trial. We also hypothesized that there would be no reduction in performance on the task with increasing sound compared to the visual-only and constant sound display.

3.2. Participants

We tested the game with 48 psychology students, computing students, and academics from the University of Newcastle. Psychology students were awarded course credits for their participation. The study was approved by the University of Newcastle’s Research Ethics Committee. Participants were predominantly male (71%) and ranged in age between 18-54 years, with an average age of 21. All participants had normal, or corrected normal, vision and hearing.

3.3. Experimental Task

The sound-augmented Buckets game was tested by comparing player performance using a repeated measure design with three conditions: no sound, constant soundand increasing sound. The first – “no sound” – condition consisted of the original, visual-only version of the Buckets game. Different sounds were used for the second and third conditions to allow the user to differentiate them. The second condition used a “constant sound,” generated using a sine wave of fixed frequency (440 Hz). Musically, this corresponds to the tone A in the fourth octave. This sound was played at constant amplitude throughout the 8 seconds of each attempt. This condition was intended as a further control for comparing performance with the “increasing sound” condition. The increasing sound was generated as a sine wave, with the frequency of the signal held constant at 261 Hz. Musically, this corresponds to middle C. The amplitude of the signal gradually increased, in a linear fashion, over the 8 seconds of each trial. The increasing sound was pre-recorded as an 8 second wav file that was triggered for play at the start of each trial. Both sounds are provided here:

AudioObject 1: Constant sound used in the experiment

AudioObject 2: Increasing sound used in the experiment

Each participant played in each of the three conditions: no sound, constant sound, and increasing sound. The order of the three conditions was counter-balanced across participants to control for learning and fatigue effects. Participants were randomly allocated an order of conditions. Regardless of the order, each participant received one minute of practice time in each condition before playing that condition competitively for 15 minutes. These 15 minutes were further divided into three blocks of five minutes each.

As each condition began, the participant was presented with the game rules. These rules emphasized the importance of both accuracy and speed in the task. At the end of each individual trial, the player received feedback for 500 milliseconds regarding their choice of buckets. A green tick was displayed below the target square if the decision was correct. This was accompanied by a cash-register sound. If the player chose incorrectly, a red cross was displayed below the target, accompanied by a sigh-of-disappointment sound. If the player timed out, an alarm clock was displayed, and a typical alarm clock sound was played.

Both the player’s response time or timeout status for each attempt were recorded for later analysis. Data regarding the correct target bucket (1-4) and the player’s actual selected bucket (1-4) for each attempt were also saved. At the completion of each of the five-minute blocks, participants received summary information about their performance, namely, the number of correct and incorrect responses. At the end of each sound condition, the subject was allowed a two-minute break before commencing the next assigned condition. Generally speaking, subjects completed the experiment in about 60 minutes.

3.4. Apparatus

The experiment was conducted within a computer laboratory. Upon arrival, each participant was assigned a workstation that displayed the Buckets game. All data was collected using an Apple Mac Pro running OS X 10.8 Mountain Lion. The game was played online using the Mozilla Firefox (Version 22) web browser and Flash Player (version 11.4). Each participant wore a full-sized headphone (AKG K44) during the whole experiment, even during the “no sound” condition. During the experiment, the volume level on the operating system was set to the lowest possible volume (1 out of 16 bars) for all players.