Inattentional deafness, sound editing and auditory gorillas

A lovely little paper has just got in press in Cognition - Gorillas we have missed: Sustained inattentional deafness for dynamic events, by a couple of ex-colleagues of mine at Royal Holloway – Polly Dalton and Nick Fraenkel. I thought I’d do a brief write-up, as it describes a couple of great experiments that involved some nifty bits of audio recording and editing; something I’ve been meaning to get around to writing on for some time.

The paper is based on an older, visual, effect described by Simons and Chabris (1999) (PDF here) and termed ‘inattentional blindness’. Essentially, this paper demonstrated that participants can fail to notice a highly salient visual stimulus, if their attention is directed towards some other aspect of the visual scene. The stimulus that these authors used was a video of six people passing basketballs to each other in a complex sequence, and the task for the participants was to count the number of passes made. During the movie, a person in a gorilla suit walked through the middle of the basketball players. Despite the bizarre nature of the manipulation, a substantial proportion of participants (between 30% and 50% depending on the exact condition) simply failed to notice the very obvious ‘gorilla in the midst’. You can see one of the videos used in the experiment below, and there’s also a nice interview with Daniel Simons where he talks about the experiment here.

Schematic of the auditory stimulus used in the experiment, reproduced from Figure 1 of Dalton and Fraenkel (2012).

So, what Polly and Nick did in their new paper is to take this visual effect, and cleverly translate it into the auditory domain. They made recordings of a complex auditory scene with two pairs of conversations happening at once – one pair of female voices and one pair of male voices – with both conversation pairs moving around the auditory ‘space’ during the recording. Also present during the recording was an additional (male) voice that walked through the scene repeatedly saying “I’m a gorilla, I’m a gorilla…” for 19 seconds.The majority of participants (90%) who were cued to listen to the male conversation did notice the ‘auditory gorilla’, however when people were cued to listen to the female conversation only 30% reported noticing the gorilla. The implication is that when we are attending to one category of stimulus (i.e. female voices) we can fail to notice even prominent stimuli which belong to an unattended category (male voices). You can try it yourself, using the below video, which contains an edited version of their stimulus. For the full effect you’ll need to use headphones:

This is clearly a complex auditory stimulus, and creating it involved some really interesting techniques. The recordings were made using an ‘artificial head’ – a (roughly) human-head-shaped recording device, with high-quality microphones positioned in each ear. Using such a device for binaural recordings is important, because the shape of the head (and the outer ear) produces subtle frequency-based distortions in perceived sounds, and the brain uses these cues to localise sounds in 3D space. The separate tracks from the two microphones form a single stereo track and when listened to on headphones, recordings of this type tend to produce a very natural-sounding audio environment. You can read more about this technique here. The two attended conversations were recorded separately from the “I’m a gorilla” stimulus, and the two recordings then mixed together to create the final stimulus – this enabled independent manipulation of the spatial placement of the gorilla stimulus within the scene (which was reversed in experiment 2).

This mixing of the two separate recordings was done using Reaper, a piece of software classed as a Digital Audio Workstation (DAW). DAW devices used to be primarily hardware based, and a digital audio lab used to include racks of equipment; samplers, sequencers etc. The vast majority of these functions can be reproduced with software nowadays. I haven’t used it myself, but Reaper looks to be a fantastic piece of professional-grade software, and is available very cheaply ($60 for an individual/educational licence). DAW software allows almost endless recording and editing possibilities for sound recordings, including studio-based recording of music, applying effects and filters, changing pitch and tempo, mixing and mastering of recordings, and even synthesis (e.g. of pure-tones, for use as auditory cues in experiments).

While Reaper looks great, my recommendation for this kind of software is Audacity, an incredibly full-featured, cross-platform (Windows, Mac and Linux), and entirely free audio editor/recorder. I’ve used Audacity a lot for really basic editing/synthesis tasks, but it has an impressive array of features and has (apparently) been used to record, mix and master entire albums. If you have some sound editing task to accomplish, it would definitely be worth investigating whether you can easily achieve it with Audacity before you splurge on some more expensive, professional software. A good list of other free sound-related software is here.

That’s all for now – happy sound editing! TTFN.

PS. For more details of the Royal Holloway attention lab’s research see their webpage here.