How to analyse reaction time (RT) data: Part 1

Reaction time tasks have been a mainstay of psychology since the technology to accurately time and record such responses became widely available in the 70s. RT tasks have been applied in a bewildering array of research areas and (when used properly) can provide information about memory, attention, emotion and even social behaviour.

This post will focus on the best way to handle such data, which is perhaps not as straightforward as might be assumed. Despite the title, I’m not really going to cover the actual analysis; there’s a lot of literature already out there about what particular statistical tests to use, and in any case, general advice of that kind is not much use as it depends largely on your experimental design. What I’m intending to focus on are the techniques the stats books don’t normally cover – data cleaning, formatting and transformation techniques which are essential to know about if you’re going to get the best out of your data-set.

For the purposes of this discussion I’ll use a simple made-up data-set, like this:

This table is formatted in the way that a lot of common psychology software (i.e. PsychoPy, Inquisit, E-Prime) records response data. From left-to-right, you can see we have three participants’ data here (1, 2, and 3 in column A), four trials for each subject (column B), two experimental conditions (C; presented in a random order), and then the actual reaction times (column D) and then a final column which codes whether the response was correct or not (1=correct, 0= error).

I created the data table using Microsoft Excel, and will do the processing with it too, however I really want to stress that Excel is definitely not the best way of doing this. It suits the present purpose because I’m doing this ‘by hand’ for the purposes of illustration. With a real data-set which might be thousands of lines long, these procedures would be much more easily accomplished by using the functions in your statistics weapon-of-choice (SPSS, R, Matlab, whatever). Needless to say, if you regularly have to deal with RT data it’s well worth putting the time into writing some general-purpose code which can be tweaked and re-used for subsequent data sets.

The procedures we’re going to follow with these data are:

1. Remove reaction times on error trials
2. Do some basic data-cleaning (removal of outlying data)
3. Re-format the data for analysis

1. Remove reaction times on error trials

As a general rule, reaction times from trials on which the participant made an error should not be used in subsequent analysis. The exceptions to this rule are some particular tasks where the error trials might be of particular interest (Go/No-Go tasks, and some others). Generally though, RTs from error trials are thought to be unreliable, since there’s an additional component process operating on error trials (i.e. whatever it was that produced the error). The easiest way of accomplishing this is to insert an additional column, and code all trials with errors as ‘0’, and all trials without an error as the original reaction time. This can be a simple IF/ELSE statement of the form:

IF (error=1) RT=RT,
ELSE RT=0

In this excel-based illustration I entered the formula: =IF(E2=1, D2,0) in cell F2, and then copied it down the rest of the column to apply to all the subsequent rows. Here’s the new data sheet:

2. Data-cleaning – Removal of outlying data

The whole topic of removing outliers from reaction time data is a fairly involved one, and difficult to illustrate with the simple example I’m using here. However, It’s a very important procedure, and something I’m going to return to in a later post, using a ‘real’ data-set. From a theoretical perspective, it’s usually desirable to remove both short and long outliers. Most people cannot push a button in response to, say, a visual stimulus in less than about 300ms, so it can be safely assumed that short RTs of, say, less than 250ms were probably initiated before the stimulus; that is, they were anticipatory. Long outliers are somewhat trickier conceptually – some tasks that involve a lot of effortful cognitive processing before a response (say a task involving doing difficult arithmetic) might have reaction times of several seconds, or even longer. However, very broadly, the mean RT for most ‘simple’ tasks tends to be around 400-700ms; this means that RTs longer than say, 1000ms might reflect some other kind of process. For instance, it might reflect the fact that the participant was bored, became distracted, temporarily forgot which button to push, etc. For these reasons then, it’s generally thought to be desirable to remove outlying reaction times from further analysis.

One (fairly simple-minded, but definitely valid) approach to removing outliers then, is to simply remove all values that fall below 250ms, or above 1000ms. This is what I’ve done in the example data-sheet in columns G and H, using simple IF statements of a similar form used for removal of the error trials:

You can see that two short RTs and one long one have been removed and recoded as 0.

3.  Re-format the data for analysis

The structure that most psychology experimental systems use for their data logging (similar to the one we’ve been using as an illustration) is not really appropriate for direct import into standard stats packages like SPSS. SPSS requires that one row on the data sheet is used for each participant, whereas we have one row-per-trial. In order to get our data in the right format we first need to sort the data, first by subject (column A), and then by condition (column C). Doing this sort procedure ensures that we know which entries in the final column are which – the first two rows of each subject’s data are always condition 1, and the second two are always condition 2:

We can then restructure the data from the final column, like so:

I’ve done this ‘by hand’ in Excel by cutting-and-pasting the values for each subject into a new sheet and using the paste-special > transpose function, however this is a stupid way of doing it – the ‘restructure’ functions in SPSS can accomplish this kind of thing very nicely. So, our condition 1 values are now in columns B:C and condition 2 values are in columns D:E. All that remains to do now would be to calculate summary statistics (means, variance, standard deviations, whatever; taking care that our 0 values are coded as missing, and not included in the calculations) for each set of columns (i.e. each condition) and perform the inferential tests of your choice (in this case, with only two within-subject conditions, it would be a paired t-test).

Next time, I’ll use a set of real reaction time data and do these procedures (and others) using SPSS, in order to illustrate some more sophisticated ways of handling outliers than just the simple high and low cutoffs detailed above.

TTFN.

How to make your own parallel port response boxes

I’ve previously written about the importance of response hardware for doing timing-accurate experiments – in a nutshell, anything you connect via USB is likely to be sub-optimal,* because the operating system only polls (or looks for an input) on the USB port some of the time – in Windows the USB polling rate is 125Hz, or every 8ms.

So, for accurate timing of responses, we want to use something other than the USB port. There are various options – my personal favourite is to use the 25-pin parallel (or ‘printer’) port. Some newer desktop models unfortunately don’t have parallel ports anymore, as they’re largely obsolete for connecting peripherals, however any model older than two or three years should have one – and you generally don’t need a super-fast, up-to-date computer for running stimulus programs and collecting data.

I needed a couple of response boxes for a project recently, and decided to just make them up myself. I came across this fantastic little paper  (PDF) which describes a simple method of taking apart a couple of standard computer mice, and rewiring the switches into a parallel port plug – this gives you up to six buttons. The circuit diagram is really, really simple:

Circuit Diagram for a six-button, parallel port response box. Reproduced from Voss, Leonhart and Stahl (2007).

It’s just the switches, and a 100-ohm resistor for each one, wired up to different pins on the data register of the port, with a common ground (pin 18). Honestly, if you were being lazy, you could even just forget about the resistors and it would still work fine. I decided not to take apart any mice, but just to use some buttons I bought off-the-shelf, as I only needed two for each box. Getting the right buttons is really important for this kind of thing – you want them to be a decent size, and have a good clicky-action, without being too difficult to depress. I also got some small plastic boxes, some multi-core cable (I used standard network cable as it’s quite stiff and robust, but almost anything will do), and some parallel port plugs. You can buy everything you need from Maplin or Radio Spares (Radio Shack, if you’re in the US) for about £10-15. I just drilled some holes in the boxes fairly roughly and secured the buttons there with a dab of epoxy resin, but you can get as fancy as you want in that respect.

The only really tricky bit is deciding which pins on the parallel port you want to wire your switches up to. This will largely be determined by which pins the software you’re going to use can read from. Psychology software like Inquisit or E-prime is able to read inputs from pins 2-9 on the data register (see below diagram) but it’s worth doing a bit of reading about the different pins on the parallel port and what they’re used for. A good place to start is here. Probably what you want to do is use one of the data pins for one pole of each switch, and wire the other pole to a common ground pin, as in the above diagram.

Parallel port pin diagram showing the three registers (data, status and control) and the ground pins (in green)

So there you have it – the most simple, inexpensive and accurate solution for recording response times in cognitive experiments. If you’re at all handy with a soldering iron you can probably knock up a couple of these in half an hour or so. If you’ve never done any electronics or soldering before, then this would be an ideal first project to get started with! This was my finished article:

Nice, huh? Happy soldering! TTFN.

*Not quite true – some of the expensive button-boxes you can buy from psychology software companies are USB, but have their own electronics inside them to get around this and time things accurately.

BrainDesire, Online Experiments, Neuro-business and a Plea for Openness

I’ve recently had some correspondence with the CEO of a company called Brain and Science which runs a site called BrainDesire. The site purports to use…

“…the latest discoveries in brain science, and applies them to dating, relationships and love. Discover BrainDesire and find out which men or women have a real potential for dating and for a serious relationship with you.”

It does this by an online test, where you enter the name of the person you are romantically interested in and the name is then used in a relatively simple priming experiment where the name is used as the prime and the task is a lexical decision. The idea is that the name of Mr or Ms Right leads to higher arousal, and therefore a faster reaction time for the lexical decision. Once the test is complete, your results are then available for a modest fee.

My initial reaction to this was “pffftt… yeah, right”, and expressing this opinion on twitter was what first attracted the attention of the CEO of the company. We ended up having quite a technical discussion by e-mail about the scientific basis of the procedure BrainDesire uses and my assessment of it is now much better informed. There are several potential issues with the test which were immediately apparent.

Firstly, the scientific basis of the test. It’s largely based on a 2006 paper by Bianchi-Demicheli, Grafton and Ortigue which demonstrates the reaction time speeding effect when a beloved’s name is used as a prime. These authors are well respected scientists who generally do outstanding work and I have no reason to suppose that there is anything wrong with the paper (yes, I could get nit-picky about the methods, but I could do that with almost any paper). Incidentally, Scott Grafton and Stephanie Ortigue comprise the Scientific Advisory Board of the company. What I do have an issue with though, is that this result has never really been convincingly replicated. There’s another paper from 2007 which uses the same task in a fMRI scanner, and pretty-much replicates the result, however the effects are much weaker and the sample size is much smaller, plus only women were used in this experiment. Not totally convincing.