Running Experiments 1: Timing is Everything.

There is a tide in the affairs of men.
Which, taken at the flood, leads on to fortune;

Julius Caeser, Act 4, scene III

This will be the first in a (probably fairly lengthy) series of posts about how to design, program and run a successful psychology experiment. For this initial post I want to go over some basics about how computers work, and what that means for running successful experiments. Of course there are many different kinds of experiments that it’s possible to run, but for the purposes of the present discussion I’m going to use as an example a canonical kind of cognitive experiment, where the dependent variable is reaction time, measured using a button-press. This kind of experiment is still widely-used, in paradigms like the dot-probe attentional task, and the Implicit Association Test.

The first thing you need to understand is that modern operating systems are very, very complicated. When you boot up Windows (for example) all you see at first is a nice, clean, uncluttered desktop, but examining the Windows Task Manager reveals a whole host of ‘background’ processes which are buzzing away invisibly all the time. These might be search indexers, printer services, network connections, anti-virus software, firewalls, and a whole mess of other stuff. If you then open a few different applications (a web browser with a few tabs open, Microsoft Word, an email program, some instant messaging application) the computer has a few more processes to juggle, as well as all the background stuff. This is normally fine, as long as you have enough RAM to handle all the requirements of these different processes – modern OSs are multi-tasking, meaning they can handle having lots of things going on at once.

The Joy of Stats

Statistics – the very word is guaranteed to bring a shudder of terror to the average undergraduate, and even full-grown lecturers have been known to quake in fear before its awesome power. Most psychology undergraduates don’t come from a hard-science or mathematics background, and statistics are probably the number one thing that they struggle with during their psychology courses. Personally, I got through my undergraduate stats exams with a mixture of vague understanding and rote memorisation, and it was only during my PhD that I actually started learning how to do things properly and, more importantly, actually understanding what I was doing, and why.

This is not the place to give any detailed information on the basics of statistics. That kind of material has been covered many, many times before by people infinitely more qualified than I. For that kind of stuff, a good place to start would be Andy Field’s book, available here. Andy explains things very clearly and is actually a very nice chap as well. What I’d like to do instead is do a quick run-down of popular stats software, and point out some resources which can help if you run into trouble.