Psychology experiments enter the post-PC era: OpenSesame now runs on Android

I’ve mentioned OpenSesame briefly on here before, but for those of you who weren’t keeping up, it’s a pretty awesome, free psychology experiment-developing application, built using the Python programming language, and it has a lot in common with PsychoPy (which is also awesome).

The recently-released new version of OpenSesame has just taken an important step, in that it now supports the Android mobile operating system, meaning that it can run natively on Android tablets and smartphones. As far as I’m aware, this is the first time that a psychology-experimental application has been compiled (and released to the masses) for a mobile OS.

This is cool for lots of reasons. It’s an interesting technical achievement; Android is a very different implementation to a desktop OS, being focused heavily on touch interfaces. Such interfaces are now ubiquitous, and are much more accessible, in the sense that people who may struggle with a traditional mouse/keyboard can use them relatively easily. Running psychology experiments on touch-tablets may enable the study of populations (e.g., the very young, very old, or various patient groups) that would be very difficult with a more ‘traditional’ system. Similarly, conducting ‘field’ studies might be much more effective; I can imagine handing a participant a tablet for them to complete some kind of task in the street, or in a shopping mall, for instance. Also, it may open up the possibility of using the variety of sensors in modern mobile devices (light, proximity, accelerometers, magnetometers) in interesting and creative ways. Finally, the hardware is relatively cheap, and (of course) portable.

I’m itching to try this out, but unfortunately don’t have an Android tablet. I love my iPad mini for lots of reasons, but the more restricted nature of Apple’s OS means that it’s unlikely we’ll see a similar system on iOS anytime soon.

So, very exciting times. Here’s a brief demo video of OpenSesame running on a Google Nexus 7 tablet (in the demo the tablet is actually running a version of Ubuntu Linux, but with the new version of OpenSesame it shouldn’t be necessary to replace the Android OS). Let me know in the comments if you have any experience with tablet-experiments, or if you can think of any other creative ways they could be used.

TTFN.

 

Open-Source software for psychology and neuroscience

Researchers typically use a lot of different pieces of software in the course of their work; it’s part of what makes the job so varied. Separate packages might be used for creating experimental stimuli, programming an experiment, logging data, statistical analysis, and preparing work for publication or conferences. Until fairly recently there was little option but to use commercial software in at least some of these roles. For example, SPSS is the de facto analysis tool in many departments for statistics, and the viable alternatives were also commercial – there was little choice but to fork over the money. Fortunately, there are now pretty viable alternatives for cash-strapped departments and individual researchers. There’s a lot of politics around the open-source movement, but for most people the important aspect is that the software is provided for free, and (generally) it’s cross-platform (or can be compiled to be so). All that’s required is to throw off the shackles of the evil capitalist oppressors, or something. 

So, there’s a lot of software listed on my Links page but I thought I’d pick out my favourite bits of open-source software, that are most useful for researchers and students in psychology.

First up – general office-type software; there are a couple of good options here. The Open Office suite has been around for 20 years, and contains all the usual tools (word processor, presentation-maker, spreadsheet tool, and more). It’s a solid, well-designed system that can pretty seamlessly read and write the Microsoft Office XML-based (.docx, .pptx) file formats. The other option is Libre Office, which has the same roots as Open Office, and similar features. Plans are apparently underway to port Libre Office to iOS and Android – nice. The other free popular options for presentations is, of course, Prezi.

There are lots of options for graphics programs, however the two best in terms of features are without a doubt GIMP (designed to be a free alternative to Adobe Photoshop) and Inkscape (vector graphics editor – good replacement for Adobe Illustrator). There’s a bit of a steep learning curve for these, but that’s true of their commercial counterparts too.

Programming experiments – if you’re still using a paid system like E-Prime or Presentation, you should consider switching to PsychoPy - it’s user-friendly, genuinely cross-platform, and absolutely free. I briefly reviewed it before, here.  Another excellent option is Open Sesame.

For statistical analysis there are a couple of options. Firstly, if you’re a SPSS-user and pretty comfortable with it (but fed up of the constant hassles of the licensing system), you should check out PSPP; a free stats program designed to look and feel like SPSS, and replicate many of the functions. You can even use your SPSS syntax – awesome. The only serious issue is that it doesn’t contain the SPSS options for complex GLM models (repeated measures ANOVA, etc.). Hopefully these will be added at some future point. The other popular option is the R language for statistical computing. R is really gaining traction at the moment. The command-line interface is a bit of a hurdle for beginners, but that can be mitigated somewhat by IDEs like R-Commander or RStudio.

For neuroscience there’s the NeuroDebian project – not just a software package, but an entire operating system, bundled with a comprehensive suite of neuroscience tools, including FSL, AFNI and PyMVPA, plus lots of others. There really are too many bits of open-source neuro-software to list here, but a good place to find some is NITRC.org.

So, there you are people; go open-source. You have nothing to lose but your over-priced software subscriptions.

TTFN.

BPS Hackathon – 21st June; LaTeX, R, Python goodness

Very exciting news here: I’ve just been invited to the first British Psychological Society (Maths, Statistics and Computing Section) Psychology open textbook hackathon!

Inspired by this event (where people got together and wrote an open-source maths textbook in a weekend) the day aims to raise awareness and skills, as well as perhaps produce some usable output.

The organisers are Thom Baguley of Nottingham Trent University (and the Serious Stats blog and book) and Sol Nte of Manchester University. They’ve very kindly invited me as a guest, so I’ll be hanging out and learning some new tricks myself, I’m sure.

Here’s the flyer for the event, with sign-up details etc. It’s free, but strictly limited to 20 places – if you’re keen, best be quick… (click the pic below for a bigger version):

 

Comment on the Button et al. (2013) neuroscience ‘power-failure’ article in NRN

Statistical Spidey knows the score.

An article was published in Nature Reviews Neuroscience yesterday which caused a bit of a stir among neuroscientists (or at least among neuroscientists on Twitter, anyway). The authors cleverly used meta-analytic papers to estimate the ‘true’ power of an effect, and then (using the G*Power software) calculated the power for each individual study that made up the meta-analysis, based on the sample size of each one. Their conclusions are pretty damning for the field as a whole: an overall value of 21%, dropping to 8% in some sub-fields. This means that out of 100 studies that are conducted into a genuine effect, only 21 will actually demonstrate it.

The article has been discussed and summarised at length by Ed Yong, Christian Jarrett, and by Kate Button (the study’s first author) on Suzy Gage’s Guardian blog, so I’m not going to re-hash it any more here. The original paper is actually very accessible and well-written, and I encourage interested readers to start there. It’s definitely an important contribution to the debate, however (as always) there are alternative perspectives. I generally have a problem with over-reliance on power analyses (they’re often required for grant applications, and other project proposals). Prospective power analyses (i.e. those conducted before a piece of research is conducted, in order to tell you how many subjects you need) use an estimate of the effect size you expect to achieve – usually derived from previous work that has examined a (broadly) similar problem using (broadly) similar methods. This estimate is essentially a wild shot in the dark (especially because of some of the issues and biases discussed by Button et al., that are likely to operate in the literature), and the resulting power analysis therefore tells you (in my opinion) nothing very useful. Button et al. get around this issue by using the effect size from meta-analyses to estimate the ‘true’ effect size in a given literature area – a neat trick.

The remainder of this post deals with power-issues in fMRI, since it’s my area of expertise, and necessarily gets a bit technical. Readers who don’t have a somewhat nerdy interest in fMRI-methods are advised to check out some of the more accessible summaries linked to above. Braver readers – press on!

An alternative approach used in the fMRI field, and one that I’ve been following when planning projects for years, is a more empirical method. Murphy and Garavan (2004) took a large sample of 58 subjects who had completed a Go/No-Go task and analysed sub-sets of different sizes to look at the reproducibility of the results, with different sample sizes. They showed that reproducibility (assessed by correlation of the statistical maps with the ‘gold standard’ of the entire dataset; Fig. 4) reaches 80% at about 24 or 25 subjects. By this criterion, many fMRI studies are underpowered.

While I like this empirical approach to the issue, there are of course caveats and other things to consider. fMRI is a complex, highly technical research area, and heavily influenced by the advance of technology. MRI scanners have significantly improved in the last ten years, with 32 or even 64-channel head-coils becoming common, faster gradient switching, shorter TRs, higher field strength, and better field/data stability all meaning that the signal-to-noise has improved considerably. This serves to cut down one source of noise in fMRI data – intra-subject variance. The inter-subject variance of course remains the same as it always was, but that’s something that can’t really be mitigated against, and may even be of interest in some (between-group) studies. On the analysis side, new multivariate methods are much more sensitive to detecting differences than the standard mass-univariate approach. This improvement in effective SNR means that the Murphy and Garavan (2004) estimate of 25 subjects for 80% reproducibility may be somewhat inflated, and with modern techniques one could perhaps get away with less.

The other issue with the Murphy and Garavan (2004) approach is that it’s not very generalisable. The Go/No-Go task is widely used and is a ‘standard’ cognitive/attentional task that activates a well-described brain network, but other tasks may produce more or less activation, in different brain regions. Signal-to-noise varies widely across the brain, and across task-paradigms, with simple visual or motor experiments producing very large signal changes and complex cognitive tasks smaller ones. Yet another factor is the experimental design (blocked stimuli, or event-related),  the overall number of trials/stimuli presented, and the total scanning time for each subject, all of which can vary widely.

The upshot is that there are no easy answers, and this is something I try to impress upon people at every opportunity; particularly the statisticians who read my project proposals and object to me not including power analyses. I think prospective power analyses are not only uninformative, but give a false sense of security, and for that reason should be treated with caution. Ultimately the decision about how many subjects to test is generally highly influenced by other factors anyway (most notably, time, and money). You should test as many subjects as you reasonably can, and regard power analysis results as, at best, a rough guide.

Another miscellaneous grab-bag of goodies, links ‘n’ stuff

In lieu of a ‘proper’ post (forgive me, dear readers, the vicious task-masters at my proper job have been wielding the whip with particular alacrity recently) I’m putting together a list of links to cool things that I’ve come across lately.

So, in no particular order:

Tal Yarkoni’s outstanding Neurosynth website has now gone modular and open-source, meaning you can embed the code for the brain-image viewer into any website, and use it to present your own data – this is seriously cool. Check out his blog-post for the details.

An interesting little comment on “Why Google isn’t good enough for academic search”. Google scholar tends to be my first port of call these days, but the points made in this discussion are pretty much bang-on.

A fantastic PNAS paper by Kosinski et al. (2013; PDF) that demonstrates that personal attributes such as sexual orientation, ethnicity, religious and political views, some aspects of personality, intelligence and many others, can be automatically and accurately (to a fairly startling degree, actually) predicted merely from analysis of Facebook ‘Likes’. A fantastic result, that really demonstrates the value of doing research using online data.

Next up is Google Refine – an interesting little idea from Google intended to assist with cleaning up and re-formatting messy data. Looks like it could be promisingly useful.

A really seriously great website on the stats language R, designed to make the transition for SPSS and SAS users as easy as possible – very clear, very nicely explained. Beautiful stuff.

Another cool website called citethisforme.com; you fill in fields (author, title, etc.) for sources you wish to cite, and it creates a perfectly formatted bibliography for you in the style (APA, Harvard etc.) you choose. A cool idea, but in practice, filling out the fields would be incredibly tedious for anything more than a few sources. Good place to learn about how to format things for different types of reference though.

I’ve previously written about the use of U-HID boards for building USB response devices; I’ve just been made aware of a similar product called Labjack, which looks even more powerful and flexible. A Labjack package is included in the standard distribution of PsychoPy too, which is cool. I’m becoming more and more a fan of PsychoPy by the way – I’m now using it on a couple of projects, and it’s working very well indeed for me.

Now a trio of mobile apps to check out. Reference ME is available for both iOS and Android, and creates a citation in a specific style (Harvard, APA, etc.) when you scan the barcode of a book – very handy! The citations can then be emailed to you for pasting into essays or whatever.

The Great Brain Experiment is a free app from the Wellcome Trust (download links for both iOS and Android here) created in collaboration with UCL. The aim is to crowdsource a massive database on memory, impulsivity, risk-taking and other things. Give it a whirl – it’s free!

Lastly Codea is a very cool-looking iPad-only app that uses the Lua programming language to enable the (relatively) easy development and deployment of ‘proper’ code, entirely on the iPad. Very cool – Wired called it ‘the Garage Band of coding’, and while it’s probably not quite that easy to use, it’s definitely worth checking out if you want to use your iPad as a serious development tool.

If you’re still hungry for more internet goodies, I encourage you most heartily to check out my Links page, which is currently in an ongoing phase of rolling development (meaning, whenever I find something cool, I put it up there).

TTFN.

 

Warhol brains.

Here is a pretty Warhol-esque picture I made using a) my own head, b) a Siemens Verio MRI scanner, c) Osirix and d) GIMP.

(Clicky for bigness)

Digital audio and video basics – two excellent videos

I’ve just come across two outstanding tutorial videos over on xiph.org - an open-source organisation dedicated to developing multimedia protocols and tools. So, the first one covers the fundamental principles of digital sampling for audio and video, and discusses sampling rates, bit depth and lots of other fun stuff – if you’ve ever wondered what a 16-bit, 128kbps mp3 is, this is for you.

The second one focusses on audio and gets on to some more advanced topics, about how audio behaves in the real world.

They’re both fairly long (30 mins and 23 mins respectively) but well worth watching. If you’re just getting started with digital audio and/or video editing and production, these could be really useful.

TTFN.

Why brain training is (probably) pernicious hogwash

The only treadmill your brain should be on is a hedonic one.

So-called brain-training tools seem to have exploded in the last few years; one estimate puts it at a $6 billion market by 2020. It’s clearly become a major industry, but what’s less clear is exactly what it does, and if it even works. The typical procedure seems to be to engage in short games, puzzles and working-memory-type tasks, and these are supposed to produce long term changes in attention, engagement and general fluid intelligence.

Whether this is actually true or not is a matter of some debate. I’m not a specialist in this area, but the received wisdom appears to be that training on specific tasks does improve performance – on those tasks. There seems to be little generalisation to other tasks, and even less to domain-general abilities like executive processing, or working memory. A high-profile study by Adrian Owen and colleagues (2010) reported exactly that – benefits in the tasks themselves, but little (if any) general benefits. A previous study from PNAS in 2008 does seem to contradict this, and reports an increase in fluid intelligence as a result of working-memory training – not only that, but they claim a dose-dependent effect, that is, more training = more increase in intelligence. The gains in that study were relatively small, and it should be also noted that the control group also apparently increased their intelligence somewhat over the same period as the experimental group – curious. There are lots of other studies around, but many have issues; small samples, poorly-controlled etc. etc.

So, the jury’s still very much out (though personally, I’m on the side of the skeptics on the issue). This hasn’t stopped a bewildering array of businesses starting up, making all kinds of wild claims, and playing on the fears of educators and parents that perhaps if they don’t provide these kinds of programs, their kids will be slipping behind the rest. All these companies have glossy, highly-polished, ethnically-balanced websites with testimonials, and lots of links to science-y looking videos that present their program as the only scientifically-proven method of increasing your child’s intelligence. A brief browse through some of these companies websites reveals that they range from the absurd (QDreams! Success at the speed of thought!) to the very, very slick indeed (e.g. Lumosity). Other examples are Cogmed (seems to be backed by Pearson publishers and, to its credit, links to a list of semi-relevant research papers), and the very simplistic PowerBrain Education - which seems to involve getting kids to do some odd-looking arm-shaking exercises. There’s literally hundreds of these companies. Some of them even seem to cater to businesses who want their employees to do these ‘exercises’.

LearningRX definitely falls into the slick category. According to this New York Times article it has 83 physical store-front franchises across the USA, where people can come to pay $80-90 an hour for one-on-one training, and they market this to parents as an alternative to traditional tutoring. A quick glance at their Scientific Advisory Board is pretty revealing – I count only one (clinical) psychologist, and a grab-bag of other professionals – mostly teachers (qualified to Masters level) with an optometrist, a chemical engineer and an audiologist. Not a single neuroscientist, and only a few qualified at doctorate level.

I’m not trying to be unnecessarily snobby about their qualifications here, I’m suggesting that the claims they make for their brain-training programs (literally: it will change your child’s life) are big ones, and we might expect that the people who developed it might be qualified in some area of brain-science. If it really, clearly worked, then of course it wouldn’t matter exactly who developed it, and what their qualifications were, but  there’s definitely reasonable doubt (if not outright disbelief) over its effectiveness.

And this is the important point. People are spending money on this - big money. Whether that’s a hard-pressed family struggling to find an extra $90 a week for their kid to have a session at one of LearningRX’s centres, or an education board deciding to institute one of these programs in its schools. Education budgets are tight enough, but these kinds of programs are being heavily invested in, and I can see why – they promise to make kids smarter, better-behaved, more attentive, and all you have to do is sit them in front of a special computer game for an hour a week. That must seem like a pretty attractive proposition for teachers. Unfortunately, if they really don’t work, then that money could be better spent on books, or musical instruments, or something else which might genuinely enrich the kids’ lives.

There’s a long and venerable history of unscrupulous people making money from pseudo-neuroscience – back in the 19th Century phrenology was described as “The science of picking someone’s pocket, through their skull.” I’d like to believe that some of these companies have a solid product that actually made a difference, but they all seem to have the whiff of snake-oil about them. For now I’m very much of the opinion that you’d probably be better off learning the piano, or Japanese, or even playing the latest Call of Duty. If you were really ambitious you could even try and get your kid to (Heaven forfend!) read the odd book now and again.

TTFN.

 

**Update 07/02/13**

I put that last sentence that mentions Call of Duty in there as a bit of flippancy, but I’ve since been informed (by Micah Allen on Twitter) of some evidence that playing action video games can indeed improve some cognitive processes such as the accuracy of visuo-spatial attention and reaction times. These results mostly originate from a single lab and so are in need of replication, but still – interesting. (I still reckon you’re probably better off with a good book though.)

Free, interactive MRI courses from Imaios.com (plus lots of other medical/anatomy material too)

A very quick post to point you towards a really fantastic set of online, interactive courses on MRI from a website called Imaios.com – a very nice, very slick set of material. The MRI courses are all free, but you’ll need to register to see the animations. Lots of other medical/anatomy-related courses on the site too – some free, some ‘premium’, and some nice looking mobile apps too.

iPad app in development to help with macular degeneration

I’ve written before about iPad apps useful for vision research, but I’ve just come across a new vision-related app, so new in fact that it’s still in the development/testing phase. It’s been produced by my old Colleague Prof. Robin Walker at Royal Holloway University and is designed as a rehabilitative tool for people with Macular Degeneration (MD).

Age-related MD is by far the most common form of blindness/vision-loss in people over 50, and involves degeneration of the visual sensitivity of the centre portion of the retina – the part of the eye which has the highest density of rods and cones. This makes tasks such as reading and recognising faces more and more difficult as the condition progresses. One way of mitigating the effects is to try and use portions of the retina which are less affected, i.e. the periphery. For reading, the ‘eccentric-vision’ and ‘steady-eye’ techniques involve fixating at a point and then moving the text through areas of the visual field which are less affected. These techniques require some practice to counteract the natural tendency to make eye-movements when reading, and it’s this training process that the app is intended to help with.

Read more about the app here, and there’s also a (pay-walled) article in the British Journal of Opthalmology here.