The Open Data Pitfall II - Now With Data

Yesterday I wrote something on why I think providing unrestricted access data from psychological experiments, as advocated by some, is not a good idea. Today I was in the opportunity to actually collect some data surrounding this issue, from the people who are neglected in this discussion: the participants.

I have used Mentimeter to ask the 60 first year students who showed up for my Biopsychology lecture whether they would participate in an experiment of which the data would be made publicly available.

At the beginning of the lecture, I gave a short introduction on open data. I referred to the LaCour case, and to Wicherts et al.’s work on the lack of willingness to share data, and emphasized the necessity of sharing data. I also mentioned that there is a debate going on on how data should be shared. I mentioned that some researchers are in favour of storing data in institutional repositories, whereas other researchers are in favour of posting data on publicly accessible repositories. I then explicitly told I would give my thoughts on the matter after I asked them two short questions via Mentimeter.

I read out two vignettes to the students:

1. “Imagine you signed up via Sona for one *name of researcher* studies on sexual arousal. Data of the study will be shared with other researchers. The dataset will be anonymized – it may contain some information such as your gender and age, but no personally identifiable information. Would you consent to participate in this study?”

2. “Imagine you signed up for the same study. However, now *name of researcher* will make the data publicly available on the internet. This means other researchers will have easier access to it, but also that anyone, such as your fellow students, companies, or the government, can see the data. Of course, the dataset will be anonymized – it may contain your gender, or age, but no personally identifiable information. Would you consent to participate in this study?”

After each vignette, they submited their response to

As I said, respondents were 60 first year psychology students, of the international bachelor in psychology of the University of Groningen, most of them German. It is my experience that this population generally guards its privacy a lot more than their Dutch counterparts – please keep this in mind.

The results? For scenario 1, 13.3% indicated they would *not* partcipate. This percentage indicates the data may be a bit skewed – for most studies I run (EEG work on visual perception and social interaction) I have a non-consent rate of about 5 to max. 10%. For my TMS work this can go op to 33%. However, given the nature of the research I used as an example (I named a researcher they know, and her research involves the role of disgust in sexual arousal – stuff like touching the inside of a toilet bowl after watching a porn clip), 13.3% might not be totally unreasonable.

For scenario 2, the percentage of non-consenters was obviously higher. But not just a little bit – it went up to a whopping 52.4%. More than half of the students present indicated they would not want to participate in this study if the data were to be made publicly available, even though I clearly indicated all data would be anonymized.

The Mentimeter result can be found here. Please note that there are 61 votes for vignette 2; one student was late and voted only for vignette 2. Feel free to remove one ‘no’ vote from the poll – it’s now 51.6% non-consenters.

What does this tell us? Well, there are some obvious caveats. First of all – this was a very ad-hoc experiment in a rather select and possibly biased group of students (ie., students who took the trouble of going to a lecture from 17:00 to 19:00 in a lecture hall 15 mins from the city centre, knowing I would lecture about consiousness, my favourite topic). Second, the experimenter (me) was biased, and even though I explicitly mentioned I would only give my view after the experiment, we all know how experimenter bias affects outcome of experiments. Maybe I did not defend the ‘open’ option furiosly enough. Maybe I made a weird face during vignette 2. Finally, the vignettes I used were about experiments in which potentially sensitive data (sexual arousal) is collected.

Nevertheless, I was surprised by the result. I expected an increase in non-consent, but not to such an extent that more than half would decline. Either I am very good at unconsciously influencing people, or this sample actually has a problem with having their data made publicly accessible. Anyway, it confirmed my hunch that in the debate on open data we should involve the people it is really about: our participants.

I do not wish to use this data as a plea against open data. But I do think researchers should talk to participants. Have a student on your IRB if you use first year participant pools, or otherwise someone from your paid participant pool. Set up a questionnaire to find out what participants find acceptable with regard to data sharing. In the end, if you post a dataset online without restrictions, it’s *their* data and *their* privacy that are at stake.

As a side note, going through some paperwork about consent forms, it actually turned that data storage and sharing in my default consent form is phrased as such:

“My data will be stored anonymously, and will only be used for scientific purposes, including publication in scientific journals.”

This formulation, which is presribed by my IRB, allows for data sharing between researchers, but forbids unrestricted (open) publication. I was actually quite happy to rediscover this – it means I can adhere to the Agenda for Open Research (or better, can not adhere to it with good reason)… publication of data would be a breach of consent in this case. If I were to put my data publicly online, I cannot keep my promise that data would only be used for scientific purposes.

But why not add something to the informed consent?

“The researcher will take care my data is stored at an institutional repository and guarantees she or he will share my data upon request with other researchers.”

Everybody happy.


This article and its reviews are distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and redistribution in any medium, provided that the original author and source are credited.