Watching out for retractions

Ivan Oransky, Distinguished Writer In Residence at New York University, co-founded the popular blog Retraction Watch, which reports on and investigates the retractions of scientific papers. We caught up with him to talk science journalism, scientific misconduct and more.

Can you give me a sense of any major developments or news in the area of research misconduct?

We [at Retraction Watch] have recently created a database of retractions, which will officially launch in a few months but is live and can be used now. That’s been one of our big projects and we’re very excited about it. We have 17,500 retractions there now which is three times more than you will find anywhere else. You can search by author, country, journal, DOI, and subject.

There are also lots of things happening in the publishing space – pre-prints are nothing new but they are definitely hitting their stride, certainly in biomedicine, which they hadn’t been [doing] before. Also [developments include] registered reports and results free peer review, which are similar [to each other] in many ways. It’s an attempt to do a couple of things, but mostly to cut down on positive publication bias, so instead of the paper coming in and the editor only sending it to review if there are amazing positive reviews, you present protocol and method and get that peer reviewed and the journal in some cases actually agrees to publish no matter what your results are. You get rid of bias against negative or null results, so that’s pretty interesting. It’s not really going to cut down on fraud, but it’s part of a larger movement.

The field is maturing. I don’t know if it’s quite mature yet, but it is maturing. I can tell this because I get Google alerts now much more frequently of people citing us. People are using Retraction Watch now to screen authors. Publishers are doing it. One publisher wrote a script so that when anyone submits a manuscript in the background it runs a search through Retraction Watch, which is cool for us.

Simply covering retractions is no longer novel, so we are now focused on bigger projects, such as writing for outlets [such as Science, NPR, JAMA and Undark]. We also try to get misconduct investigation reports whenever we can – it’s a big push for us.

In general, why do you think the process of retracting a paper take so long?

Using the new Retraction Database, you can easily run an analysis of how long retractions take, but all you’re learning is how long it takes from publication to retraction. For me, the interesting interval is how long it takes from legitimate allegation to retraction. We understand why it might take a long time to investigate and that lawyers get involved and that can make things drag on. Although I don’t think people are working as quickly as they should, this is at least an explanation. But the part that still puzzles me – and I’ve been looking at this for eight years – is why does it take such a long time [it can take three years] for a journal to retract when they have a letter from a university [explaining why a paper should be retracted]? If you are afraid of lawyers – and a lot of journals are – the letter covers you. If this had been happening 13 years ago, I would say journals didn’t have a system, but now there are systems for this. I’ll be cynical for a second: when you look at the average time from publication to retraction, it’s three years – it’s bimodal because a lot are happening quickly and some really slowly, the record being 80 years! Now, how long do citations count for impact factor [of a given journal]? Two years, so it [the delay] is like getting time served instead of having to serve more time in jail. I’m being provocative, but I may be on to something. The point is the longer you delay the more citations you get, and papers are still cited after they are retracted which is also not a good thing. It’s all back to impact factor.

Some may describe you as a medical doctor turned journalist. Can you tell me about this transition?

It wasn’t a sudden move. I was doing medicine and journalism at the same time through medical school; during my internship, I used to carry two pagers: one for the hospital – in case Mrs Smith needed a prescription or Mr Smith had a heart attack or whatever! – and the other if I wanted to interview someone. I would run home every night and write something, write a couple of columns, and I thought I should go for this. I had always done both things. In high school, I was editor of my high school newspaper. I’ve been doing this since I was a teenager, so by the time I made a career break I was 27 years old and doing this for a decade, and definitely making mistakes, so not at the pro level yet, but writing for big newspapers even while in medical school.

How and when did you become interested in research misconduct?

When I look back I ended up digging up on the subject way back in college. One of my beats [at The Harvard Crimson] was covering the faculty – hiring and firing, getting tenure and lots of other issues – and that was the first time I thought about it. Then, when I was in medical school, I was one of the editors of the medical student section of JAMA, which no longer exists, and I learned so much about peer review there; they even let me sit in on manuscript meetings. And then when I was working at The Scientist we would profile scientists, and look, for example, at how often papers were cited.

Can you tell me more about the role of journalists in helping to combat research misconduct?

In 2015 the AMA journal of ethics asked me to write a piece on why science journalism is so bad: the argument I made is that at every stage the incentives are misaligned. It’s human nature to do this but the incentive is we reward behaviour we actually don’t want to – it starts when you apply for a grant and you have to have done half the work already – it has to be something like cure cancer or explain relativity once and for all. It’s just not a realistic expectation and it’s at every stage. You get the grant, then you should publish and the journal only wants the amazing results and then the university puts out a press release and the journalists who get the press release are under the same pressure. If we are going to make a difference here, it’s not to say we should abandon everything downstream – I don’t think we can. The equivalent would be: do we really care about speeding on the highway? Yeah we should, but is there a root cause of that? There are root causes you have to deal with, but that doesn’t mean you have to get rid of every speed camera or speed trap. I think it has to be a mix but the incentives are the thing you have to attack.

Given these challenges facing science journalism, what’s your advice to early career journalists?

Focus on the quality of the work – at least you’ll have that and you’ll do better. People ask: should you get a masters degree? I am biased because I teach at a program [at NYU]; I think you don’t have to do a program like that but you do have to get your resume to look like someone whom an internship coordinator would want to hire. If you have no journalism or writing experience that’s a pretty high activation energy needed, so [in that case] a journalism program, as expensive as it might be, might be the best way to go, especially in a place like London or New York where a lot of the outlets are. I would also say do not be afraid of specialising. My students, in one of their early classes, are assigned a beat and a lot of them bristle at that because they think it will shut off opportunities but having one specialty doesn’t actually prevent you from having another, and nowadays you need to be the person a commissioning editor thinks of when they need a story. Again, you don’t want to be too narrow. If you say I will only write about purple moths, well, that’s great and I’m sure you will be the expert on purple moths, but there’s not a hell of a market for that. In a Venn diagram, narrow enough that you can really dive in and broad enough that there is still a market for it.

What’s your favourite aspect of being a journalist?

I still really get excited about revealing stories no one knew about before – it could be fascinating science but that’s a small percentage at this stage – I like holding institutions accountable. It’s a critical function of journalism, and others can do it too, such as NGOs and government. I think it’s tremendously important. I’m still addicted to breaking news. Every journalist is to some extent, so when I can manage to break news while holding some institution accountable and trying to effect change, it sounds trite and obvious, but that’s what gets me up in the morning and a lot of the other things I do support that function. That’s like the job you have to have to support whatever bad habit you have.

Anthea Lacchia is studying for an MSc in Science Communication at Imperial College London

Banner Image: Document pile, Wikimedia Commons.