There is still a lot of medical research based on faulty cell lines and antibodies, although matters are slowly improving. This is written in a new publication by several science sociologists, led by Willem Halffman. Surprisingly, it matters very little whether or not a journal maintains strict quality control measures.

Associate Professor Willem Halffman and his PhD candidate at the Institute for Science in Society, Serge Horbach, made a shocking discovery in 2017: over 30,000 scientific studies used the wrong type of cells. For instance, skin cells turned out to actually be colon cells, or human cells were actually taken from mice. Now it appears that things have improved four years later, but it is doubtful whether that is due to crackdowns by scientific journals.

Scientists prefer to do their experiments on the same cells used by other researchers; this allows them to better compare results. That is why there are standardised cell lines for all manner of research, for example research into cancer medications. The cells’ qualities -such as whether they’re skin- or kidney-cells, or if they contain some kind of DNA adaptation- are all registered in databases.

Mutated

However, it is known that a lot of cell lines have acquired problems, sometimes intentionally. Over time these have mutated or become polluted with different types of cells, due to laboratory errors, for example. Antibodies and laboratory animals that are used a lot in medical or biological research have similar issues. Their names and numbers are reported on blacklists -that are always getting longer. These lists are meant to prevent scientists from using the list items for research.

More and more journals maintain guidelines and checklists for the same reason, such as the journals like Nature and Biomed Central. Halffman: ‘Researchers who want to publish in these journals have to specify which cells or antibodies they’ve used exactly. Sometimes they also have to demonstrate how they explicitly checked this.’ This is done to prevent faulty conclusions from being drawn from experiments.

Two million publications

Halffman and his colleagues wanted to investigate whether these extra checks done by journals actually helped. To do this, they used an automated method to check nearly two million publications made over the last ten years for the use of faulty cell lines, antibodies, or lab animals.

The sociologists discovered that the use of these items has decreased over the last few years, owing to the peer review. They published their discovery in the renowned trade magazine International Journal of Cancer; they were headed by Halffman’s former Master’s student Wytske Hepkema, who is now a PhD candidate at the Technische Universität Berlin.

‘A researcher who doesn’t want to be held accountable can easily go to a different journal’

This is clearly a positive development. Halffman suspects that it is the repercussion of the careful cultural change that is taking place in the scientific community right now. ‘There is increasing awareness for the reliability of studies, partly owing to the replication crisis in the fields of medicine and psychology, five to ten years ago.’ At the time, a lot of studies led to different results on repeat experiments. A lot of journals are also leading the discussion about the importance of experiment verifiability in their own columns. ‘The International Journal of Cancer is one of the journals that has taken the lead on this front. That is why we wanted them to publish our research.’

Remarkably, the positive trend that he and his colleagues noticed had little to do with whether or not journals maintained strict guidelines for issues such as cell lines; on average, scientific journals that did not check for these issues performed the same as those that did have guidelines. Halffman suspects that this is because of a lack of power, especially in smaller journals. ‘If a researcher doesn’t want to be held accountable through a checklist, they can easily go to a different journal that won’t maintain such strict guidelines.’ Researchers often have dozens of journals to choose from when deciding where to publish.

Window-dressing

According to Halffman, sometimes it is also the case that journals have guidelines only as a sort of superficial gesture. ‘If authors can easily ignore the guidelines, then they’re just window-dressing. The in-depth checking of research material can’t be left to reviewers; they often lack the required expertise.’ But it can be done, according to Halffman, which is demonstrated by major journals such as Nature, who are much stricter. ‘They require you to submit every little detail of your research material. Journals like this can serve as a good example, because everybody is keen to publish in them.’

But the question remains why scientists are knowingly working with problematic materials in the first place. Can’t they simply check these things in the databases? ‘Some people will claim that it doesn’t matter for their research that some of the cells were different than initially thought. It is also a matter of money: having cell lines genetically checked takes time and effort.’ Finally, according to Halffman there are also some scientists who are entirely unaware of the existence of the blacklists – as strange as that may sound.