Great citations: how to avoid referencing questionable evidence

Have you ever cited peer-reviewed scientific evidence that eventually turned out to be contested or incorrect? Unfortunately, there are many ways in which such questionable research can creep into your writing without you even realising it. The scope of questionable research practices is increasingly exposed and documented and we should respond by becoming more critical and careful when it comes to citing academic work.

Evidently, little can be done about publications that seem to present valid evidence but are in fact underpinned by poor and irreproducible research practices, data manipulation or even fraud. Sometimes, however, maintaining good citation hygiene is enough to steer clear of questionable research.

Researchers often continue to cite retracted publications for years after the retraction, thus perpetuating poor evidence in scientific literature, studies show. This rarely happens on purpose but rather because of poor retraction signalling and far-from-ideal citation practices.

• THE Campus spotlight collection: tips for success in academic publishing
• Are you ready to publish? Top tips on how to prepare your manuscript
• THE podcast: pointers on writing and publishing for academics

Citing retracted research should be easily avoidable, at least in theory, but there are other types of contested evidence that are not as obvious. Here are some tips on how to assess the quality of cited evidence to avoid falling into any traps.

1) Pay attention to signs of retraction and correction

To avoid further uptake of retracted and corrected research, publishers are supposed to make such status updates clearly visible. Therefore, whenever you find an exciting piece of research you want to cite, make sure that you check the status of the article on the publisher’s website. Other scholarly repositories may not be as reliable, since data exchange between publishers and other searchable databases is not always ideal. Even publishers do not always perfectly comply with the retraction and correction guidelines, such as those published by COPE, and status updates may not be immediately obvious even on their websites.

2) Use bibliographic managers that signal problematic research

Using bibliographic managers, such as Mendeley, Zotero and Endnote, is good practice for maintaining overall citation hygiene. However, it can also lead to situations when retracted or corrected publications persist in personal libraries without status updates and are subsequently used by unsuspecting researchers. You might consider making sure that your bibliographic management software triggers retraction alerts whenever an article in your library is retracted or corrected. Some bibliographic managers support integration with retraction databases, such as Retraction Watch, which contains data on more than 30,000 retracted articles.  

3) Check your reference lists with new software solutions

Retracted research is only the tip of the iceberg when it comes to questionable scientific literature. Plenty of poor and irreproducible research will never get retracted. How to do your best to avoid citing such research, apart from scrutinising every publication you want to cite? Fortunately, there are emerging technological solutions, such as Scite.ai, that enable you to check entire reference lists to see whether any of your references have been already cited critically by other researchers. Similarly, the platform can identify research that has been cited positively. This can help you make better decisions about what to cite and what to avoid.

4) When something looks suspicious, check with anonymous peer review websites

Anonymous peer review websites, such as PubPeer, have gained a lot of attention after revealing several cases of academic misconduct and research fraud. The resource contains post-publication comments by anonymous volunteer reviewers that might be useful for you to make up your mind. If in doubt about a particular piece of research, checking with anonymous peer review websites, such as PubPeer, might be a good idea. 

5) If for some reason you do need to cite retracted or corrected articles, acknowledge their status

You might find yourself in a situation when you need to cite retracted or corrected research. Either you want to make a critical comment or refer to a historical precedent involving scientific error or academic misconduct. Even in such cases, it is important to maintain good citation practice and clearly flag the publication's status in both the text of the paper and references.

Navigating scientific evidence is not easy, especially when you delve into a new research area or are just starting on your research path. Taking precautionary steps, such as those described in this article, will help you ensure you are using all available tools to stay clear of citing potentially questionable research.

Dmitry Malkov is a research analyst at Elsevier.

If you found this interesting and want advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the THE Campus newsletter.