Skip to content >>

an Economist Intelligence Unit business healthcare

Bazian presents a poster on the quality of reporting in systematic reviews

August 29th 2014

At Bazian we spend a large amount of time appraising and reporting on the quality of healthcare research. In addition to the various tools available to help appraise the quality of published research, there are a number of reporting standards that aim to encourage better quality reporting. Transparent reporting of methodologies and findings facilitate quality assessment and replication to test research findings.

Based on a hunch, one of our Information Specialists - Elly O’Brien - decided to undertake some research into how well journals that endorse the PRISMA reporting standards for systematic reviews actually adhere to those guidelines. This preliminary research was presented as a poster at The Chartered Institute of Library and Information Professionals Health Libraries Group conference in Oxford earlier this year.

The research involved analysing a random sample of issues within a sample of freely available PRISMA journals. A total of 23 articles were analysed. Encouragingly, compliance was generally good, with an average of 67% of the criteria being met.

Some key findings of the research were:

  • All of the systematic reviews clearly stated their objectives.
  • 87% described themselves as a systematic review in the article title.
  • 70% of the systematic reviews reported assessing the quality of the studies included in the review, but many did not report the tool used and 12% of the studies did not report the results of the quality assessment in the main paper.
  • Searches were reported in varying detail, 91% of systematic reviews reported the databases they searched, but only 22% reported their full search strategy.
  • 83% of systematic reviews reported the process for selecting studies and only 74% reported the methods for data extraction.
  • The source of funding was reported in 39% of systematic reviews.
  • Only 13% of systematic reviews reported assessing bias across studies, such as publication bias.

The main conclusions of the research were:

  • Improved reporting of studies would improve fellow researchers’ ability to reproduce or build upon research, judge potential sources of bias and assess methodological quality—especially where there may be a discrepancy between the methods used and what was reported.
  • Improved reporting would reduce the effort spent in devising, validating and using search filters to identify systematic reviews in bibliographic databases. It would also allow greater auto-indexing in bibliographic databases, which would improve the reliability and consistency of built-in search filters, and make database records available more quickly.

See the PDF version of Elly’s poster here.