Bazian presents a poster on the quality of reporting in systematic reviews

August 29th 2014

At Bazian we spend a large amount of time appraising and reporting on the quality of healthcare research. In addition to the various tools available to help appraise the quality of published research, there are a number of reporting standards that aim to encourage better quality reporting. Transparent reporting of methodologies and findings facilitate quality assessment and replication to test research findings.

Based on a hunch, one of our Information Specialists - Elly O’Brien - decided to undertake some research into how well journals that endorse the PRISMA reporting standards for systematic reviews actually adhere to those guidelines. This preliminary research was presented as a poster at The Chartered Institute of Library and Information Professionals Health Libraries Group conference in Oxford earlier this year.

The research involved analysing a random sample of issues within a sample of freely available PRISMA journals. A total of 23 articles were analysed. Encouragingly, compliance was generally good, with an average of 67% of the criteria being met.

Some key findings of the research were:

All of the systematic reviews clearly stated their objectives.
87% described themselves as a systematic review in the article title.
70% of the systematic reviews reported assessing the quality of the studies included in the review, but many did not report the tool used and 12% of the studies did not report the results of the quality assessment in the main paper.
Searches were reported in varying detail, 91% of systematic reviews reported the databases they searched, but only 22% reported their full search strategy.
83% of systematic reviews reported the process for selecting studies and only 74% reported the methods for data extraction.
The source of funding was reported in 39% of systematic reviews.
Only 13% of systematic reviews reported assessing bias across studies, such as publication bias.

The main conclusions of the research were:

Improved reporting of studies would improve fellow researchers’ ability to reproduce or build upon research, judge potential sources of bias and assess methodological quality—especially where there may be a discrepancy between the methods used and what was reported.
Improved reporting would reduce the effort spent in devising, validating and using search filters to identify systematic reviews in bibliographic databases. It would also allow greater auto-indexing in bibliographic databases, which would improve the reliability and consistency of built-in search filters, and make database records available more quickly.

See the PDF version of Elly’s poster here.