Bazian presents a poster on the quality of reporting in systematic reviews
August 29th 2014
At Bazian we spend a large amount of time appraising and reporting on the quality of healthcare research. In addition to the various tools available to help appraise the quality of published research, there are a number of reporting standards that aim to encourage better quality reporting. Transparent reporting of methodologies and findings facilitate quality assessment and replication to test research findings.
Based on a hunch, one of our Information Specialists - Elly O’Brien - decided to undertake some research into how well journals that endorse the PRISMA reporting standards for systematic reviews actually adhere to those guidelines. This preliminary research was presented as a poster at The Chartered Institute of Library and Information Professionals Health Libraries Group conference in Oxford earlier this year.
The research involved analysing a random sample of issues within a sample of freely available PRISMA journals. A total of 23 articles were analysed. Encouragingly, compliance was generally good, with an average of 67% of the criteria being met.
Some key findings of the research were:
- All of the systematic reviews clearly stated their objectives.
- 87% described themselves as a systematic review in the article title.
- 70% of the systematic reviews reported assessing the quality of the studies included in the review, but many did not report the tool used and 12% of the studies did not report the results of the quality assessment in the main paper.
- Searches were reported in varying detail, 91% of systematic reviews reported the databases they searched, but only 22% reported their full search strategy.
- 83% of systematic reviews reported the process for selecting studies and only 74% reported the methods for data extraction.
- The source of funding was reported in 39% of systematic reviews.
- Only 13% of systematic reviews reported assessing bias across studies, such as publication bias.
The main conclusions of the research were:
- Improved reporting of studies would improve fellow researchers’ ability to reproduce or build upon research, judge potential sources of bias and assess methodological quality—especially where there may be a discrepancy between the methods used and what was reported.
- Improved reporting would reduce the effort spent in devising, validating and using search filters to identify systematic reviews in bibliographic databases. It would also allow greater auto-indexing in bibliographic databases, which would improve the reliability and consistency of built-in search filters, and make database records available more quickly.
See the PDF version of Elly’s poster here.