Quality appraisal in systematic reviews of public health interventions: an empirical study on the impact of choice of tool on meta-analysis

Voss, Peer H. and Rehfuess, Eva A. (31. July 2012): Quality appraisal in systematic reviews of public health interventions: an empirical study on the impact of choice of tool on meta-analysis. In: Journal of Epidemiology and Community Health, Vol. 67, No. 1: pp. 98-104

Abstract

Introduction Systematic reviews are a cornerstone of evidence-based public health. The method of appraising the quality of different intervention and observational study designs in such reviews remains an important challenge. This article examines the applicability of selected quality appraisal tools (QATs) and the impact of choice of tool on the meta-analysis of a published systematic review. Methods The authors selected a systematic review on the effectiveness of hand washing with soap in preventing diarrhoea, covering a range of epidemiological study designs. 6 QATs were used to assess 13 studies meeting their inclusion criteria; component sections/questions were coded numerically to derive a summary score between -1 (low quality) and +1 (high quality) for each QAT and study. Heterogeneity in study quality was evaluated graphically using traffic light schemes and spider charts. Random effects meta-analysis was undertaken for all studies; sensitivity analyses for each QAT included only those studies with a score of 0 or above. Results The authors found substantial heterogeneity in summary scores for a given study. Their main meta-analysis yielded an OR of 0.60 (95% CI 0.47 to 0.77) with most sensitivity analyses giving similar pooled effect sizes with wider CIs. Discussion The six QATs differ greatly in applicability across study designs, approach to quality appraisal (ie, scale vs checklist, presence/absence of summary score), coverage of domains and quality of component questions and answers. Learning from advantages and disadvantages of each QAT, we recommend research into the development of a reliable QAT with a broad applicability across study designs.

• Item Type: Journal article
• Form of publication: Publisher's Version
• Faculties: Medicine > Institute for Medical Information Processing, Biometry and Epidemiology
• Subjects: 600 Technology > 610 Medicine and health
• ISSN: 0143-005X
• Alliance/National Licence: This publication is with permission of the rights owner freely accessible due to an Alliance licence and a national licence (funded by the DFG, German Research Foundation) respectively.
• Annotation: First publ. online. Print publ. January 2013
• Language: English