Understanding and Appraising Systematic Reviews and Meta-Analysis

Systematic reviews and meta-analysis seek to answer a pre-framed research question to lead to a valid answer through a systematic, explicit and reproducible method of locating; identifying, including and appraising appropriate trials. The results are synthesized considering the methodological rigor of included trials. While the meta-analysis quantitatively pools the results from individual included studies, the systematic review summarizes the findings as qualitative conclusions. These reviews are crux of evidence based dentistry for various stake-holders, i.e., clinicians, researchers and policy-makers. Although the meticulous methodology of systematic review and meta-analysis minimizes the elements of bias, yet the validity and reliability of their findings should be explored prior to translating their conclusions to practice. The goal of this paper is to familiarize readers with rationale, conduct and appraisal of systematic review and meta-analysis. Further, guidance is provided on tracing potential elements of bias in the review to enable readers to judge the quality of evidence generated from the review.

AMSTAR, Evidence based dentistry, Meta-analysis, Systematic review.

1. http://www.ncbi.nlm.nih.gov/pubmed (Accessed on 1st July, 2017)

2. Forrest JL, Miller SA. Translating Evidence-Based Decision Making into Practice: EBDM Concepts and Finding the Evidence. J Evid Base Dent Pract 9:59-72, 2009.

3. SUNY Downstate Medical Center. Guide to Research Methods, the Evidence-Based Pyramid. Available at: http://library.downstate.edu/ EBM2/2100.htm. (Accessed on 5th May, 2015)

4. Grant JM, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Infor Lib J 26:91-108, 2009.

5. Glossary of terms in the Cochrane Collaboration. Version 4.2.5. Updated 2005. Available at: http://community.cochrane.org/glossary (Accessed on 5th May, 2015)

6. http://www.ncbi.nlm.nih.gov/pubmed/clinical (Accessed on 24th May, 2015)

7. http://www.cochranelibrary.com/ (Accessed on 24th May, 2015)

8. http://www.cochranelibrary.com/topic/Dentistry%20%26%20oral%20 health/ (Accessed on 24th May, 2015)

9. The journal of evidence based dental practice. http://www.jebdp.com/ (Accessed on 7th June, 2015)

10. http://www.ncbi.nlm.nih.gov/pubmedhealth/about/DARE/ (Accessed on 24th May, 2015)

11. Evidence based Dentistry. http://www.nature.com/ebd/index.html (Accessed on 24th May, 2015)

12. Sacks HS, Berrier J, Reitman D, Ancona-Berk VA, Chalmers TC. Meta-analyses of randomized controlled trials. N Engl J Med 316: 450, 1987.

13. Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol 44: 1271–8, 1991.

14. Oxman AD, Guyatt GH, Singer J, Goldsmith CH, Hutchison BG, Milner RA, et al. Agreement among reviewers of review articles. J Clin Epidemiol 44: 91–8, 1991.

15. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 7: 10, 2007.

16. Shea BJ, Bouter LM, Peterson J, et al. External validation of a measurement tool to assess systematic reviews (AMSTAR). PLoS One 2: e1350, 2007.

17. Shea BJ, Hamel C, Wells GA, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol 62: 1013–20, 2009.

18. CASP, Critical Appraisal Skills Programme. Appraisal Tools. Available at: http://www.phru.nhs.uk/Pages/PHD/resources.htm. (Accessed on 4th June, 2015)

19. National Institute for Health and Care Excellence (NICE). The guidelines manual. Appendix B: Methodology checklist: Systematic reviews and meta-analysis. Available from http://publications.nice.org.uk/the-guidelines-manual-appendices-bi-pmg6b/appendix-b-methodology-checklist-systematic-reviews-and-meta-analyses (Accessed on 24th May, 2015)

20. Rapid appraisal protocol internet database. Rapid user guide 1.2. The Joanna Briggs Institute. Australia: Adelaide, 2006.

21. Clarkson J, Harrison JE, Ismail AI, Needleman I, Worthington H. Evidence based dentistry for effective practice. London: Taylor and Francis Group 228p, 2003

22. Needleman I. Introduction to evidence based medicine. In Clarkson J, Harrison JE, Ismail AI, Needleman I, Worthington H. Evidence based dentistry for effective practice. London: Taylor and Francis Group 2003. Chapter 1: 1-17, 2003

23. Maia LC, Antonio AG. Systematic reviews in dental research. A Guideline. J Clin Pediatr Dent 37: 117-24, 2012.

24. Higgins JPT, Green S. Cochrane Handbook for Systematic review of Interventions Versioon 5.1.0 [updated March 2011] The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org. (Accessed on 4th May, 2015)

25. Mittal N, Goyal A, Jain K, Gauba K, Jain K. Pediatric dental sedation research: where do we stand today? J Clin Pediatr Dent 2015; 39: 286-93.

26. Higgins JPT, Lasserson T, Chandler J, Tovey D, Churchill R. Standards for the conduct of new Cochrane Intervention Reviews 2012 V2.3. Methodological Expectations of Cochrane Intervention Reviews: Standards for the conduct and reporting of new Cochrane Intervention Reviews 2012. Version 2. 2013: Cochrane, London.

27. Egger M, Smith DG. Meta-analysis: Bias in location and selection of studies. Br Med J 316, 61–6, 1998.

28. Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G. Language bias in randomised controlled trials published in English and German. Lancet 350: 326 –9, 1997

29. Schlosser RW, Wendt O, Angermeier K, Shetty M. Searching for and finding evidence in augmentative and alternative communication: Navigating a scattered literature. Augment Altern Comm 21, 233 –55, 2005.

30. Hyung Bok Yoo H, Quebuz TT. Locating and selecting appraisal studies for reviews. Chest 125: 798, 2004

31. Schlosser RW. Appraising the quality of systematic reviews. Focus: Technical briefs 2007, 17: 1-8

32. WHO; International clinical trial registry search platform (ICTRP). www.who.int/ictrp/ (Accessed on 24th May, 2015)

33. Light R, Pillemer D. Summing up: The science of reviewing research. Cambridge, MA: Harvard University Press, 1998.

34. Rosenthal R. The “file drawer problem” and tolerance for null results. Psych Bull 86: 638–41, 1978.

35. Rothstein H, Sutton AJ, Borenstein M. Publication bias in meta-analysis: Prevention, assessment and adjustments. Chichester, UK: John Wiley & Sons, 2005.

36. Bhandari M, Devereaux PJ, Guyatt GH, Cook DJ, Swiontkowski MF, Sprague S et al. An observational study of orthopaedic abstracts and subsequent full-text publications. J Bone Joint Surg Am 84: 615-21, 2002.

37. Krzyzanowska MK, Pintilie M and Tannock IF. Factors associated with failure to publish large randomized trials presented at an oncology meeting. JAMA 290: 495-501, 2003.

38. Smith WA, Cancel QV, Tseng TY, Sultan S, Vieweg J, Dahm P. Factors associated with the full publication of studies presented in abstract form at the annual meeting of the American Urological Association. J Urol 177: 1084-8, 2007.

39. Cook DJ, Guyatt GH, Ryan G, Clifton J, Buckingham L, Willan A et al. Should unpublished data be included in meta-analyses? Current convictions and controversies. JAMA 269: 2749-53, 1993.

40. Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA 263: 1385-9, 1990.

41. Sterne JAC, Egger M, Davey Smith G. Investigating and dealing with publication bias and other biases. Systematic Reviews in Health Care: Meta-analysis in Context, Egger M, Davey Smith G, Altman DG (eds). BMJ Publication Group: London, 2001; 189-208.

42. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. Br Med J 315:629-34, 1997.

43. Anzures-Cabrera J, Higgins JPT. Graphical displays for meta-analysis: an overview with suggestions for practice. Res Syn Meth 2010; 1; 66-80

44. Katrak P, Bialocerkowski AE, Massy-Westropp N, Kumar VSS, Grimmer KA. A systematic review of the content of critical appraisal tools. BMC Med Res Method 4:22, 2004

45. Zeng X, Zhang Y, Kwong JSW, Zhang C, Li S, Sun F, Niu YM, Du L. The methodological quality assessment tools for pre-clinical and clinical studies, systematic review and meta-analysis, and clinical practice guideline: a systematic review. J Evid Based Med 8: 2-10, 2015.

46. Crowe M, Sheppard L. A review of critical appraisal tools show they lack rigor: Alternative tool structure is proposed. J Clin Epidemiol 64:79-89, 2011.

47. Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. Br Med J 343: d5928, 2011.

48. Mittal N, Bhatia HP, Haider K. Clinical outcomes of methods of intracanal reinforcement in grossly mutilated primary anterior teeth. Int J Clin Pediatr Dent 8: 48-54, 2015

49. Lau J, Ioannidis JP, Schmid CH. Summing up evidence: one answer is not always enough. Lancet 351: 123–7, 1998.

50. Higgins JP, Thompson SG, Deeks JJ, Altmann DG. Measuring inconsistency in meta-analyses. Br Med J 327: 557–60, 2003.

51. Montori VM, Swiontkowski MF, Cook DJ. Methodologic issues in systematic reviews and meta-analyses. Clin Orthop Relat Res 413: 43-54, 2003

52. Hatala R, Keitz S, Wyer P, Guyatt G. Tips for learners of evidence-based medicine: Assessing heterogeneity of primary studies in systematic reviews and whether to combine their results. CMAJ 172: 661-5, 2005.

53. Schoenfeld PS, Loftus EV Jr: Evidence-based medicine (EBM) in practice: understanding tests of heterogeneity in metaanalysis. Am J Gastroenterol 100: 1221-3, 2005.

54. Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S et al. Grading quality of evidence and strength of recommendations.Br Med J 328:1490, 2004.

55. Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A et al. Going from evidence to recommendations. Br Med J 336:1049–51, 2008.

56. Guyatt GH, Oxman AD, Kunz R, Jaeschke R, Helfand M, Liberati A et al. Incorporating considerations of resources use into grading recommendations. Br Med J 336:1170–3, 2008.

57. Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schunemann HJ. What is ‘‘quality of evidence’’ and why is it important to clinicians? Br Med J 336:995-8, 2008.

58. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. Br Med J 336:924–6, 2008.