|Year : 2017 | Volume
| Issue : 1 | Page : 1-2
Meta-analysis: Minimizing errors
Jayant N Palaskar
Editor-in-Chief, Journal of Dental and Allied Sciences, Department of Prosthodontics, Sinhgad Dental College and Hospital, Pune, Maharashtra, India
|Date of Web Publication||2-May-2017|
Jayant N Palaskar
Department of Prosthodontics, Sinhgad Dental College and Hospital, S. No. 44/1, Off Sinhgad Road, Vadgaon Budruk, Pune - 411 041, Maharashtra
Source of Support: None, Conflict of Interest: None
|How to cite this article:|
Palaskar JN. Meta-analysis: Minimizing errors. J Dent Allied Sci 2017;6:1-2
A meta-analysis is a powerful tool to cumulate and summarize the knowledge in a research field through statistical instruments and to identify the overall measure of a treatment's effect by combining several individual results. Although meta-analysis and its outcomes are gold standards in the field of review, researchers must be aware of what precautions should be taken before taking up any meta-analysis study. Any error committed inadvertently by the researchers can lead to misleading conclusions; an investigator should be eternally vigilant against such errors which may undermine the purpose of meta-analysis. For this reason, it is important to write a prospective analysis protocol, which specifies the objectives and methods of the meta-analysis. Having a protocol restricts the risk of biased and selective reporting.
| Inclusion and Exclusion Criteria of Studies to Be Included in Meta-Analysis|| |
Researchers should be absolutely clear about their research question specified in the protocol. The research question framed has to be very narrow to avoid heterogeneity of studies to be included. On the other hand, a very broad question will permit researchers to include many studies which may not be directly related to the research question framed for the meta-analysis and indeed, will lead to increase in heterogeneity of the study that may result in nonconclusive, nongeneralizable outcomes. I would like to reinstate that inclusion of homogeneous (similar) studies will help in drawing generalized conclusions.
For pooling articles for the analysis, at least three to four databases must be searched. Most valid databases such as PubMed, MEDLINE, EMBASE, and The Cochrane Central Register of Controlled Trials are recommended. Other databases such as Google Scholar, Scopus, Ebsco, and ResearchGate may also be utilized.
Monitoring quality of included studies
It is imperative to maintain a strict check on the quality of studies included in the analysis. Cochrane guidelines give structured formats to establish the internal and external validity of the studies to be included. Internal validity helps in minimizing bias and external validity will help in extrapolating the outcomes of the analysis to a larger population. The investigator has to exhibit constant vigilance against bias such as the articles' author's standing in the field, reputation of the journal, in which the said article is published, and country where the study was conducted. Validity results may also be influenced by investigators' opinion. There should be no bias at any stage of meta-analysis as it will adversely affect the outcome of the analysis. Different types of bias have been dealt by me in my last editorial.
Fixed- versus random-effects estimate
There are two common models of a meta-analysis to estimate treatment effects of any study included in the analysis: Fixed- and random-effects estimates. If the random-effects estimate is more beneficial, researchers should consider whether it is reasonable to conclude that the treatment was more effective in smaller studies. This is because the weightage given to each included study through the random-effect model is less influenced by the sample size than that given by means of the fixed-effects model. One must note that if there is no evidence of heterogeneity between studies, the fixed- and random-effects estimates will be identical, so there will be an actual difficulty in identifying the small-study effect. Researchers who are concerned about the influence of small-study effects on the results of a meta-analysis in which there is evidence of between-study heterogeneity should compare and address the fixed- and random-effects estimates of the treatment effect.
| Managing Heterogeneity of Selected Studies|| |
The degree of heterogeneity of the selected studies is another important limitation, and the random-effects model should be used during the data analysis phase to incorporate in the treatment effect the identifiable or nonvariability between studies. It is fundamental to observe that exploring heterogeneity in a meta-analysis should start at the stage of protocol writing, by identifying a priori which factors are likely to influence the treatment effect. Visual inspection of the meta-analysis plots may show whether the results of a subgroup of studies have the same overall direction of the treatment effect. One should pay attention to meta-analysis, in which results have a discordant treatment effect for groups of studies and no explanation of variance has been done. Sources of variation should be identified, and their impact on effect size should be quantified using statistical tests and methods such as analysis of variance or weighted meta-regression.
Testing effects suggested by data and not planned a priori considerably increases the risk of false-positive results. To minimize this error, it is important to identify the effects to test before data collection and analysis; otherwise, one may adjust the P value according to the number of analysis performed. In general, post hoc analysis should be deemed exploratory and not conclusive.
| References|| |
Biondi-Zoccai G, Landoni G, Modena MG. A journey into clinical evidence: From case reports to mixed treatment comparisons. HSR Proc Intensive Care Cardiovasc Anesth 2011;3:93-6.
Greco T, Zangrillo A, Biondi-Zoccai G, Landoni G. Meta-analysis: Pitfalls and hints. Heart Lung Vessel 2013;5:219-25.
Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions
Version 5.1.0. The Cochrane Collaboration, 2011. Available from: www.handbook.cochrane.org
. [Updated 2011 Mar; Last accessed date on 2017 Apr 14].
Ng TT, McGory ML, Ko CY, Maggard MA. Meta-analysis in surgery: Methods and limitations. Arch Surg 2006;141:1125-30.
Harrison F. Getting started with meta-analysis. Methods Ecol Evol 2011;2:1-10.
Wang R, Lagakos SW, Ware JH, Hunter DJ, Drazen JM. Statistics in medicine – Reporting of subgroup analyses in clinical trials. N Engl J Med 2007;357:2189-94.
Walker E, Hernandez AV, Kattan MW. Meta-analysis: Its strengths and limitations. Cleve Clin J Med 2008;75:431-9.