Meta-analysis: Difference between revisions

From Citizendium
Jump to navigation Jump to search
imported>Robert Badgett
imported>Robert Badgett
(Started problems.)
Line 89: Line 89:
===Outcome reporting bias===
===Outcome reporting bias===
Meta-analyses in which a smaller proportion of included trials provide raw data for inclusion in the meta-analysis are more likely to be positive.<ref name="pmid17284696">{{cite journal |author=Furukawa TA, Watanabe N, Omori IM, Montori VM, Guyatt GH |title=Association between unreported outcomes and effect size estimates in Cochrane meta-analyses |journal=JAMA |volume=297 |issue=5 |pages=468–70 |year=2007 |month=February |pmid=17284696 |doi=10.1001/jama.297.5.468-b |url=http://jama.ama-assn.org/cgi/pmidlookup?view=long&pmid=17284696 |issn=}}</ref> This may be due a bias against reporting negative results.<ref name="pmid15161896">{{cite journal |author=Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG |title=Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles |journal=JAMA |volume=291 |issue=20 |pages=2457–65 |year=2004 |month=May |pmid=15161896 |doi=10.1001/jama.291.20.2457 |url=http://jama.ama-assn.org/cgi/pmidlookup?view=long&pmid=15161896 |issn=}}</ref>
Meta-analyses in which a smaller proportion of included trials provide raw data for inclusion in the meta-analysis are more likely to be positive.<ref name="pmid17284696">{{cite journal |author=Furukawa TA, Watanabe N, Omori IM, Montori VM, Guyatt GH |title=Association between unreported outcomes and effect size estimates in Cochrane meta-analyses |journal=JAMA |volume=297 |issue=5 |pages=468–70 |year=2007 |month=February |pmid=17284696 |doi=10.1001/jama.297.5.468-b |url=http://jama.ama-assn.org/cgi/pmidlookup?view=long&pmid=17284696 |issn=}}</ref> This may be due a bias against reporting negative results.<ref name="pmid15161896">{{cite journal |author=Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG |title=Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles |journal=JAMA |volume=291 |issue=20 |pages=2457–65 |year=2004 |month=May |pmid=15161896 |doi=10.1001/jama.291.20.2457 |url=http://jama.ama-assn.org/cgi/pmidlookup?view=long&pmid=15161896 |issn=}}</ref>
==Problems with meta-analyses==
===Obsolescence===
The conclusions of meta-analyses may be mitigated by research published after the search date of the meta-analysis. This may occur by the time the meta-analysis has been published.<ref name="pmid17638714">{{cite journal |author=Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D |title=How quickly do systematic reviews go out of date? A survival analysis |journal=Ann. Intern. Med. |volume=147 |issue=4 |pages=224–33 |year=2007 |month=August |pmid=17638714 |doi= |url= |issn=}}</ref> Strategies have been developed for updating meta-analyses.<ref name="pmid18586179">{{cite journal |author=Sampson M, Shojania KG, McGowan J, ''et al'' |title=Surveillance search techniques identified the need to update systematic reviews |journal=J Clin Epidemiol |volume=61 |issue=8 |pages=755–62 |year=2008 |month=August |pmid=18586179 |doi=10.1016/j.jclinepi.2007.10.003 |url=http://linkinghub.elsevier.com/retrieve/pii/S0895-4356(07)00365-4 |issn=}}</ref>


==References==
==References==

Revision as of 14:05, 20 April 2009

This article is developing and not approved.
 Main Article
 Discussion
Related Articles  [?]
Bibliography  [?]
External Links  [?]
Citable Version  [?]
 
https://en.citizendium.org/wiki/index.php?title=Meta-analysis&printable=yes
This editable Main Article is under development and subject to a disclaimer.

Template:TOC-right Meta-analysis is defined as "a quantitative method of combining the results of independent studies (usually drawn from the published literature) and synthesizing summaries and conclusions which may be used to evaluate therapeutic effectiveness, plan new studies, etc., with application chiefly in the areas of research and medicine."[1]

A meta-analyses is a subset of systematic reviews in which the results of the studies are numerically pooled.

Validity of meta-analysis

Studies on the validity of meta-analyses conflict.[2][3][4] Some of the conflict may be due to the methods used to compare the meta-analyses.[5]

Methods of meta-analysis

Selecting studies for inclusion

Although meta-analyses in general are very inclusive, arguments exist for only including the best trials.[6]

Assessing the quality of trials

For more information, see: Randomized controlled trial.


Cochrane bias scale

The Cochrane Collaboration uses a six item tool.[7]

Jadad score

The Jadad score may be used to assess quality and contains three items:[8]

  1. Was the study described as randomized (this includes the use of words such as randomly, random, and randomization)?
  2. Was the study described as double blind?
  3. Was there a description of withdrawals and dropouts?

Each question is scored one point for a yes answer. In addition, for questions and 2, a point is added if the method was appropriate and a point is deducted if the method is not appropriate (e.g. not effectively randomized or not effectively double-blinded).

Studies with groups having zero events

Excluding studies with zero events may exaggerate effect sizes.[9] An alternative is to use a continuity correction.

Displaying results

Study results may be grouped and displayed with a Forest plot.

(CC) Photo: Robert Badgett
Forest Plot showing meta-analysis of randomized controlled trials of differing target glucose control and mortality for diabetes mellitus type 2. Note the heterogeneity (P<0.05 and high I2 in circled in red) due to increased death when the glycosylated hemoglobin A (Hb A1c) target was 6.0% in the ACCORD trial[10]

Measuring consistency of study results

Consistency can be statistically tested using either the Cochran's Q or I2.[11] The I2 is the "percentage of total variation across studies that is due to heterogeneity rather than chance."[11] These numbers are usually displayed for each group of studies on a Forest plot.

In interpreting of the Cochran's Q, heterogeneity exists if its p-value is < 0.05 or possibly if < 0.10[12][13].

The following has been proposed for interpreting I2:[11]

  • Low heterogeneity is I2 = 25%
  • Moderate heterogeneity is I2 = 50%
  • High heterogeneity is I2 = 75%

or according to the Handbook of the Cochrane Collaboration:[14]

  • 0%-40%: might not be important
  • 30%-60%: may represent moderate heterogeneity
  • 50%-90%: may represent substantial heterogeneity
  • 75%-100%: considerable heterogeneity

Variations on meta-analysis

Cumulative meta-analysis

Cumulative meta-analysis has been used to show that 25 off 33 randomized controlled trials of streptokinase not necessary[15] and have shown the delay in adoption of evidence by experts[16].

Individual patient data meta-analysis

Individual patient data meta-analysis (IPD meta-analysis) may have more long lasting results than other meta-analyses.[17]

Network meta-analysis

A network meta-analysis pools studies in order to compare to treatments that have not been directly compared.[18][19] Network meta-analyses are commonly not well performed[20]and can have misleading conclusions.[21][22]

Factors associated with higher quality meta-analyses

Meta-analyses by the Cochrane Collaboration tend to be of higher quality.[23]

Individual data meta-analyses, in which the records from individual patients are pooled together into one dataset, tend to have more stable conclusions.[17]

Factors associated with lower quality meta-analyses

About a third of meta-analyses that happen to precede large randomized controlled trials will conflict with the results of the trial.[2]

Conflict of interest

Meta-analyses produced with a conflict of interest are more likely to interpret results as positive.[24]

Publication bias

Publication bias against negative studies may threaten the validity of meta-analyses that are positive and all the studies included within the meta-analysis are small.[25][26]

In performing a meta-analyses, a file drawer[27] or a funnel plot analysis[26][28] may help detect underlying publication bias among the studies in the meta-analysis.

Outcome reporting bias

Meta-analyses in which a smaller proportion of included trials provide raw data for inclusion in the meta-analysis are more likely to be positive.[29] This may be due a bias against reporting negative results.[30]

Problems with meta-analyses

Obsolescence

The conclusions of meta-analyses may be mitigated by research published after the search date of the meta-analysis. This may occur by the time the meta-analysis has been published.[31] Strategies have been developed for updating meta-analyses.[32]

References

  1. ↑ National Library of Medicine. Meta-analysis. Retrieved on 2007-12-06.
  2. ↑ 2.0 2.1 LeLorier J, Grégoire G, Benhaddad A, Lapierre J, Derderian F (August 1997). "Discrepancies between meta-analyses and subsequent large randomized, controlled trials". N. Engl. J. Med. 337 (8): 536–42. PMID 9262498. [e] Cite error: Invalid <ref> tag; name "pmid9262498" defined multiple times with different content
  3. ↑ Villar J, Carroli G, Belizán JM (March 1995). "Predictive ability of meta-analyses of randomised controlled trials". Lancet 345 (8952): 772–6. PMID 7891492. [e]
  4. ↑ Cappelleri JC, Ioannidis JP, Schmid CH, et al (1996). "Large trials vs meta-analysis of smaller trials: how do their results compare?". JAMA 276 (16): 1332–8. PMID 8861993. [e]
  5. ↑ Ioannidis JP, Cappelleri JC, Lau J (April 1998). "Issues in comparisons between meta-analyses and large trials". JAMA 279 (14): 1089–93. PMID 9546568. [e]
  6. ↑ Slavin RE (January 1995). "Best evidence synthesis: an intelligent alternative to meta-analysis". J Clin Epidemiol 48 (1): 9–18. PMID 7853053. [e]
  7. ↑ Higgins JPT, Green S (editors). Table 8.5.a: The Cochrane Collaboration's Tool for assessing risk of bias. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 [updated September 2008]. The Cochrane Collaboration, 2008. Available from www.cochrane-handbook.org.
  8. ↑ Jadad AR, Moore RA, Carroll D, et al (1996). "Assessing the quality of reports of randomized clinical trials: is blinding necessary?". Control Clin Trials 17 (1): 1–12. DOI:10.1016/0197-2456(95)00134-4. PMID 8721797. Research Blogging.
  9. ↑ Diamond GA, Bax L, Kaul S (October 2007). "Uncertain effects of rosiglitazone on the risk for myocardial infarction and cardiovascular death". Ann. Intern. Med. 147 (8): 578–81. PMID 17679700. [e]
  10. ↑ Gerstein HC, Miller ME, Byington RP, et al (June 2008). "Effects of intensive glucose lowering in type 2 diabetes". N. Engl. J. Med. 358 (24): 2545–59. DOI:10.1056/NEJMoa0802743. PMID 18539917. Research Blogging.
  11. ↑ 11.0 11.1 11.2 Higgins JP, Thompson SG, Deeks JJ, Altman DG (September 2003). "Measuring inconsistency in meta-analyses". BMJ 327 (7414): 557–60. DOI:10.1136/bmj.327.7414.557. PMID 12958120. PMC 192859. Research Blogging.
  12. ↑ Fleiss JL (December 1986). "Analysis of data from multiclinic trials". Control Clin Trials 7 (4): 267–75. PMID 3802849. [e]
  13. ↑ Dickersin K, Berlin JA (1992). "Meta-analysis: state-of-the-science". Epidemiol Rev 14: 154–76. PMID 1289110. [e]
  14. ↑ Higgins JPT, Green S:Cochrane Handbook for Systematic Reviews of Interventions. Cochrane Collaboration (2008). Retrieved on 2008-10-23.
  15. ↑ Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC (July 1992). "Cumulative meta-analysis of therapeutic trials for myocardial infarction". N. Engl. J. Med. 327 (4): 248–54. PMID 1614465. [e]
  16. ↑ Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC (July 1992). "A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction". JAMA 268 (2): 240–8. PMID 1535110. [e]
  17. ↑ 17.0 17.1 Poynard T, Munteanu M, Ratziu V, et al (June 2002). "Truth survival in clinical research: an evidence-based requiem?". Ann. Intern. Med. 136 (12): 888–95. PMID 12069563. [e] Cite error: Invalid <ref> tag; name "pmid12069563" defined multiple times with different content
  18. ↑ Lumley T (August 2002). "Network meta-analysis for indirect treatment comparisons". Stat Med 21 (16): 2313–24. DOI:10.1002/sim.1201. PMID 12210616. Research Blogging.
  19. ↑ Salanti G, Kavvoura FK, Ioannidis JP (April 2008). "Exploring the geometry of treatment networks". Ann. Intern. Med. 148 (7): 544–53. PMID 18378949. [e]
  20. ↑ Song F, Loke YK, Walsh T, Glenny AM, Eastwood AJ, Altman DG (2009). "Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews". BMJ 338: b1147. PMID 19346285. PMC 2665205. [e]
  21. ↑ Kent DM, Thaler DE (September 2008). "Stroke prevention--insights from incoherence". N. Engl. J. Med. 359 (12): 1287–9. DOI:10.1056/NEJMe0806806. PMID 18753641. Research Blogging.
  22. ↑ Chou, Roger; Susan Carson, Benjamin Chan (2009-02-01). "Gabapentin Versus Tricyclic Antidepressants for Diabetic Neuropathy and Post-Herpetic Neuralgia: Discrepancies Between Direct and Indirect Meta-Analyses of Randomized Controlled Trials". Journal of General Internal Medicine 24 (2): 178-188. DOI:10.1007/s11606-008-0877-5. Retrieved on 2009-01-26. Research Blogging.
  23. ↑ Olsen O, Middleton P, Ezzo J, et al (2001). "Quality of Cochrane reviews: assessment of sample from 1998". BMJ 323 (7317): 829–32. PMID 11597965. [e]
  24. ↑ Veronica Yank, Drummond Rennie, and Lisa A Bero, “Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study,” BMJ 335, no. 7631 (December 8, 2007), http://www.bmj.com/cgi/content/abstract/335/7631/1202 (accessed December 7, 2007).
  25. ↑ Sutton AJ, Duval SJ, Tweedie RL, Abrams KR, Jones DR (2000). "Empirical assessment of effect of publication bias on meta-analyses". BMJ 320 (7249): 1574–7. PMID 10845965. [e]
  26. ↑ 26.0 26.1 Egger M, Davey Smith G, Schneider M, Minder C (1997). "Bias in meta-analysis detected by a simple, graphical test". BMJ 315 (7109): 629–34. PMID 9310563. [e] Cite error: Invalid <ref> tag; name "pmid9310563" defined multiple times with different content
  27. ↑ Pham B et al. (2001). "Is there a "best" way to detect and minimize publication bias? An empirical evaluation". Evaluation & the Health Professions 24: 109–25. PMID 11523382. [e]
  28. ↑ Terrin N et al. (2005). "In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias". J Clin Epidemiol 58: 894–901. DOI:10.1016/j.jclinepi.2005.01.006. PMID 16085192. Research Blogging.
  29. ↑ Furukawa TA, Watanabe N, Omori IM, Montori VM, Guyatt GH (February 2007). "Association between unreported outcomes and effect size estimates in Cochrane meta-analyses". JAMA 297 (5): 468–70. DOI:10.1001/jama.297.5.468-b. PMID 17284696. Research Blogging.
  30. ↑ Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG (May 2004). "Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles". JAMA 291 (20): 2457–65. DOI:10.1001/jama.291.20.2457. PMID 15161896. Research Blogging.
  31. ↑ Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D (August 2007). "How quickly do systematic reviews go out of date? A survival analysis". Ann. Intern. Med. 147 (4): 224–33. PMID 17638714. [e]
  32. ↑ Sampson M, Shojania KG, McGowan J, et al (August 2008). "Surveillance search techniques identified the need to update systematic reviews". J Clin Epidemiol 61 (8): 755–62. DOI:10.1016/j.jclinepi.2007.10.003. PMID 18586179. Research Blogging.

External links

Retrieved from "https://citizendium.org/wiki/index.php?title=Meta-analysis&oldid=352832"