How to do Meta AnalysisArindam BasuAssociate Director,Fogarty International Training ProgramKolkata, IndiaFebruary, 2005arin.phone: 919830153666What is meta analysis?Quantitative approach for systematically combining results of previous research to arrive at conclusions about the body of research.What does it mean?Quantitative : numbersSystematic : methodicalcombining: putting togetherprevious research: whats already doneconclusions: new knowledgeThe popularity of meta analysesNumber of Meta Analysis publications are steadily increasing since 1993. We graphed the counts of journal articles included “meta analysis” as “publication type” from Pubmed, from years 1993 through 2004Four Steps of Meta AnalysisIdentify your studiesDetermine eligibility of studiesInclusion: which ones to keepExclusion: which ones to throw outAbstract Data from the studiesAnalyze data in the studies statisticallyIdentify your studiesBe methodical: plan firstList of popular databases to searchPubmed/MedlineEmbaseCochrane Review/Trials RegisterOther strategies you may adoptHand search (go to the library.)Personal references, and emailsweb, eg. Google ()How to Search for literatureFormulate your question appropriatelyIf you are searching pubmedUse Medical Subject Headings (MeSH) 1Lookup word in text word, abstract, title 2Combine 1 with 2 using boolean logicSet up proper filtersFor Others, use text word, abstractHuh?!, what is boolean logic?Boolean Logic: ANDthis is AND zone, covering common area between two ellipsesBoolean Logic ORthis is boolean OR, covering the two ellipsesExample: Research IssueLets say we want to know whether streptokinase is protective for death from acute myocardial infarction. How should we set up a search strategy? We will search pubmed onlyThe Search“streptokinase”text word OR “acute myocardial infarction”text word produces ALL articles that contain EITHER streptokinase OR acute myocardial infarction anywhere in the text inclusive, manystreptokinase text word AND “acute myocardial infarction” text word will capture only those subsets that have BOTH streptokinase AND acute myocardial infarction anywhere in the text restrictive, fewNext, we shall look at the PUBMED Screen Choose your DATABASE hereRemember to choose both PUBMED, and MESH for formulating search. Choose PUBMED CENTRAL for free articles!Keep some, throw out others Cannot include all studiesKeep the ones with high levels of evidencegood qualitycheck with QUOROM guidelinesUsually, MA done with RCTs Case series, and case reports definitely outSelection problems are major problemsread the article I sentMA = Meta Analysis; RCT = Randomized Controlled TrialPlan of ActionARE THE STUDIES ELIGIBLE FOR MA (STEP I)?DISCARDYESNOENTER INTO A SPECIFIED FORMATABSTRACT THE DATAHow to Abstract Data: GuidelinesCreate a spreadsheet (Excel, or OpenOffice Calc)For each study, create the following columns:name of the studyname of the author, year published number of participants who received interventionnumber of participants who were in control armnumber who developed outcomes in interventionnumber who developed outcomes in control armLets do that to our streptokinase myocardial infarction study, next Spreadsheet Data for Strepto StudyWe created seven columnstrial: trial identity codetrialname: name of trialyear: year of the studypop1: study populationdeaths1: deaths in studypop0: control populationdeaths0: deaths in controlWe got like 22 studies to do our meta analysis, after allAnalyze Data StatisticallyCombine data to arrive at a summary, 3 measuresEffect Size (Odds Ratio)Variance with 95% Confidence IntervalTest of heterogeneityTwo GraphsForest PlotFunnel PlotExamine why the studies are heterogeneous, if they areUse Statistical Packages, several choices Lets see what we got for streptokinase versus deaths from AMISummary Estimates for Strepto StudyMantel Haenszel OR=0.7795% Confidence Interval0.72, 0.83Test of Heterogeneity:Chi-square (df=21) = 31.5P-Value = 0.07The pooled Odds Ratio shows that those receiving streptokinase at AMI are about 77% at risk of death (23% less likely to die)That in 95 out of 100 such meta analyses, the pooled Odds Ratio would lie between 0.72 and 0.83, indicating a statistically significant protective effectThat these studies were not significantly heterogeneousForest PlotThe dotted line passes across null, or 1.0The Risk Estimate of each study is lined up on each side of the dotted line, with 95% CI spread as the lineThe diamond in the below is the summary estimateThe two ends of the diamond indicate 95% CIThey call it a forest plot so that you dont miss the wood for the trees! The size of the black square box indicates weight of the studyFunnel Plot: what and how to readPlots the effect size against the sample size of the studyTo study a funnel plot, look at its LOWER LEFT corner, thats where negative or null studies are locatedIf EMPTY, this indicates “PUBLICATION BIAS”Note that here, the plot fits in a funnel, and that the left corner is not all that empty, but we cannot rule out publication biasIssues in meta analysisChoosing a modelFixed effects model or random effects?Bias in meta analysispoor quality of trialspublication biasQuality control in meta analysisQUOROM guidelinesStatistical Software for meta analysisFixed Effects or Random Effects Model?conduct if it is reasonable to assume underlying Rx effect is SAME for all studiesPooling: Mantel Haenszel ORTest: test of heterogeneityIf significant, go for random effects modelshort 95% CI for summarysmaller summary estimateOR=0.77 0.72,0.83Conduct if test of heterogeneity is significant (shows heterogeneity)Assume that TRUE log odds ratio comes from a normal distributionMethod: DerSimonian Lairs method (DSL) of calculating Odds RatioOR=0.78 0.69,0.88Fixed Effects ModelRandom Effects ModelBias in MetaanalysisPoor Quality of TrialsTo avoid them, learn more at CONSORT statementPublication Biasstudy showing beneficial effects of new treatment more likely to be published than one showing no effectnegative trials assumed to contribute less; never show up in the literature baseuse several approaches to avoid thisUse Funnel Plots to examine the influence of publication biasQuality Control in MA:QUOROM Table Detailed Guidelines A Good Checklist Use it for reporting Meta Analysis Systematic reviewsStatistical Software for Meta AnalysisHuge Checklist/Free Software:EpiMeta: from Epi InfoRevman: from Cochrane Collaboration“meta” package in R for statistical computingNon-freemeta module in STATASummarizing.Defined meta analysis quantitative research synthesisOutlined basic stepsInformation retrievalData AbstractionData AnalysisModel Selection: Fixed Effects or Random effectsOutlined some issues and listed softwareFeel free to shoot questions at arin.