1 SelectionBiasandSensitivityAnalysis1 OverviewofselectionbiasRosenbaum ssensitivityanalysisStatalab runningsensitivityanalysiswithrbounds 2 ReadingsforThisSession Guo S Fraser M 2009 PropensityScoreAnalysis ThousandOaks CA SagePublications Inc Sections8 1 8 3 8 6ofChapter8 Rosenbaum P R 2002 Observationalstudies 2nded NewYork Springer Chapter4 Rosenbaum P R 2005 Sensitivityanalysisinobservationalstudies InB S EverittandD C Howell Eds EncyclopediaofStatisticsinBehavioralScience pp 1809 1814 NewYork JohnWiley Sons Ltd 3 Overviewofselectionbias 4 ImportancetoStudySelectionBias Althoughthenewmethodsofpropensityscoreanalysis PSA addresssomeofthecorechallengesfoundincausalinferencesofobservationalstudies theycannotcompetewithrandomizedcontrolledtrial RCT onakeyissue PSAonlycontrolsforselectionbiasgeneratedbymeasuredcovariates whileRCTbalancedataandcontrolsforbiasgeneratedbybothmeasuredandunmeasuredcovariates Itisforthisreason researchersusingPSAshouldbealertedbyselectionbiases particularlybythosegeneratedbytheunmeasuredcovariates 5 SourcesofSelectionBias Self selectionResearcher sselectionTheclassicexample thefailuretopredictHarryS Truman s1948presidentialvictoryThemodernexample internetsurveyNonprobabilitysamplingAdministrator sselectionGeographicselectionorgeographicmismatchMeasurementerror nonrandomerrorRater seffectinlongitudinalstudySelectionduetoattrition 6 OvertBiasversusHiddenBias Rosenbaum 2002 p 71 Anobservationalstudyisbiasedifthetreatedandcontrolgroupsdifferpriortotreatmentinwaysthatmatterfortheoutcomesunderstudy Anovertbiasisonethatcanbeseeninthedataathand forinstance priortotreatment treatedsubjectsareobservedtohavelowerincomesthancontrols Ahiddenbiasissimilarbutcannotbeseenbecausetherequiredinformationwasnotobservedorrecorded Overtandhiddenbiasesthataffectcausalinferencearecontrolledinwell implementedrandomizedexperiments Balancingthedataonhiddenbiasisespeciallyaprobleminobservationalstudies 7 Rosenbaum ssensitivityanalysis 8 TheBasicIdea 1 Correctingthehidden selectionproblemessentiallyinvolvestakingstepssuchasrespecifyingananalyticmodelbyusingadditionalvariables collectingadditionaldata redesigningastudyasarandomizedexperiment ormakingotherendeavors Evidently thesestrategiescanbeexpensiveandtime consuming Itisoftenpreferable then tostartbyconductingsensitivityanalysestoestimatethelevelofbias Thistypeofanalysisseekstoanswerthequestion Howsensitivearefindingstohiddenbias Althoughsensitivityanalysisisexploratory itisanimportantstepinanalysesusingthecorrectivemodels RosenbaumandRubin 1983 andRosenbaum 2002 2005 haverecommendedthatresearchersroutinelyconductsensitivityanalysesinobservationalstudies 9 TheBasicIdea 2 Thecrucialideaofthismethodis Onemanipulatestheestimatedoddsofreceivingaparticulartreatmenttoseehowmuchtheestimatedtreatmenteffectsmayvary Whatonewantstofindisthattheestimatedtreatmenteffectsarerobusttoaplausiblerangeofselectionbiases Berk 2003 p 231 Ifastudyisfreeofhiddenbias thentwounitswithsameobservedcovariatesxshouldhavesamechancesofreceivingtreatment Sothedifferenceinoddsratioofreceivingtreatmentbetweentwounitswiththesamecovariatesxindicatesthelevelofsensitivitytohiddenselectionbias 10 TheBasicIdea 2 Considertwounitsjandk iftheyhavethesameobservedcovariatesx i e x j x k butdifferentchancesofreceivingtreatment i e j k thenit slikelythatthereishiddenbias Asensitivityanalysisasks Howwouldinferencesabouttreatmenteffectsbealteredbyhiddenbiasesofvariousmagnitudes Supposethe sdifferatagivenx Howlargewouldthesedifferenceshavetobetoalterthequalitativeconclusionsofastudy Rosenbaum 2002 p 106 11 TheBasicIdea 3 Theoddsthatunitsjandkreceivethetreatmentareand respectively TheoddsratioisThesensitivityanalysisgoesfurthertoassumethatthisoddsratioforunitswithsamexwasatmostsomenumberof 1 thatis 12 TheBasicIdea 4 Bytheabovedefinitions if were1 then j k wheneverx j x k sothestudywouldbefreeofhiddenbias If 2 thentwounitsthatappearsimilar thathavethesamex coulddifferintheiroddsofreceivingthetreatmentbyasmuchasafactorof2 sooneunitmightbetwiceaslikelyastheothertoreceivetreatment Thus isameasureofthedegreeofdeparturefromastudythatisfreeofhiddenbias Asensitivityanalysiswillconsiderseveralpossiblevaluesof andshowhowtheinferencesmightchange Astudyissensitiveifvaluesof closeto1couldleadtoinferencesthatareverydifferentfromthoseobtainedassumingthestudyisfreeofhiddenbias Astudyisinsensitiveifextremevaluesof arerequiredtoaltertheinference Rosenbaum 2002 p 107 13 TheBasicIdea 5 TheoriginalframeworkofthesensitivityanalysisforhiddenbiascamefromCornfieldetal 1959 However theCornfieldetalstudyignoredsamplingvariability whichishazardousexceptinextremelylargesamples Rosenbaum 2002 attendedtosamplingvariationbydevelopingmethodstocomputetheboundsoninferencequantities suchaspvaluesorconfidenceintervals Thus foreach 1 weobtainanintervalofpvaluesthatreflectuncertaintybecauseofhiddenbias As increases thisintervalbecomeslonger andeventuallyitbecomesuninformative includingbothlargeandsmallpvalues Thepoint atwhichtheintervalbecomesun