1Data Collection:Overview and Design MatrixResearch Methods for Public AdministratorsDr. Gail JohnsonDr. G. Johnson, www.researchdemystified.orgSteps in the Research ProcessPlanning 1. Determining Your Questions 2. Identifying Your Measures and Measurement Strategy 3. Selecting a Research Design 4. Developing Your Data Collection Strategy 5. Identifying Your Analysis Strategy 6. Reviewing and Testing Your Plan2Dr. G. Johnson, www.researchdemystified.orgPlanning the Data Collection StrategyIf you dont know where you are going, you can wind up anywhereDr. G. Johnson, www.researchdemystified.orgData Collection StrategynConsists of three inter-related stepsnFirst: what data collection methods make sense given:nWhat you want to knownNumbers or stories?nWhere the data residesnEnvironment, files, peoplenResources availableDr. G. Johnson, www.researchdemystified.orgData Collection StrategynSecond: how much structure is needed? nQuantitativenUse when you want to do statistical analysis, want to be precise, know exactly what you want to measure and/or want to cover a large group nQualitativenUse when you want anecdotes or in-depth information, when you are not sure what you want to measure, and/or there is no need to quantify5Dr. G. Johnson, www.researchdemystified.orgData Collection StrategynThird: how much data to collect?nIdentify the population of interest nDecide whether to collect data from the entire population or a sample?nIf a samplenHow large a sample?nRandom or non-randomnSampling will be presented laterDr. G. Johnson, www.researchdemystified.orgQuantitative ApproachnHighly structured: check-a-box formatnClosed-ended questionsnPrecise measures every timenReliable: same exact measures every timenUnforgiving of mistakesnHarder to develop than open-ended questionsnEasier to analyze than qualitative datancomputer will easily crunch the numbers7Dr. G. Johnson, www.researchdemystified.orgQualitative ApproachnOpen-ended questions, fill-in-the-blanksnEasier to develop than closed-ended questionsnForgiving of mistakesnOften yields “rich”, i.E. Lots of data8Dr. G. Johnson, www.researchdemystified.orgQualitative ApproachnLabor intensive to gathernHarder to analyze: nTime-consuming with many possible meanings nRisk of bias in interpretingnWe may unconsciously see things that are not really there but fit our viewpoint9Dr. G. Johnson, www.researchdemystified.orgIn Practice: A Mix Is Often UsednA research project might combine several approaches, including both quantitative and qualitative: nAvailable data, interviews and a surveynSurvey and a focus groupnAvailable data with observations10Dr. G. Johnson, www.researchdemystified.orgFirst: Where Does the Data Reside?nDoes the data already exist?nWhere are the best sources of data?nWho collected the data?nHow reliable is the data?nCan you get access to it?11Dr. G. Johnson, www.researchdemystified.orgFirst: Where Does the Data Reside?nDo you have to collect new data?nIs the data in the environment?nIs it in a filing cabinet or computer database?nIs it in news stories?nIs it in legislative language?nIs it with people?12Dr. G. Johnson, www.researchdemystified.orgYour Local School: Where Is This The Data? nNumber of studentsnNumber of teachersnEducational background of teachersnNumber of books and other learning resourcesnNumber of graduatesnKnowledge and skills learned by studentsnSatisfaction of parents13Dr. G. Johnson, www.researchdemystified.orgConsider Operational Definitions And Trade-offs nOperational definitions: nDo students who get GEDs count in graduation rates?nWhich data might be easier to obtain?nWhich data might be harder to obtain?nTrade-offs often have to be madenData collection limitations must be acknowledged14Dr. G. Johnson, www.researchdemystified.orgData Collection: General GuidelinesnData collection must be consistentnEstablish the plan and rules and stick to themnUse available data if already exists.nFaster, cheaper, easiernBut find out how they: nCollected the datanDefined the variablesnEnsured accuracy of the data15Dr. G. Johnson, www.researchdemystified.orgData Collection: General GuidelinesIf you must collect original data:nEstablish procedures that are consistently followed nMaintain accurate records of definitions, codingnPre-test, pre-test, pre-test, pre-testnVerify accuracy of coding, data input 16Dr. G. Johnson, www.researchdemystified.orgDesign Matrix: A Planning ToolnA tool that helps focus on detailsnIt is a visualnFocus is on content not writing stylenIt is a living documentnMany changes before it is finalnThis is generic formatnChange it to fit your styleDr. G. Johnson, www.researchdemystified.orgDesign Matrix: A Planning Tool nBasically tracks the planning stepsnResearch question and sub questionsnLikely measuresnLikely designnLikely sources of informationnLikely data collection strategynLikely sampling choice18Dr. G. Johnson, www.researchdemystified.orgDesign Matrix FormChange as NeededQuestio