Questionnaire Design and Evaluation,Nanjing Normal University Ni Chuanbin 2010.11,Shanxi Normal University 2010.11 Lin Fen,2,Questionnaire Design and Evaluation,1. Introduction 2. Questionnaire Design and Evaluation,3,1. Introduction,1.1 Definition 1.2 Time consideration 1.3 Cost consideration 1.4 Length of a Questionnaire 1.5 Advantages and disadvantages,4,1.1 Definition,A questionnaire is a set of questions designed to collect information from a respondent. A questionnaire may be interviewer-administered or respondent-completed, using paper-and-pencil methods of data collection, telephone methods or computer-assisted modes of completion or Internet data collection. Questionnaires play a central role in the data collection process. They have a major impact on respondent behavior, interviewer performance, collection cost and respondent relations and therefore on data quality.,5,1.2 Time considerations-1,Survey Time Planning Form,Hours is the best estimate of the actual number of hours required to complete the task. Duration is the amount of time that will pass until the task is completed.,6,1.2 Time considerations-2,Many researchers underestimate the time required to complete a research project. The best advice is to be generous with your time estimates. Things almost always take longer than we think they should. Most researchers have to divide their time among many projects. They simply cannot give all their time to any one project. For example, my estimate of goal clarification may be four hours, but other commitments allow me to spend only two hours a day on this study. My hours estimate is four hours, and my duration estimate is two days. To arrive at the final time estimates, add the individual estimates. The hours estimate is used for budget planning and the duration estimate is used to develop a project time line.,7,1.3 Cost Considerations,Some of the most common costs are: Proposal typing and editing. Cover letter and questionnaire typing. Addressing mailing envelopes. Following up on non-respondents. Mailing list cost (if necessary). Artwork. Incentives. Data entry and verification. Statistical analysis programmer. Distribution of the final report.,8,1.3 Cost Considerations : Incentives,Provide incentives as a motivation for a properly completed questionnaire. What does the respondent get for completing your questionnaire? Altruism is rarely an effective motivator. Attaching a dollar bill to the questionnaire works well. If the information you are collecting is of interest to the respondent, offering a free summary report is also an excellent motivator. Whatever you choose, it must make the respondent want to complete the questionnaire.,9,1.4 Length of a Questionnaire,The optimal length of an organizational survey questionnaire has always been a controversial topic. While the design of the instrument should try to minimize the disruption related to the survey, shorter is not always better. The length of your questionnaire should be based on the minimum amount required to sufficiently cover the necessary topics. As a general rule, with only a few exceptions, long questionnaires get less response than short questionnaires. In fact, the shorter the better. Response rate is the single most important indicator of how much confidence the researchers can place in the results. Therefore, a researcher must do everything possible to maximize the response rate. One of the most effective methods of maximizing response is to shorten the questionnaire.,10,1.4 Length of a Questionnaire,Does length of questionnaire matter? A randomised trial of response rates to a mailed questionnaire.Iglesias C, Torgerson D.Department of Health Studies, University of York, UK.,OBJECTIVE: To assess whether length of questionnaire affects response rates. METHODS: A quasi-randomised trial of women aged 70 years and over in a general practice in England. Three questionnaires of different lengths: a clinical questionnaire (four pages); the same questionnaire plus the EuroQol (five pages); the same questionnaire plus the SF-12 (seven pages). The impact of length on the proportion of returned questionnaires and item completion rates was assessed. RESULTS: In total, 847 questionnaires were mailed; response rates were 49%, 49% and 40% to the short, medium and long questionnaires, respectively. This difference was statistically significant when the short questionnaire was compared against the longest instrument (9% difference; 95% confidence interval (CI) of difference = 0.3% to 16.6%). Item completion rates for the clinical questionnaire did not differ. Respondents did not differ in age or self-reported health status between the three groups. CONCLUSIONS: Increasing the length of a questionnaire from five to seven pages reduces response rates from women aged 70 years and over. However, lengthening a questionnaire does not seem to affect the quality of responses.,11,1.5 Advantages and disadvantages,(1) compared with interview A. Advan