1Measure Training Results nSome ConceptsnSome experiencesnSome lessons2Training ProcessnNeeds AnalysisnDesignnDeliverynEvaluation3Four Evaluation LevelsnReaction How do participants feel about the program?nLearning To what extent did the participants increase knowledge, improve skills, and/or change attitude?nBehavior To what extent did their job behavior change?nResults What final results occurred? (Quantity, quality, safety, sales, costs, profits, ROI)4Level One: ReactionWhat is evaluated?nContentnTrainernMethodologynMaterialnFacilitiesnLogisticsnRegistration5Example QuestionsnProbability you will use ideas from this session in your worknContent relevance to my jobnPractical examples and experiencenSpeakers knowledge of subject areanSpeakers presentation/facilitation skillsnSpeakers ability to respond to questionsnGroup participation6Level One: ReactionWhat is evaluated?nOverall7Example QuestionsnWhat did you like most about the program?nWhat did you like least about the program?nIn what ways could this program be improved?nWould you recommend this course to others who are interested in the subject?nWhich of the following features were important in your decision to attend? Rank in order of importance)8Level One: ReactionHow to evaluate?nQuestionnairesnInterviewnFocus GroupnPhone survey9Level One: ReactionQuestionnaire: scalesnExcellentWellFairPoornExcellentVery GoodGoodFairPoornStrongly Agree Disagree7 6 5 4 32 1nHigh Low 109 87 6 5 4 32 110Level One: ReactionAdvantagesnEasy to executenParticipants are freenSome relationship to on job performanceIssuesnPurely subjective Halo effect (smile sheet) Horn effect Central tendencynTime pressure11Level One: ReactionTip 1: highlight purpose, solicit cooperationnTo determine the degree that the seminar met your needs, we would like you to give us your honest opinionnTo make our future sessions as meaningful as possible, we would appreciate your candid evaluation of this program.nYour comments will be used to improve future offerings of this seminar.12Level One: ReactionTip 2: Encourage comments Comments are more accurate than scores Tip 3: Set up baseline scores Historical data Benchmarking dataPilot group data (Formative evaluation) Tip 4: Combine questionnaires, interviews, focus groups Tip 5: instant feedback for multi subject courses13Level Two: LearningHow to evaluate?nTestnDemonstrationnPresentationnDiscussionnRole Play14Level Two: LearningAdvantagesnPressure to participantsnPressure to trainernFirst step to change behaviorIssuesnTensionnReliabilitynCreditabilitynDifficultynDifferentiationnNot the best indicator for on job change15Level Two: LearningTip 1: Fully utilize testnTest is good to evaluate knowledge based training Functional skills New Employee Orientation Supervisor ABCnBe serious about test result of formal trainingnInstant self scoring and symbolic incentive for informal trainingnReliability, Creditability, Difficulty and DifferentiationnSet up baselinenFinal score vs. gain1617Level Two: LearningTip 2: Set up rules in advance for demonstration and presentation Topic Time Criteria to evaluate Evaluators are key success factor Tip 3: People dont speak might be best learner Tip 4: People good at role play might not be good learner18Example:Management SkillsnParticipants selected 3 topics from 11nTopic requirementnTime and assessorsnPracticenRight before the stage.Participants knew the topicnAssessors interacted with and provided feedback to participants19Example: Leadership StyleDescribe a situation when you used what you learned about leadership styles to improve the effectiveness of a relationship. Be sure to cover: qWhat motivated you to examine the style difference qWhat did you find and how qWhat did you do to flex your style qWhat were the results20Example: Performance Appraisal21Level Three: BehaviorHow to evaluate?nObservationnPerformance Appraisal InformationnSupervisor feedback (oral or written)nCustomer feedback (oral or written) nEmployee feedback (oral or written)22Level Three: BehaviorAdvantagesnDirect purpose of trainingnRecognized by supervisor and management teamIssuesnTime consumingnExpertise scarcitynCooperation from supervisor/customernMultiple “cause- effect”23Level Three: BehaviorTip 1: Select program carefullynBehavior based Company behavior Professional skills Managerial skills Customer service (smile, response)nTrainers cooperation Trainers expertise in evaluation Documents ready before training24Level Three: BehaviorTip 2: Evaluation time in accordance with skills practice Tip 3: Supervisors cooperation is key success factor Commitment when register Mutual benefit Brief the key points25Level Three: BehaviorTip 4: “Take advantage” of vendor Key behavior identification Questionnaires design Time Result analysis26Example: Time Management27TOT: Trans