Practical Approaches toUsability TestingJulie NowickiPresidentOptavia CorporationWEB2000GoalsPut usability testing in perspectiveIn the bigger picture of usabilityCompared to other techniquesUnderstand the breadth of usability testing methods availableBe able to focus and choose the right methodDepending on the web sites development stageBalancing resource costs against benefits of various methods1TopicsThe role of usability testingDefinition and comparison to other techniquesDesigning the usability testPlanningChoosing specific methodsTuxedos or jeans?Summary: Getting what you wantAnd what your customers want!2E-commerce and UsabilityThree E-commerce segmentsB2C (Business-to-Consumer)B2B (Business-to-Business)B2E (Business-to-Employee)All focus on the customerAbility of customer toeffectively complete a task, withhigh satisfaction levelsis key to the bottom lineUsability3Qualities of a usable Web siteLearnabilityCan customers easily grasp how to use the site the very first time?MemorabilityDo customers remember how the next time?Ease-of-useCan customers navigate easily?Is the design consistent and predictable?EfficiencyCan customers complete tasks and transactions effectively?AppealMore than visual: do customers feel positive about the experience?4How can usability testing help?LearnabilityMeasure time-on-taskMeasure success rateEase-of-useMeasure errors/false startsTrack navigation patterns (and why)Record comprehension of words and graphicsEfficiencyMeasure time-on-taskAppealRecord facial expressions and commentsAdminister post-test survey5Characteristics of a usability testUses real customers (users)Customers perform actual tasks with the site or a prototypeData is recordedMay be qualitative or quantitativeIncludes customer commentsProvides objective informationFacilitated by a skilled specialistObserved by a skilled specialist6Limitations of usability testingBased on a few customersGuinea pig effectTest environment cannot duplicate the real worldUsers know they are being watchedNot thorough by natureTesting a few tasks, not every aspect of site usageSome issues are not readily apparentUse of color and fontsAccessibility7Usability testing comparedWeb site log statisticsAutomated use testing Focus groupsQA and system testingUser acceptance testingProven usability techniques:Heuristic evaluation and guidelines reviewsWalkthroughs8TipUsability encompasses customer-centered design and evaluation.HoweverUsability testing is an evaluation technique.Usability testing is not a design technique.9Customer-centered design processAnalysisEvaluationUnderstand objectivesUnderstand customersUnderstand their tasksDevelop design conceptSpecify the detailsTest with real customers (usability testing)Review with design specialistsDesignIterative!10Actual TimelineMonth 12345-7 VisionField StudyCustomer analysisTask analysisWorkflowReview Analysis DocConceptual ModelPrototype SketchesStart Interact Spec Doc1st Online PrototypeRefine Interact Spec Doc2nd Online Prototype(real code)Finish Interact Spec DocDevelopUserAssistanceWalkthroughUsability Testing with SketchesUsability Testing with Online PrototypeGuidelines ReviewUsability TestingOngoing mini-reviews11Anatomy of a usability testPlan and DesignReport and RecommendConductPlan: Answer the big questionsSelect the test methodsGet participants12Overview of usability testing methodsTo prepare for the test, you must1.Plan Answer some big-picture questions Understand objectives, customers, and tasks2.Choose the test methods Considering a number of parameters, Pick the best usability testing methods for your situation13Planning:Big decisions to make firstWhat are you trying to accomplish?When will you do the test?Based on product lifecycleWhat are the customer profiles?What tasks will customers do?How will you present the results?How will you make recommendations?Explore design alternatives14What are you trying to accomplish?Find out how your site compares to the competitionFind out if your new design works forExisting customersNew customersGet statistically sound data15Tips for setting objectivesPrioritize the four usability componentsLearnabilityEase-of-useEfficiencyAppealBe specificNO “Registration is easy to use.”YES “New customers should be able to complete the registration process in 5 minutes without help.”16TipUsability testing done early in the design cyclereduces risk.17The Cost to Make a ChangeAdapted from Rapid Application Development by James MartinUsabilityU-test a mustUsability18Dont get fooled by online development tools!Sketches are highly advantageousThey are easy to throw awayPeople dont assume the system is donePeople are more likely to suggest changesUsability testing can be done with sketchesTip19TipUsability testing done late in the production processhelps you plan for the next iteration.20Who are the customers?Marketing information such asAgeIncomeEducationCareerExperience with the InternetExperience with online shoppingAnticipated frequency of use (your site)Environment they work inHome, office21What are the tasks?Creating task scenarios is a must!Dont use language that leads the userNO “Go to My Profile and change your address to”YES “You have moved. Change your address to”22Presenting the resultsConsider allowing observers during the testMake sure they do not interfereHighlight tapeCompelling and efficientResults of data collectionSummary of issues, prioritized byUsability criticalityImplementation effortRecommendations23The next stepOnce you haveconsidered what you want to accomplish, and whenidentified your customer profilesdetermined the tasks you want customers to performdecided how to communicate the resultsyou are ready to design the test methods24Test methods:Parameters that you must selectEquipment and facilitiesWhat to measureQuantitative and qualitativeHow to collect measurementsUse of participants (protocol)How many participantsLevel of facilitationLevel of participant verbalization25Equipment and facilitiesConference roomPaper notesOne cameraPaper notesPortable labLogging softwareLab facilityLogging software26Example Usability Labs27Advantages of video tapeRecords very valuable, compelling qualitative dataPage navigationExpressionsComments in contextProvides a real snippet of an actual customer experienceWatch outSomeone needs to understand the equipmentTime to setup, review, create highlight tape28What to measure: QuantitativeTime on taskNumber of errorsSuccessful completion of task (yes/no)How you collect measurements is determined by facility and equipment29What to watch for: QualitativeNavigation patternsGetting lost?PredictabilityUnderstand graphics?Understand words?Looking for it to work like something else?FlexibilityLooking for a quicker way?Looking for help?30Protocols (Use of participants)IndividualPairedComparisonGood for competitive analysisBetween-subjects: each customer uses one siteWithin-subjects: each customer uses both sites31How many participants?5-8 participants will find 80% of usability problems4 is good minimum numberQualitative dataEarly prototypeQualitative and quantitativeLater in developmentPaired or comparison protocols3 4 5 6 7 8 9 10 11 12Statistically sound results32Level of facilitationFacilitator in another roomFacilitator in same roomDiscussion after each taskFacilitator and participant interact during task33Level of participant verbalizationThink-aloudHeads-down work34InteractionXQualitative dataDiscussion-orientedInformal environmentThink-aloudQuantitative dataHeads-down taskFormalLabMostly qualitativeUser works aloneFormalLabHighLowLow High Participant verbalizationFacilitation35Putting it all togetherLoFi Prototype(paper)HiFi Prototype(online)Pre-productionProduction siteQualitative dataQuantitative DataConference rmConf rm or labLabLabHi facilitationMediumMediumLo facilitationHi participant verbalizationLo participant verbalization3-4 users4-8 users4-8 users6-12 usersParticipants 1 hour1-2 hours1-2 hoursParticipants 2+ hoursIndividualsIndividualsIndivs or pairsIndivs or pairsDaysWeeksInformal Formal36Now that weve said thatCan you do informal testing on a production site?Sure!Can you do quick, but formal testing in a lab on a prototype?Sure!37What ifYoure in the fast-paced e-commerce environmentYou want customer data but you need itNow (yesterday)Whenever you make significant changesYou want guidance in making design decisions or changes38VariationLoFi Prototype(paper)HiFi Prototype(online)Pre-productionProduction siteQualitative dataQuantitative DataConference rmConf rm or labLabLabHi facilitationMediumMediumLo facilitationHi participant verbalizationLo participant verbalization3-4 users4-8 users4-8 users6-12 usersParticipants 1 hour1-2 hours1-2 hoursParticipants 2+ hoursIndividualsIndividualsIndivs or pairsIndivs or pairsDaysWeeksInformal Formal39Resource costs will be determined by your choicesCost of facilities and equipmentTime for each participantYour time resourcesPlanning testConducting testReporting and decision-makingStaff resources and skill levelsGenerally all go up as you move from informal to formal40Results“When we introduced an electronic expense payment system, users declared the application incredibly efficient as early as the testing phase. That never would have happened if we hadnt had the usability group to help us meet our business objectives. No company should ever move forward on a project in todays business environment without some usability in their back pocket.”From Supervisor of Integrated Business Practices41Usability is more than just usability testingIt is a processJulie NowickiPresidentOptavia CorporationJNowicki608 260 900042