Team IRALARBreanna Heidenburg - Michael Lenisa - Daniel WentzelAdvisor: Dr. MalinowskiThe Project Why is it importantThe Goals System breakdown Image recognition Point transformation User InterfaceThe ResultsWhat is our project?Track a users eye and use the information to control a computer cursor.Enhances Human Computer InteractionSpeed of use Hands-free use3 Part System Image Processing Application Calibration and Mapping system GUI designed for gaze-based interactionSystems developed concurrently and independentlySeparate Applications at run-timeHardware and Image Processing ApplicationImage Processing ApplicationUser Interface Application Image Processing & Cursor Control ThreadShared Process DataUDP Server ThreadUI ThreadUDP channelOS channelHardware Camera QuickCam Pro for Notebooks Visible Spectrum Camera Polarizer Tiffen 25mm polarizing filter Removes glare from eye reflections Lighting Diffused LED Slightly distracting to the user, but necessary to provide light for the camera LitEye LE-500 High resolution (SVGA) Color Display Translucent or opaque operation Stationary relative to users eyeImage Processing ApplicationReal time pupil tracking system Developed in C using OpenCV image processing libraries Traditional image processing and blob trackingCapabilities Locate and determine center of pupil in image Low light and high reflection environments All eye colors Data logging and static test modes Packaged into self contained Windows installer for easy deployment onto any computerQuery Frame From CameraRed ChannelSmoothContrast StretchExtract BlobsReject False PositivesCalculate Blob CenterFinal RecognitionSummary The Good Dynamically adapts to changing lighting conditions and eye types Maintains performance in low-light and specularly noisy conditions The Bad Still relies on Logitech camera drivers Extreme reflections still cause problemsExamples of performance in poor conditionsLow Light Difficult False PositiveNote: Image brightness and contrast artificially enhanced for human visibilityCalibration and point mappingImage Processing ApplicationImage Processing & Cursor Control ThreadSystem for mapping the location of the center of the pupil to a pixel on a computer screenMust Calibrate for each User Geometry The eye is not flat but a screen is User Customization All eyes are different Everyone wears the HMD differently User Training Calibration system also acts as a quick tutorial3 dimensional best fit plane Currently using a 4th degree best fitXpix = A1 + Xeye*B1 + Yeye*C1 Ypix = A2 + Xeye*B2 + Yeye*C2Calibration sub-system determines these coefficientsHow do we solve the problem? Multiple Variable Linear Regression Least Squares Y = B0 + B1x1 + + Bkxk Uses matrix algebra to obtain a coefficient matrix B = (XX)-1XYCursor position error actual vs. determined position Horizontal and vertical error in screen pixelsError MeshAccuracy varies with position and skill of user Corners of screen most difficult to calibrate Focusing on a rapidly changing location requires skillLimitations 1 Accuracy of human vision system Eye SaccadesOriginal error goal 2% of screen dimension on both axesAchieved error 1.18% Horizontal 1.46% VerticalHow do we click? Monitor eye movements Identify pauses “Dwell time” - When eye position is focused on a single area for a period of time Currently set at 5 frames (200 mS) Generally, it takes 230 mS for a hand to click a mouse.Improved interaction speed Trackpad vs. Mouse vs. Gaze Tracking 53% Increase over Trackpad 12% Increase over Traditional MouseCustom GUI interface and CommunicationsImage Processing ApplicationUser Interface Application Image Processing & Cursor Control ThreadUI ThreadCustom GUI for Gaze Tracking ApplicationsWhy? Gaze tracking accuracy limited by inherent properties of the human vision system Traditional GUI too small and intrusive for use with transparent HMD Demonstrate applications of gaze trackingModern GUI Design WPF using XAML layout Windows Presentation Foundation eXtensible Application Markup Language XAML is similar to HTML Uses tags and code-behind in a similar style to javascript GUI coded in C#Multiple pages within the interfaceScreens for functionality testing even gamesAbility to minimize interfaceMain Menu ScreenLarge text and buttonsWide spacing between optionsSimple layoutPlacement of features in high-accuracy areasImage Processing ApplicationUser Interface Application Image Processing & Cursor Control ThreadUI ThreadOS channelWhy Allows processes to communicate Allows relay of time sensitive information2 Communication Channels OS Channel Omni-directional (Image Processing to User Interface) UDP Channel Multi-directional Separate Thread in Image Processing ApplicationImage