Copyright LEXTA GmbH 2006On the difficulties in determining effort estimation accuracy in practiceMeLLow WorkshopUrsula Lbbert-PassingOctober 17-18, 2006In the IT Management lifecycle, LEXTA covers benchmarking, cost cutting and sourcing as well as professionalising processes and projectsLIFECYCLE IT MANAGEMENTIT BenchmarkingProfessionalising IT processesIT SourcingIT Cost CuttingProfessionalising IT projects2Accuracy_v2.pptEstimation accuracy is considered as THE quality criterion for an estimation method The most common formula for determining effort estimation accuracy (in percent) in practice is Positive value = Over-estimation Negative value = Under-estimation (budget overrun)3Accuracy_v2.pptEstimation accuracy is determined by comparing values from different points of timeProject executionEstimationMeasurement of actual effortProject lifetimeUnexpected event (e. g. requirement shift)Comparison: Estimation accuracyt4Accuracy_v2.pptFour factors “dilute” the effect of the estimation method on estimation accuracyInformation gapFaulty measurement of actual effortProject managementUnexpected eventsEstimation methodEstimation accuracy5Accuracy_v2.pptAll factors contribute jointly to estimation (in)accuracy FactorContributes to estimation (in)accuracy, becauseEstimation method Method takes/does not take all relevant information into account (parameters, number of aspects an expert can consider) Method produces (in)correct conclusions from available information (e. g. by model assumptions, extent of expert experience) Information gap At time of estimation, available project information (requirements, architecture, team etc.) is incomplete assumptions necessary At time of actual effort measurement, more (complete) information is available “Information gap” Project managementProject management influences actual effort, e. g. by (in)efficient coordination of activities and teams Parkinsons Law“Unexpected eventsUnexpected events lead to unexpected effort Information gap“ between time of estimation and time of actual effort measurement Strictly, estimated and measured effort are not comparable any more Faulty measure- ment of actual effortMeasured effort is not necessarily the actual effort due to Manipulation of measured effort: Incentive exists when employees are evaluated by their estimation accuracy Faulty measurement: When actual effort does not “fit” into the measurement scheme (e. g. project phases), it may be booked somewhere non-appropriate Effort measured for one measurement category (e. g. one phase) is not necessarily the actual effort No measurement of overtime Measured effort is too lowNoise“6Accuracy_v2.pptThere are remedies to exclude three of the noise“ factorsInformation gapFaulty measurement of actual effortProject managementUnexpected eventsEstimation methodEstimation accuracyDocument input information for estimates and update regularlyUpdate estimates after unexpected eventsAssure quality of effort measurementsDo not evaluate employees based on their estimation accuracy Match structures of effort measurement and estima- tion templates Measure overtimeEx-post estimates are often recommended but imprac- tical in practice (extra effort)7Accuracy_v2.ppt