1 1 SlideSlide 2 2 SlideSlideChapter 12Chapter 12 Simple Linear Regression Simple Linear RegressionnnSimple Linear Regression ModelSimple Linear Regression ModelnnLeast Squares MethodLeast Squares MethodnnCoefficient of DeterminationCoefficient of DeterminationnnModel AssumptionsModel AssumptionsnnTesting for SignificanceTesting for SignificancennUsing the Estimated Regression EquationUsing the Estimated Regression Equation for Estimation and Prediction for Estimation and PredictionnnComputer SolutionComputer SolutionnnResidual Analysis: Validating Model AssumptionsResidual Analysis: Validating Model Assumptions 3 3 SlideSlideSimple Linear Regression ModelSimple Linear Regression Modely y = = 0 0 + + 1 1x x + + where:where: 0 0 and and 1 1 are called are called parameters of the modelparameters of the model, , is a random variable called the is a random variable called the error term error term. .n n The The simple linear regression modelsimple linear regression model is: is:n n The equation that describes how The equation that describes how y y is related to is related to x x and and an error term is called the an error term is called the regression modelregression model. . 4 4 SlideSlideSimple Linear Regression EquationSimple Linear Regression EquationnnThe The simple linear regression equationsimple linear regression equation is: is: E E( (y y) is the expected value of ) is the expected value of y y for a given for a given x x value. value. 1 1 is the slope of the regression line. is the slope of the regression line. 0 0 is the is the y y intercept of the regression line. intercept of the regression line. Graph of the regression equation is a straight line.Graph of the regression equation is a straight line.E E( (y y) = ) = 0 0 + + 1 1x x 5 5 SlideSlideSimple Linear Regression EquationSimple Linear Regression EquationnnPositive Linear RelationshipPositive Linear RelationshipE E E( ( (y y y) ) )x x xSlope Slope 1 1is positiveis positiveRegression lineRegression lineInterceptIntercept 0 0 6 6 SlideSlideSimple Linear Regression EquationSimple Linear Regression EquationnnNegative Linear RelationshipNegative Linear RelationshipE E E( ( (y y y) ) )x x xSlope Slope 1 1is negativeis negativeRegression lineRegression lineInterceptIntercept 0 0 7 7 SlideSlideSimple Linear Regression EquationSimple Linear Regression EquationnnNo RelationshipNo RelationshipE E E( ( (y y y) ) )x x xSlope Slope 1 1is 0is 0Regression lineRegression lineInterceptIntercept 0 0 8 8 SlideSlideEstimated Simple Linear Regression EquationEstimated Simple Linear Regression EquationnnThe The estimated simple linear regression equationestimated simple linear regression equation is the estimated value of is the estimated value of y y for a given for a given x x value. value. b b1 1 is the slope of the line. is the slope of the line. b b0 0 is the is the y y intercept of the line. intercept of the line. The graph is called the estimated regression line.The graph is called the estimated regression line. 9 9 SlideSlideEstimation ProcessEstimation ProcessRegression ModelRegression Modely y = = 0 0 + + 1 1x x + + Regression EquationRegression EquationE E( (y y) = ) = 0 0 + + 1 1x xUnknown ParametersUnknown Parameters 0 0, , 1 1Sample Data:Sample Data:x x y yx x1 1 y y1 1. . . . . . . x xn n y yn nb b0 0 and and b b1 1provide estimates ofprovide estimates of 0 0 and and 1 1EstimatedEstimatedRegression EquationRegression Equation Sample StatisticsSample Statisticsb b0 0, , b b1 1 1010 SlideSlideLeast Squares MethodLeast Squares MethodnnLeast Squares CriterionLeast Squares Criterionwhere:where:y yi i = = observedobserved value of the dependent variable value of the dependent variable for the for the i ithth observation observation y yi i = = estimatedestimated value of the dependent variable value of the dependent variable for the for the i ithth observation observation 1111 SlideSlidennSlope for the Estimated Regression EquationSlope for the Estimated Regression EquationLeast Squares MethodLeast Squares Method 1212 SlideSlidenny y-Intercept for the Estimated Regression Equation-Intercept for the Estimated Regression Equation Least Squares MethodLeast Squares Methodwhere:where:x xi i = value of independent variable for = value of independent variable for i ithth observationobservationn n = total number of observations = total number of observations_ _y y = mean value for dependent variable = mean value for dependent variable_ _x x = mean value for independent variable = mean value for independent variabley yi i = value of dependent variable for = value of dependent variable for i ithth observationobservation 1313 SlideSlideReed Auto periodically hasReed Auto periodically hasa special week-long sale. a special week-long sale. As part of the advertisingAs part of the advertisingcampaign Reed runs one orcampaign Reed runs one ormore television commercialsmore television commercialsduring the weekend preceding the sale. Data from aduring the weekend preceding the sale. Data from asample of 5 previous sales are shown on the next slide.sample of 5 previous sales are shown on the next slide.Simple Linear RegressionSimple Linear RegressionnnExample: Reed Auto SalesExample: Reed Auto Sales 1414 SlideSlideSimple Linear RegressionSimple Linear RegressionnnExample: Reed Auto SalesExample: Reed Auto SalesNumber ofNumber of TV Ads TV AdsNumber ofNumber ofCars SoldCars Sold1 13 32 21 13 314142424181817172727 1515 SlideSlideEstimated Regression EquationEstimated Regression EquationnnSlope for the Estimated Regression EquationSlope for the Estimated Regression Equationnny y-Intercept for the Estimated Regression Equation-Intercept for the Estimated Regression EquationnnEstimated Regression EquationEstimated Regression Equation 1616 SlideSlideScatter Diagram and Trend LineScatter Diagram and Trend Line 1717 SlideSlideCoefficient of DeterminationCoefficient of DeterminationnnRelationship Among SST, SSR, SSERelationship Among SST, SSR, SSEwhere:where: SST = total sum of squaresSST = total sum of squares SSR = sum of squares due to regressionSSR = sum of squares due to regression SSE = sum of squares due to errorSSE = sum of squares due to errorSST = SST = SSR SSR + + SSE SSE 1818 SlideSlidennThe The coefficient of determinationcoefficient of determination is: is:Coefficient of DeterminationCoefficient of Determinationwhere:where:SSR = sum of squares due to regressionSSR = sum of squares due to regressionSST = total sum of squaresSST = total sum of squaresr r2 2 = SSR/SST = SSR/SST 1919 SlideSlideCoefficient of DeterminationCoefficient of Determinationr r2 2 = SSR/SST = 100/114 = .8772 = SSR/SST = 100/114 = .8772 The regression relationship is very strong; 88%The regression relationship is very strong; 88%of the variability in the number of cars sold can beof the variability in the number of cars sold can beexplained by the linear relationship between theexplained by the linear relationship between thenumber of TV ads and the number of cars sold.number of TV ads and the number of cars sold. 2020 SlideSlideSample Correlation CoefficientSample Correlation Coefficientwhere:where: b b1 1 = the slope of the estimated regression = the slope of the estimated regression equation equation 2121 SlideSlideThe sign of The sign of b b1 1 in the equation in the equation is “+”. is “+”.Sample Correlation CoefficientSample Correlation Coefficientr rxyxy = +.9366 = +.9366 2222 SlideSlideAssumptions About the Error Term Assumptions About the Error Term 1. 1. The error The error is a random variable with mean of zero. is a random variable with mean of zero.2. 2. The variance of The variance of , denoted by , denoted by 2 2, is the same for, is the same for all values of the independent variable. all values of the independent variable.3. 3. The values of The values of are independent. are independent.4. 4. The error The error is a normally distributed random is a normally distributed random variable. variable. 2323 SlideSlideTesting for SignificanceTesting for Significance To test for a significant regression relationship, weTo test for a significant regression relationship, we must conduct a hypothesis test to determine whether must conduct a hypothesis test to determine whether the value of the value of 1 1 is zero. is zero. Two tests are commonly used:Two tests are commonly used:t t Test TestandandF F Test Test Both the Both the t t test and test and F F test require an estimate of test require an estimate of 2 2, , the variance of the variance of in the regression model. in the regression model. 2424 SlideSlidennAn Estimate of An Estimate of Testing for SignificanceTesting for Significancewhere:where:s s 2 2 = MSE = SSE/( = MSE = SSE/(n n - - 2) 2)The mean square error (MSE) provides the estimateThe mean square error (MSE) provides the estimateof of 2 2, and the notation , and the notation s s2 2 is also used. is also used. 2525 SlideSlideTesting for SignificanceTesting for SignificancennAn Estimate of An Estimate of To estimate To estimate we take the square root of we take the square root of 2 2. . The resulting The resulting s s is called the is called the standard error ofstandard error of the estimatethe estimate. . 2626 SlideSlidennHypothesesHypotheses nnTest StatisticTest StatisticTesting for Significance: Testing for Significance: t t Test Test 2727 SlideSlidennRejection RuleRejection RuleTesting for Significance: Testing for Significance: t t Test Testwhere: where: t t is based on a is based on a t t distribution distributionwith with n n - 2 degrees of freedom - 2 degrees of freedomReject Reject HH0 0 if if p p-value -value a a or or t t t t 2828 SlideSlide1. 1. Determine the hypotheses.Determine the hypotheses.2. 2. Specify the level of significance.Specify the level of significance.3. 3. Select the test statistic.Select the test statistic. = .05 = .054. 4. State the rejection rule.State the rejection rule.Reject Reject HH0 0 if if p p-value -value 3.182 (with 3.182 (with3 degrees of freedom)3 degrees of freedom)Testing for Significance: Testing for Significance: t t Test Test 2929 SlideSlideTesting for Significance: Testing for Significance: t t Test Test5. 5. Compute the value of the test statistic.Compute the value of the test statistic.6. 6. Determine whether to reject Determine whether to reject HH0 0. .t t = 4.541 provides an area of .01 in the upper = 4.541 provides an area of .01 in the uppertail. Hence, the tail. Hence, the p p-value is less than .02. (Also,-value is less than .02. (Also,t t = 4.63 3.182.) We can reject = 4.63 3.182.) We can reject HH0 0. . 3030 SlideSlideConfidence Interval for Confidence Interval for 1 1n n HH0 0 is rejected if the hypothesized value of is rejected if the hypothesized value of 1 1 is not is not included in the confidence interval for included in the confidence interval for 1 1. .n n We can use a 95% confidence interval for We can use a 95% confidence interval for 1 1 to test to test the hypotheses just used in the the hypotheses just used in the t t test. test. 3131 SlideSlidennThe form of a confidence interval for The form of a confidence interval for 1 1 is: is:Confidence Interval for Confidence Interval for 1 1wherewhere is the is the t t value providing an area value providing an areaof of a a/2 in the upper tail of a /2 in the upper tail of a t t distribution distributionwith with n n - 2 degrees of freedom- 2 degrees of freedomb b1 1 is the is thepointpointestimatorestimatoris theis themarginmarginof errorof error 3232 SlideSlideConfidence Interval for Confidence Interval for 1 1Reject Reject HH0 0 if 0 is not included in if 0 is not included inthe confidence interval for the confidence interval for 1 1. .0 0 is not included in the confidence interval. is not included in the confidence interval. Reject Reject HH0 0= 5 +/- 3.182(1.08) = 5 +/- 3.44= 5 +/- 3.182(1.08) = 5 +/- 3.44or 1.56 to 8.44or 1.56 to 8.44nnRejection RuleRejection Rulenn95% 95% Confidence Interval for Confidence Interval for 1 1nnConclusionConclusion 3333 SlideSlidennHypothesesHypotheses nnTest StatisticTest StatisticTesting for Significance: Testing for Significance: F F Test TestF F = MSR/MSE = MSR/MSE 3434 SlideSlidennRejection RuleRejection RuleTesting for Significance: Testing for Significance: F F Test Testwhere:where:F F is based on an is based on an F F distribution with distribution with1 degree of freedom in the numerator and1 degree of freedom in the numerator andn n - 2 degrees of freedom in the denominator - 2 degrees of freedom in the denominatorReject Reject HH0 0 if if p p-value -value F F 3535 SlideSlide1. 1. Determine the hypotheses.Determine the hypotheses.2. 2. Specify the level of significance.Specify the level of significance.3. 3. Select the test statistic.Select the test statistic. = .05 = .054. 4. State the rejection rule.State the rejection rule.Reject Reject HH0 0 if if p p-value -value 10.13 (with 10.13 (with 1 d.f.1 d.f.in numerator andin numerator and 3 d.f. in denominator) 3 d.f. in denominator)Testing for Significance: Testing for Significance: F F Test TestF F = MSR/MSE = MSR/MSE 3636 SlideSlideTesting for Significance: Testing for Significance: F F Test Test5. 5. Compute the value of the test statistic.Compute the value of the test statistic.6. 6. Determine whether to reject Determine whether to reject HH0 0. . F F = 17.44 provides an area of .025 in the upper = 17.44 provides an area of .025 in the upper tail. Thus, the tail. Thus, the p p-value corresponding to -value corresponding to F F = 21.43 = 21.43 is less than 2(.025) = .05. Hence, we reject is less than 2(.025) = .05. Hence, we reject HH0 0. .F F = MSR/MSE = 100/4.667 = 21.43 = MSR/MSE = 100/4.667 = 21.43 The statistical evidence is sufficient to concludeThe statistical evidence is sufficient to concludethat we have a significant relationship between thethat we have a significant relationship between thenumber of TV ads aired and the number of cars sold. number of TV ads aired and the number of cars sold. 3737 SlideSlideSome Cautions about theSome Cautions about theInterpretation of Significance TestsInterpretation of Significance Testsn n Just because we are able to reject Just because we are able to reject HH0 0: : 1 1 = 0 and = 0 and demonstrate statistical significance does not enabledemonstrate statistical significance does not enableus to conclude that there is a us to conclude that there is a linear relationshiplinear relationshipbetween between x x and and y y. .n n Rejecting Rejecting HH0 0: : 1 1 = 0 and concluding that the = 0 and concluding that therelationship between relationship between x x and and y y is significant does is significant does not enable us to conclude that a not enable us to conclude that a cause-and-effectcause-and-effectrelationshiprelationship is present between is present between x x and and y y. . 3838 SlideSlideUsing the Estimated Regression EquationUsing the Estimated Regression Equationfor Estimation and Predictionfor Estimation and Predictionwhere:where:confidence coefficient is 1 - confidence coefficient is 1 - andandt t /2 /2 is based on ais based on a t t distributiondistributionwith with n n - 2 degrees of freedom - 2 degrees of freedomnnConfidence Interval Estimate of Confidence Interval Estimate of E E( (y yp p) )nnPrediction Interval Estimate of Prediction Interval Estimate of y yp p 3939 SlideSlideIf 3 TV ads are run prior to a sale, we expectIf 3 TV ads are run prior to a sale, we expectthe mean number of cars sold to be:the mean number of cars sold to be:Point EstimationPoint Estimation y y = 10 + 5(3) = 25 cars = 10 + 5(3) = 25 cars 4040 SlideSlidennExcels Confidence Interval OutputExcels Confidence Interval OutputConfidence Interval for Confidence Interval for E E( (y yp p) ) 4141 SlideSlideThe 95% confidence interval estimate of the mean The 95% confidence interval estimate of the mean number of cars sold when 3 TV ads are run is:number of cars sold when 3 TV ads are run is:Confidence Interval for Confidence Interval for E E( (y yp p) )25 25 + + 4.61 = 20.39 4.61 = 20.39 to 29.61 carsto 29.61 cars 4242 SlideSlidennExcels Prediction Interval OutputExcels Prediction Interval OutputPrediction Interval forPrediction Interval for y yp p 4343 SlideSlideThe 95% prediction interval estimate of the The 95% prediction interval estimate of the number of cars sold in one particular week when 3 number of cars sold in one particular week when 3 TV ads are run is:TV ads are run is:Prediction Interval forPrediction Interval for y yp p25 25 + + 8.28 = 16.72 8.28 = 16.72 to 33.28 carsto 33.28 cars 4444 SlideSlideResidual AnalysisResidual Analysisn n Much of the residual analysis is based on anMuch of the residual analysis is based on an examination of graphical plots. examination of graphical plots.n n Residual for Observation Residual for Observation i in n The residuals provide the best information about The residuals provide the best information about . .n n If the assumptions about the error term If the assumptions about the error term appear appear questionable, the hypothesis tests about the questionable, the hypothesis tests about the significance of the regression relationship and the significance of the regression relationship and the interval estimation results may not be valid. interval estimation results may not be valid. 4545 SlideSlideResidual Plot Against Residual Plot Against x xnnIf the assumption that the variance of If the assumption that the variance of is the same is the same for all values of for all values of x x is valid, and the assumed is valid, and the assumed regression model is an adequate representation of the regression model is an adequate representation of the relationship between the variables, thenrelationship between the variables, then The residual plot should give an overallThe residual plot should give an overall impression of a horizontal band of points impression of a horizontal band of points 4646 SlideSlidex x0 0Good PatternGood PatternResidualResidualResidual Plot Against Residual Plot Against x x 4747 SlideSlideResidual Plot Against Residual Plot Against x xx x0 0ResidualResidualNonconstantNonconstant Variance Variance 4848 SlideSlideResidual Plot Against Residual Plot Against x xx x0 0ResidualResidualModel Form Not AdequateModel Form Not Adequate 4949 SlideSlidennResidualsResidualsResidual Plot Against Residual Plot Against x x 5050 SlideSlideResidual Plot Against Residual Plot Against x x 5151 SlideSlideEnd of Chapter 12End of Chapter 12