Computing PrimerforApplied LinearRegression, Third EditionUsing SPSSKatherine St. Clair & Sanford WeisbergDepartment of Mathematics, Colby CollegeSchool of Statistics, University of MinnesotaAugust 3, 2009c?2005, Sanford WeisbergHome Website: www.stat.umn.edu/alrContentsIntroduction10.1Organization of this primer40.2Data files50.2.1Documentation50.2.2Getting the data files forSPSS60.2.3Getting the data in text files60.2.4An exceptional file60.3Scripts60.4The very basics70.4.1Reading a data file70.4.2Saving text output and graphs90.4.3Normal,F,tand2tables110.5Abbreviations to remember120.6Copyright and Printing this Primer131Scatterplots and Regression131.1Scatterplots131.2Mean functions191.3Variance functions191.4Summary graph19vviCONTENTS1.5Tools for looking at scatterplots191.6Scatterplot matrices202Simple Linear Regression232.1Ordinary least squares estimation232.2Least squares criterion232.3Estimating2252.4Properties of least squares estimates252.5Estimated variances252.6Comparing models: The analysis of variance252.7The coefficient of determination,R2262.8Confidence intervals and tests272.9The Residuals283Multiple Regression313.1Adding a term to a simple linear regression model313.2The Multiple Linear Regression Model313.3Terms and Predictors313.4Ordinary least squares323.5The analysis of variance323.6Predictions and fitted values334Drawing Conclusions354.1Understanding parameter estimates354.1.1Rate of change354.1.2Sign of estimates354.1.3Interpretation depends on other terms in the mean function354.1.4Rank deficient and over-parameterized models354.2Experimentation versus observation364.3Sampling from a normal population364.4More onR2364.5Missing data364.6Computationally intensive methods375Weights, Lack of Fit, and More395.1Weighted Least Squares395.1.1Applications of weighted least squares405.1.2Additional comments40CONTENTSvii5.2Testing for lack of fit, variance known405.3Testing for lack of fit, variance unknown405.4GeneralFtesting415.5Joint confidence regions426Polynomials and Factors436.1Polynomial regression436.1.1Polynomials with several predictors436.1.2Using the delta method to estimate a minimum or a maximum466.1.3Fractional polynomials466.2Factors466.2.1No other predictors476.2.2Adding a predictor: Comparing regression lines486.3Many factors496.4Partial one-dimensional mean functions496.5Random coefficient models517Transformations537.1Transformations and scatterplots537.1.1Power transformations537.1.2Transforming only the predictor variable537.1.3Transforming the response only537.1.4The Box and Cox method557.2Transformations and scatterplot matrices557.2.1The 1D estimation result and linearly related predictors567.2.2Automatic choice of transformation of the predictors567.3Transforming the response577.4Transformations of non-positive variables578Regression Diagnostics: Residuals598.1The residuals598.1.1Difference between eande598.1.2The hat matrix598.1.3Residuals and the hat matrix with weights598.1.4The residuals when the model is correct608.1.5The residuals when the model is not correct608.1.6Fuel consumption data608.2Testing for curvature61viiiCONTENTS8.3Nonconstant variance618.3.1Variance Stabilizing Transformations618.3.2A diagnostic for nonconstant variance618.3.3Additional comments628.4Graphs for model assessment628.4.1Checking mean functions628.4.2Checking variance functions639Outliers and Influence659.1Outliers659.1.1An outlier test659.1.2Weighted least squares659.1.3Significance levels for the outlier test659.1.4Additional comments669.2Influence of cases669.2.1Cooks distance669.2.2Magnitude ofDi669.2.3ComputingDi669.2.4Other measures of influence669.3Normality assumption6710 Variable Selection6910.1 The Active Terms6910.1.1 Collinearity6910.1.2 Collinearity and variances7010.2 Variable selection7010.2.1 Information criteria7010.2.2 Computationally intensive criteria7010.2.3 Using subject-matter knowledge7010.3 Computational methods7010.3.1 Subset selection overstates significance7110.4 Windmills7110.4.1 Six mean functions7110.4.2 A computationally intensive approach7111 Nonlinear Regression7311.1 Estimation for nonlinear mean functions7311.2 Inference assuming large samples73CONTENTSix11.3 Bootstrap inference7411.4 References7412 Logistic Regression7512.1 Binomial Regression7512.1.1 Mean Functions for Binomial Regression7512.2 Fitting Logistic Regression7512.2.1 One-predictor example7612.2.2 Many Terms7712.2.3 Deviance7712.2.4 Goodness of Fit Tests7712.3 Binomial Random Variables7712.3.1 Maximum likelihood estimation7712.3.2 The Log-likelihood for Logistic Regression7712.4 Generalized linear models77References79Index810IntroductionThis computer primer supplements the book Applied Linear Regression (alr),third edition, by Sanford Weisberg, published by John Wiley & Sons in 2005.It shows you how to do the analyses discussed in alr using one of severalgeneral-purpose programs that are widely available throughout the world. Allthe programs have capabilities well beyond the uses desc