The Simple Regression Model y b0 b1x u 1Economics 20 Prof Anderson Some Terminology w In the simple linear regression model where y b0 b1x u we typically refer to y as the nDependent Variable or nLeft Hand Side Variable or nExplained Variable or nRegressand 2Economics 20 Prof Anderson Some Terminology cont w In the simple linear regression of y on x we typically refer to x as the nIndependent Variable or nRight Hand Side Variable or nExplanatory Variable or nRegressor or nCovariate or nControl Variables 3Economics 20 Prof Anderson A Simple Assumption w The average value of u the error term in the population is 0 That is w E u 0 w This is not a restrictive assumption since we can always use b0 to normalize E u to 0 4Economics 20 Prof Anderson Zero Conditional Mean w We need to make a crucial assumption about how u and x are related w We want it to be the case that knowing something about x does not give us any information about u so that they are completely unrelated That is that w E u x E u 0 which implies w E y x b0 b1x 5Economics 20 Prof Anderson x1x2 E y x as a linear function of x where for any x the distribution of y is centered about E y x E y x b0 b1x y f y 6Economics 20 Prof Anderson Ordinary Least Squares w Basic idea of regression is to estimate the population parameters from a sample w Let xi yi i 1 n denote a random sample of size n from the population w For each observation in this sample it will be the case that w yi b0 b1xi ui 7Economics 20 Prof Anderson y4 y1 y2 y3 x1x2x3x4 u1 u2 u3 u4 x y Population regression line sample data points and the associated error terms E y x b0 b1x 8Economics 20 Prof Anderson Deriving OLS Estimates w To derive the OLS estimates we need to realize that our main assumption of E u x E u 0 also implies that w Cov x u E xu 0 wWhy Remember from basic probability that Cov X Y E XY E X E Y 9Economics 20 Prof Anderson Deriving OLS continued w We can write our 2 restrictions just in terms of x y b0 and b1 since u y b0 b1x w E y b0 b1x 0 w E x y b0 b1x 0 wThese are called moment restrictions 10Economics 20 Prof Anderson Deriving OLS using M O M w The method of moments approach to estimation implies imposing the population moment restrictions on the sample moments w What does this mean Recall that for E X the mean of a population distribution a sample estimator of E X is simply the arithmetic mean of the sample 11Economics 20 Prof Anderson More Derivation of OLS w We want to choose values of the parameters that will ensure that the sample versions of our moment restrictions are true w The sample versions are as follows 12Economics 20 Prof Anderson More Derivation of OLS wGiven the definition of a sample mean and properties of summation we can rewrite the first condition as follows 13Economics 20 Prof Anderson More Derivation of OLS 14Economics 20 Prof Anderson So the OLS estimated slope is 15Economics 20 Prof Anderson Summary of OLS slope estimate w The slope estimate is the sample covariance between x and y divided by the sample variance of x w If x and y are positively correlated the slope will be positive w If x and y are negatively correlated the slope will be negative w Only need x to vary in our sample 16Economics 20 Prof Anderson More OLS w Intuitively OLS is fitting a line through the sample points such that the sum of squared residuals is as small as possible hence the term least squares w The residual is an estimate of the error term u and is the difference between the fitted line sample regression function and the sample point 17Economics 20 Prof Anderson y4 y1 y2 y3 x1x2x3x4 1 2 3 4 x y Sample regression line sample data points and the associated estimated error terms 18Economics 20 Prof Anderson Alternate approach to derivation w Given the intuitive idea of fitting a line we can set up a formal minimization problem w That is we want to choose our parameters such that we minimize the following 19Economics 20 Prof Anderson Alternate approach continued w If one uses calculus to solve the minimization problem for the two parameters you obtain the following first order conditions which are the same as we obtained before multiplied by n 20Economics 20 Prof Anderson Algebraic Properties of OLS w The sum of the OLS residuals is zero w Thus the sample average of the OLS residuals is zero as well w The sample covariance between the regressors and the OLS residuals is zero w The OLS regression line always goes through the mean of the sample 21Economics 20 Prof Anderson Algebraic Properties precise 22Economics 20 Prof Anderson More terminology 23Economics 20 Prof Anderson Proof that SST SSE SSR 24Economics 20 Prof Anderson Goodness of Fit w How do we think about how well our sample regression line fits our sample data w Can compute the fraction of the total sum of squares SST that is explained by the model call this the R squared of regression w R2 SSE SST 1 SSR SST 25Economics 20 Prof Anderson Using Stata for OLS regressions w Now that we ve derived the formula for calcul