Week 6, Lecture 12, Part 5: What is Collinearity? - YouTube

now that we've talked about issues of non-constant  variance and linearity and what transformations  

we can use to try to correct for the structure  of our data to make sure that our assumptions  

hold we're now going to cross into a new  problem that we may have and we're going to  

you know ponder whether or not this is an issue  when our predictors are strongly correlated with  

each other or in other words what are we going  to do when one predictor is a linear combination  

of other predictors so an example of this  would be if we have an x1 and x2 that are  

perfectly correlated with each other they  are going to be what we're going to call them  

co-linear so for example we could imagine  that x2 is a perfect linear combination of x1  

such that 5 again is where the  relationship would cross the y axis

when x equals 0 and again the slope of that  line would be 0.5 so we can imagine that there  

could be instances in which our predictors are  perfectly co-linear with each other this is going  

to be an issue that we call multi-colinearity  where one or more of our predictors are nearly  

linear related to the others and if one of the  predictors is almost perfectly predicted from  

the other set of variables then we are also  going to have multi-collinearity in the model  

so for the rest of the lecture we're going to  talk about how multi-collinearity could affect our  

statistical inference as well as our predictions  and we're going to talk about how we can detect  

multi-co-linearity once we have detected it  what can we do to resolve any issues that it may  

have on our actual statistical inference i  want to begin by talking about the effects  

that multi-collinearity can have so  the first effect that you can look for  

is your fitted values your y hats are probably  not going to be affected by multicollinearity  

what is going to be affected is the  variability in your beta estimates so the  

standard errors around your estimated coefficients  are going to be artificially larger because  

we're not as certain as to what partial effect is  really driving the true underlying relationship  

between our covariates and our predictors  so when we have a high standard error of  

our betas that's going to mean that fewer of  those estimated coefficients are significant  

even when a true relationship may actually exist  another artifact of multicollinearity is that  

our estimated coefficients are going to be  really sensitive to minor changes in the model  

so if there are really large differences in your  estimated coefficients when you leave one variable  

out and then include it that is a good indication  that there may be some multicollinearity  

in your model another effect of multicollinearity  could be the fact that the sample that you have  

is not really generalizable to the total overall  population and so if you had a new sample you may  

end up getting a very different model because  again we're having a hard time distinguishing  

exactly what partial effect is really driving  the true underlying relationship and what is  

nice is that any coefficients or covariates  that are not multi-collinear with each other  

they should not be affected by this so if there  are coefficients that are widely swinging around  

that should be an indication that those are the  covariates or that are co-linear with each other