Variance Inflation Factor (VIF) for Detecting Multicolinearity in Python - YouTube

Hey Hi, Collinearity is a situation in

which 2 features are heavily correlated with each other.

You can plot a heatmap & decide which 2 features are correlated to each other and

keep one of them.

However multicolinearity is a more difficult problem to solve

wherein multiple features

may be correlated to 1 particular feature & thus removing multicollinearity in case

of linear regression, is a difficult challenge.

In this video, I talk about Variance Inflation Factor

or VIF in short which deals with multicolinearity.

So stay tuned.

Let's start by importing the necessary modules.

The whole purpose of this

exercise is to remove multicollinearity for a linear regression data set.

I have chosen the Boston House prediction data set.

and this is what is given the description of the data set

I save all my feature data into a variable called as X.

I save all my target data into a variable called as Y

and I save all the column names into a list called

as names.

I create a data frame just out of the feature columns.

So for detecting multicolinearity, I don't need the target columns, I just need the

feature columns.

So this is how my input data frame looks like I just have columns which are from my futures.

So the idea of variance inflation factor or VIF is I take one column in one iteration

as my target variable and the remaining columns as my feature variables.I fit a linear regression

model on it.

Calculate the R square value and I take the inverse of one minus the R squared value and

I calculate the VIF which is variance inflation factor.

So in my first iteration, CRIM column would be my target variable.

& the rest of the columns would be my features.

I will fit a linear regression model & calculate the VIF score.

So your VIF is nothing else, but the inverse of 1 - Rsquare score of the model that you fit

That is what I'm doing here

So I run through all the columns.

I select Y as that particular column and it appears in my list and X when it does not

appear in my list.

So here in my first tension, my Y will only contain the CRIM column and X will contain

all the remaining columns. I fit a simple linear regression module.

& I calculate the Rsquared score.

Later the inverse of one minus R square gives me the VIF value.

Then I run this this is what I get.

The RSquare value of CRIM column is 0.52 keeping all other columns as features

And the VIF is 2.1

The higher the VIF, the more are the chances of it being removed

In my linear regression model

For example.

I have my VIF factor of RM column to be equal to 77.95 or close to 78 whereas the R squared

term was 0.99, so as you can clearly see RM column can be explained as a combination of

multiple other columns or other features.So the RM column can be neglected by creating

a linear regression.

Similarly higher the value of variance inflation factor, the more are the chances that you

can drop it off when you build a linear regression model.

This is all that I had in terms of explaining what variance inflation factor or VIF is and how

it can be used to remove multicolinearity from your data.

If you do have any questions with what we covered in this video then feel free to ask

in the comment section below & I'll do my best to answer those.

If you enjoy these tutorials & would like to support them then the easiest way is to simply like the video & give it a thumbs up

& also it's a huge help to share these videos with anyone who you think would find them useful.

Be sure to subscribe for future videos & thank you all for watching.