StatQuest: Logistic Regression - YouTube

If you can fit a line you can fit a squiggle if you can, make me laugh you can, make me giggle stat quest

Hello, i'm josh stormer and welcome to stat quest today we're going to talk about logistic regression

This is a technique that can be used for traditional statistics as, well as machine learning so let's get right to it

Before we dive into logistic regression let's take. A, step back and review, linear regression in

Another stat quest, we talked, about linear regression

We had some data

Weight and size

then, we fit a line to it and

With that line, we could do a lot of things

First we could calculate r-squared and determine if weight and size are

correlated large values imply a large effect, and

Second calculate a p-value to determine if the r-squared value is statistically significant and

Third, we could use the line, to predict, size given weight if a, new, mouse has this weight

Then this, is the size that, we predict, from the weight although

We didn't mention it at the time using data to predict something falls under the category of machine learning

So plain old linear regression is a form of machine learning

We also talked a little bit about multiple regression

Now, we are trying to predict, size, using weight and blood volume

Alternatively we could, say that, we are trying to model size using weight and blood volume

Multiple regression, did the same things that normal regression did

we calculated r-squared and

we calculated the p-value and

We could predict, size, using weight and blood volume and

This, makes multiple regression a slightly fancier machine learning method

We also talked, about how, we can use discrete measurements like genotype to predict size if you're

Not familiar with the term genotype don't freak out it's. No, big deal just know that it refers to different types of mice

lastly, we could compare models

So on the left side we've got normal regression, using weight to predict size and

We can, compare those predictions to the ones, we get from multiple regression, where we're using weight and blood volume to predict size

Comparing the simple model to the complicated one tells us if we need to measure weight and blood volume to accurately predict

Size or if we can get, away, with just weight

Now that we remember all the cool, things, we can, do with linear regression

Let's talk, about logistic regression

Logistic regression is similar to linear regression

except

Logistic regression predicts whether something, is true or false instead of predicting something continuous, like, sighs

these mice are obese and

These mice are not

Also instead of fitting a line to the data logistic regression fits an s-shaped logistic function

The curve goes from zero to one?

And that, means that the curve tells you the probability that a mouse is obese based on its weight

If we weighed a very heavy mouse?

There is a high probability that the new, mouse is obese?

If we weighed an intermediate mouse

Then there is only a 50% chance of the mouse is obese?

Lastly, there's only a small probability that a light mouse is obese

Although, logistic regression, tells the probability that a mouse is obese or not it's usually used for classification

For example if the probability of mouse is obese is greater than 50%

Then we'll classify it as obese

Otherwise we'll classify it as not obese

Just like with linear regression, we can, make simple models in this case, we can have obesity predicted, by weight or?

more complicated models in this case obesity is predicted by weight and genotype in

This, case, obesity is predicted. By weight and genotype and age and

Lastly, obesity is predicted by weight genotype, age and

Astrological sign in other words just like linear regression logistic

Regression can work with continuous data, like weight and age and discrete data like genotype and astrological sign

We can, also test to see if each variable is useful for predicting obesity

however

Unlike normal regression, we can't easily compare the complicated model to the simple model and we'll talk more about, why in a bit

Instead we just test to see if a variables affect on the prediction is significantly different from zero

If not it, means that the variable is not helping the prediction

We use, wald's tests to figure this out we'll talk, about that in another stat quest in

This, case, the astrological sign is totes useless

That statistical jargon for not helping

That, means we can, save time and space in our study. By leaving it out

Logistic regressions ability to provide probabilities and classify, new samples using continuous and discrete measurements

Makes it a popular machine learning method

One big difference between linear regression and logistic regression is how the line is fit to the data

With linear regression, we fit the line, using least squares

In other words, we find the line that minimizes the sum of the squares of these residuals

We also use the residuals to calculate r. Squared and to compare simple models to complicated models

Logistic regression doesn't have the same concept of a residual so it can't use least squares and it can't calculate r squared

instead it uses something called maximum likelihood

There's a whole stack quest on maximum likelihood so see that for details but in a nutshell

You, pick a probability scaled. By weight of observing an obese mouse just like this curve and

You, use that to calculate the likelihood of observing a, non obese mouse that weighs this much and

then you calculate the likelihood of observing, this mouse and

you, do that for all of the mice and

Lastly, you multiply all of those likelihoods together that's the likelihood of the data given this line

then you shift the line and calculate a new, likelihood of the data and

then ship the line and calculate the likelihood, again, and

again

Finally the curve with the maximum value for the likelihood is selected bam

in summary logistic regression can be used to classify samples and

it can, use different types of data like, size and/or genotype to do that classification and

it can, also be used to assess what variables are useful for classifying samples ie

Astrological sign is totes useless

Hooray, we've made it to the end of another exciting stat quest do you, like this StackQuest, and want to see more please subscribe

if you, have suggestions for future stat quests, well put them in the comments below, until next time quest on