F1 Score in Machine Learning - YouTube

Hi guys

We have already seen accuracy, precision and recall as measures of a machine learning

model performance.

In this video, I'll explain what F1 score is why is it required when you already have

precision and recall which do a fairly good job in evaluating a machine learning model?

There is a famous precision recall trade off and I'll explain it using a simple example.

Say, suppose we have a pond of fish and we know the total number of fishes within.

Our goal

is to build a model that catches red fishes.

In the first model that I make.

It consist of two fishing nets which have baits containing food which my red fishes like.

what do we observe?

we observe that we end up capturing two red fishes, which is

what we desired.

Precision is out of all the samples at the classifier classified as positive

what portion of it where actually correct the following test that we have performed

shows great precision that means out of the two red fishes that we have got both of them

are red but if you look carefully there are many more red fishes in my pond so we missed

out on five red fishes which should have been caught by my classifier.

So, my precision was

really high here, but my recall was less.

So what I do I make a model that increases recall now.

In order to do that, I make a bigger

fishnet.

I change the bait which is now like by all the fishes and I capture a lot of fishes now

using the new bait and just for a recap recall is out of all the positive samples what portion

did my classifier pickup.

Now, if you see in this diagram carefully say, for example, approximately

there are eight fishes out of eight.

I'm able to capture six red fishes, which is doing a good job.

Now the problem with this model now is I am capturing a lot of blue fishes in order to

incorporate large amount of red fishes.So my precision, although is high but my recall

is very low.

Sorry my yes.

My recall is high but my precision is very low.

So this is the fundamental trade off between precision and recall in our model with high

precision, most of most are all of the fishes were caught were red it had a low recall,

but if you design a model with high recall, that was we got most of the red fishes we

had a low precision we also caught a lot of blue fishes.

Can we combine precision and recall

to have an overall score which depicts how well our model is performing?

In comes my F1 score.

So F1 score is basically the harmonic mean of my precision and recall.

Now, you might wonder why are we taking harmonic mean and why not arithmetic mean or geometric mean?

Consider

a simple example of thousand samples and a confusion matrix, that looks like this.

I have 85 True positives, 890 False positive 15 false negatives and 10 true negatives.

My accuracy is around 1% my precision based on the formula here, which is TP upon TP plus

FP.

Which is 85 upon 85 plus 890 is 1% again but my recall if you notice carefully is 85 upon

85 plus 15 which is 85% so if now if I use arithmetic mean my F1 score would turn out

to be 43% but if you look at the confusion matrix, you can easily say that my model is

not performing well and 43% for such a model is not a realistic score

when we now look

at harmonic mean.It says 16% which looks more of a realistic score for this given model

rather than 43%

what F1 score does by taking into consideration my harmonic mean

is it punishes extreme values and our desire of a machine learning model for an F1 score

is the F1 score should be really high

In order to have a high F1 score, you need to

have both a high precision as well as a recall value.

Thank you so much for watching the video

hope it was information.