Calculating Power and the Probability of a Type II Error (A One-Tailed Example) - YouTube

Let's look at an example of calculating power

and the probability of a type II error, which we often called beta.

This example is done in the setting of a one-sample Z test on a mean

but similar logic holds for other types of tests.

Suppose we are about to randomly sample 36 values from a normally distributed population,

where the population standard deviation sigma is known to be 21,

but the population mean mu is unknown.

And we wish to test the null hypothesis that the population mean is 50

against this one-sided alternative hypothesis

and we feel that a significance level of 0.09 is appropriate for our test.

Since we are sampling from a normally distributed population,

and the population standard deviation sigma is known

then the appropriate test statistic is this Z test statistic.

If the null hypothesis is true, this Z test statistic will have the standard normal distribution.

And the first question, is for what values of Z will we reject the null hypothesis?

Here I've plotted out the standard normal curve.

Our alpha level is 0.09 and our alternative hypothesis is that mu is less than 50,

so we're going to put the entire alpha level in the left side of the distribution.

If we go to software or the standard normal table

we can find that the Z value with an area of 0.09 to the left is, to 2 decimal places, -1.34.

And so we are going to reject the null hypothesis

If the Z value that we get in our sample is less than or equal to -1.34.

Now we're going to reexpress the rejection region in terms of the sample mean

because that's going to help with our power calculations later on.

So, for what values of X bar will we reject the null hypothesis?

Here's our information from before,

and we said that we are going to reject the null hypothesis

if the observed value of Z is less than or equal to -1.34.

Well this this is our Z test statistic here, and we can isolate X bar,

just multiply by sigma over the square root of n and add mu_0.

and we get that X bar is equal to mu_0 plus sigma over the square root of n times Z.

Well in this case

that's going to be 50 plus 21 over the square root of 36 times -1.34,

and this works out to 45.31.

So we had this rejection region expressed in terms of Z before,

but now we can express it in terms of X bar.

We're going to reject the null hypothesis if X bar is less than or equal to 45.31.

Visually that looks like this.

I've plotted out the sampling distribution of X bar if the null hypothesis is in fact true.

And we are going to reject the null hypothesis

if the value of our sample mean is less than or equal to 45.31.

And this area here is the given alpha level of 0.09.

And that is the probability that we reject the null hypothesis if it is in fact true.

Or in other words, the probability of committing a type I error when the null hypothesis is actually true.

If the true value of the population mean is 43, what is the probability we commit a type II error?

If you recall, a type II error is not rejecting the null hypothesis when it is false.

Our null hypothesis here is that mu is 50,

and our null hypothesis is in fact false

because we're saying the true mean is 43.

So the probability of a type II error in this setting

is the probability that we do not reject the null hypothesis if mu is actually 43.

If you recall, we reject the null hypothesis if X bar is less than or equal to 45.31,

and so we're not going to reject the null hypothesis when it's greater than that

and so the probability of a type II error is going to be

the probability that X bar takes on a value bigger than 45.31,

given the true value of mu is actually 43.

Let's see what that looks like visually.

In white I have the sampling distribution of X bar if the hypothesized value is correct.

And in blue I have the sampling distribution of X bar if mu is actually 43

which we're assuming it to be for this question.

So this distribution in blue is centered here at 43.

We are rejecting the null hypothesis

if the sample mean is less than or equal to 45.31

and we're going to not reject the null hypothesis if it's greater than 45.31,

so the probability of a type II error in this spot

is going to be the area to the right of this value, given mu is actually 43.

Let's shade that area in.

I'm shading that area in red here because it's an error.

It is the probability of committing a type II error if the true value of mu is actually 43 ,

the probability of not rejecting the null hypothesis when it is wrong.

To find that area, we want to find the probability that X bar takes on a value bigger than 45.31,

given the true value of mu is 43.

X bar in this setting is a normally distributed random variable

and we're going to standardize this in the usual way.

And we standardize this by saying that

Z is equal to X bar minus the true value of mu over sigma over the square root of n.

So to do this probability calculation over here we're simply going to standardize it.

We're going to say that this is equal to

the probability that Z takes on a value that's bigger

than 45.31 minus the true value of mu, which is 43,

divided by sigma, which you may recall was 21, that was given earlier.

And our sample size n was 36.

And so this is going to be the probability

the random variables Z takes on a value that's bigger than 0.66.

And if we go to the standard normal curve,

we can see that the area out to the right of 0.66 is approximately 0.255.

And so our probability of a type II error in this spot is approximately 0.255.

And we call that beta, the probability of a type II error.

The power of a test is the probability of rejecting the null hypothesis when it is false.

And that's simply going to be 1-beta.

And in this setting that's 1-0.255, or 0.745.

Let's take a look at another example of a power calculation.

Suppose this time mu is actually 40,

so the null hypothesis is still wrong but the population mean is now 40.

What is the power of the test?

What is the probability of rejecting the null hypothesis,

given that it is false, as it is in this scenario.

So here our power is the probability of rejecting the null hypothesis given the true value of mu is 40,

and recall that we're rejecting the null hypothesis

when X bar is less than or equal to 45.31

so we could express power in terms of the probability of a type 2 error, but we don't have to.

Its easier typically to just leave it in this fashion.

Power is the probability of rejecting the null hypothesis

and we're rejecting the null hypothesis when X bar is less than or equal to 45.31,

given the true value of mu is actually 40.

Let's see what that looks like visually.

Here again I've plotted out the sampling distribution of X bar if the null hypothesis is true,

and the sampling distribution of X bar when mu is actually 40, which is our current situation.

So the sampling distribution of X bar now is centered over here at 40.

We are rejecting the null hypothesis if we get a sample mean that's less than or equal to 45.31,

so let's shade that area in.

I've shaded this area in green this time because this is the power,

and power is the probability of rejecting the null hypothesis when it's false,

and that's a good thing.

And so here power is going to be

the probability that we get a sample mean that's less than or equal to 45.31,

given the true value of mu is actually 40.

And we're going to standardize this again in the usual way,

and say that Z is equal to X bar minus the true mean

over sigma over the square root of n

and come down here and say this is going to be equal to the probability

that Z takes on a value less than or equal to 45.31 minus the true mean, 40,

over sigma which was 21

and the square root of n and n was 36.

This is going to be equal to the probability

that a random variables Z takes on a value that's less than or equal to 1.517.

And if we go to our standard normal curve

we can find that the area out to the left of 1.517 is approximately 0.935,

and that is the power of the test in this situation.

Here we did all of the power calculations

under the alternative hypothesis that mu is actually less than the hypothesized value.

The same logical arguments hold for other alternatives

like it's greater than the hypothesized value or not equal to, but the required areas will change.

So when faced with one of these power calculations,

it's important to think through it and make sure you're coming up with the correct areas.