Surveys and Samples - YouTube

What we have here is six examples of possible surveys

that might be conducted.

What we want to do is critique them, see whether they're

actually doing things that will give meaningful results,

or results that you can actually make some

judgment based on.

So let's read the directions.

It says, comment on the way the following samples have

been chosen.

For the unsatisfactory cases, suggest a way to improve the

sample choice.

All right.

So A, you want to find whether wealthier people have more

nutritious diets by interviewing people coming out

of a five-star restaurant.

So this is skewed on two directions.

One, if you're only interviewing people out of a

five-star restaurant, you're going to have a

disproportionate number of wealthy people.

And even the non-wealthy people who are eating at the

five-star restaurant, they feel like they have enough

money to eat there, so they're probably not that not-wealthy.

They're probably pretty close to the threshold of wealthy.

The other skew is that you're interviewing people at a

restaurant, so you're disproportionately probably

going to get people who eat out, as opposed to people who

cook for themselves.

And depending on whether this is a really healthy

restaurant, maybe some type of organic, vegan, who knows

what, restaurant, where you're going to get a

disproportionate number of healthy people.

Or maybe it's a really unhealthy restaurant.

Maybe it's they put a lot of butter and sugar in the food.

Who knows what it's all about?

But this seems skewed on two dimensions.

The best way to do it is, survey some proportion of

wealthy people in an environment where it isn't

being skewed by the type of food they eat.

So you, once again, you wouldn't want to survey people

at the organic grocery, because once again, you'd be

skewing for people who maybe eat a little healthier.

You'd want to do it maybe at some type of wealthy type of

event, or something that's not food-oriented.

I'd have to think more about what that might be.

And you'd also want to survey people who are not wealthy, so

at some other type of event where most of the population

doesn't fall under the wealthy category.

Or probably the best thing would be sampling people at an

event that doesn't discriminate on wealth, and

just surveying people and just ask them, in a broad bucket,

do you make more than x?

Whatever your definition of wealthy is.

Or is your wealth, or is your income less than x?

And then ask people what do they eat on a daily basis,

something like that might work.

So in general, I'm not a fan, not a fan.

I don't think this passes mustards.

It's biased on really two dimensions.

B, you want to find out if there is a pedestrian crossing

needed at a certain intersection by interviewing

people walking by that intersection.

So this is better than the first survey, but it still

seems a little biased.

And it depends how you define needed.

Is needed if the people walking around the pedestrian

crossing think it's needed, then this is OK.

But if you want to objectively find out whether people in

your community think it's needed, you want to have a

broader sample.

So you might want to also include drivers.

Because obviously, the pedestrian crossing is going

to affect them.

You want to interview taxpayers in general.

And obviously a lot of people might be all of the above.

And you also might be missing pedestrians who would be

walking by that intersection if there was a pedestrian

crossing, but they're avoiding it, and so those are the

people who would want it the most, but they're not even

going to be there because they don't think that it's worth

walking around there because you can't cross the street.

So you're missing a whole sample of people.

The best way I would do this is I would look within a,

maybe a three block radius, or maybe a six block radius, of

all of the homes and offices, and I would randomly survey a

sample of those people.

And I think that would be a better indication of whether

the pedestrian crossing is needed.

And actually, even a better way would be do

it without a survey.

You'd actually want to look at patterns of people walking and

all of that, and compared to other things

that might be needed.

But this one definitely doesn't work for me.

Now C, you want to find out if women talk more than men by

interviewing an equal number of men and women.

That seems reasonable.

I guess if your survey, if you do it in a non-biased

environment-- if you don't do it at, like, a business school

or a law school where people might talk disproportionately,

or it might be skewed towards women or men more.

I don't know which way it would be skewed.

But as long as you do it in a neutral place, and I guess

their survey is, maybe they'll ask some arbitrary question.

And they'll just time, or they'll count how many words

the women talk versus the number of words the men talk.

That seems like a reasonable way to see if women

talk more than men.

And once again, as long as you're not talking to some

skewed sample, or biased sample, this

seems pretty good.

And it's good if it's an equal number.

There's an equal number of men and women in the world.

This seems to be-- I'll give it a smiley face right here.

D, you want to find whether students in your school get

too much homework by interviewing a stratified

sample of students from each grade level.

Well the first thing I want to do is, what is

a stratified sample?

And all that means is, you're going to sample people in the

same proportion that they exist in the population.

So let's say that your population-- your school--

had, say they had 100 ninth graders.

It has 200 tenth graders.

300 eleventh graders.

And 400 twelfth graders.

Actually, the school I attended actually, it went in

the reverse order.

I think we had like 800 ninth graders and 400 twelfth

graders since so many people dropped out.

But anyway, we don't have to be realistic here.

So you had a total school population

of about 1,000 students.

So if you want to do a stratified sample, let's say

you wanted to have a sample size of,

let's say, 50 students.

So you wanted to sample 50 students, and you wanted it to

be a stratified sample.

What that means is, you had 100 ninth graders out of 1,000

in your school.

So 10% of your school's population is ninth graders.

So in your stratified sample, 10% also

need to be ninth graders.

So 10% of 5-- sorry-- 10% of 50 is 5.

So you'd survey 5 ninth graders.

In both situations, this is 10% of the population, it's

going to be 10% of your sample.

Likewise, you had 20% were tenth graders in the

population, so 20% of your sample.

Or 20% times 50 is 10 tenth graders should be surveyed.

And then 30% of your school is eleventh graders, so 30% of

your survey.

So that should be 15 eleventh graders.

And then finally 40% were twelfth graders.

So in this situation, 40% of your survey, which is 20,

should be twelfth graders.

So that's all-- that's what a stratified sample means.

Now let's talk about whether this is

actually a good survey.

You want to find out whether students in your school get

too much homework, and you're interviewing

students in your school.

The one thing you definitely will find out is you will find

out whether students in your school think-- think-- they

think they get too much homework by interviewing a

stratified sample.

That would be true.

That's what you would find out.

Now, whether you could say that in a broader sense

they're getting too much homework, you might want to

include a larger audience, especially the other

stakeholders in the whole homework doing process,

especially parents and teachers.

And actually a better way to do it would be, probably to

compare the students in your school to a very similar

school that has a different amount of homework, and maybe

see the results on standardized tests.

And you want to control.

You want to make sure that there isn't a huge difference

between the income levels, or the education levels of the

kids' parents, and all that.

I mean, it becomes much more involved.

But this one still seems a little bit narrow, and all

you're going to find out is whether students think they

get too much homework.

And I can almost guarantee you, they will think that they

get too much homework, and so this survey might be a little

bit of a waste of time.

So I will put a frowny face.

E, you want to find out whether there should be more

public buses running during rush hour by interviewing

people getting off the bus.

So once again, what you're going to find out is whether

people who ride, people getting off the bus think

there should be more buses.

So I won't write it here, but you're not going to find out

objectively whether there should be more buses.

Frankly, the best way to evaluate that, I think, is to

just see how full, on average, the buses are.

I would actually poll the actual buses and see how full

they are during rush hour.

If 90% of them are 100% full, or people can't get on a bus

because they're full and that's happening frequently,

and their total wait time is just getting ridiculous,

that's probably a better assessment than

just surveying people.

But even beyond that, this is skewed on several levels.

One, you are surveying only the people riding the bus.

You're not surveying the other people in the community who

might have an opinion on this: drivers, taxpayers who would

have to pay for more buses.

And then even within the bus-riding population, you

have to think about when they're getting off the bus.

If you're getting people off the bus during rush hour, I

can almost guarantee you that they're going to say, yeah,

we'd like more buses because then they would have less wait

time, and the buses would be less crowded, and they

wouldn't have to sit next to anybody.

So even if it's not needed, they're going to say it is.

But if you interview people maybe getting off the bus

during lunchtime, who don't use the bus during rush hour,

they might say no, we don't need more buses at rush hour.

We need more at lunchtime.

So you're going to get different answers depending on

when the people are getting off the bus, and obviously,

you're going to get very different answers from people

not riding the bus.

So if you want a broader audience, if you're kind of

the city councilman trying to figure out what to do, I, once

again, like the crosswalk, I would survey a random sample

of people within three miles of the bus route, or

something like that.

I think that might be a better result.

F, F-- I'll switch colors.

You want to find out whether children should be allowed to

listen to music while doing their homework by interviewing

a stratified sample of male and female

students in your school.

Once again, stratified sample just means that, OK, if my

school is 60% male, then 60% of my sample will be male.

And that doesn't really seem relevant here.

F seems very similar to the, do students think they get too

much homework?

Just as almost every student is going to say they get too

much homework, almost every student should say they should

be allowed to listen to music if they want to.

And that's not going to be-- that's not really going to

answer the question of whether children should be allowed.

If you want a broader sample, you might-- just like the

situation with the homework-- include

parents and include teachers.

But really, if you wanted to answer this question, the best

way would actually be to look at, to survey students who

listen to music while doing their homework, and students

who don't listen to music while doing their homework,

and see their relative grade performance, and that might

give you a better answer.

So once again, I think this one's a little weak.

And the bus one was also a little weak.

So the only good one here, if it's done right, is C.

The other ones, they all needed a little bit of work to

actually be surveys that would give meaningful information.