Reversion to the Mean - YouTube

It’s a cold winter morning. You wake up fresh and  rested. You get ready and go to the nearest cafe  

to start the day with a cup of coffee. You  order your favorite: an iced caramel latte.  

As you wait for your coffee, you make small  talk with the stranger next to you in line.  

You two quickly hit it off and the  conversation just oozes chemistry.  

Oh, and the coffee is here too. Right on queue.  You just know it’s going to be good as always, and  

it really is. Just the right amount of milk, not  too sweet. The caramel is there too, but it isn't  

overpowering. “How awesome that the barista got  it right this time?,” you think to yourself. The  

conversation continues, and in the back of your  mind, you just know: “Today can’t get any better.”

…and then slowly the day comes to an end. You wake  up the next morning still buzzing from the energy  

of the day before, your mind all excited to try  and recreate that perfect experience, that perfect  

conversation, that perfect cup of coffee. You  get ready just the same and head out to the cafe,  

only to realize your favorite spot has been taken.  Oh, and there’s a staff shortage so the queue is  

extra long. Everyone’s kinda doing their own  thing, kinda cranky. Even the coffee today  

is a bit too sweet. There was no refreshing aroma  this time, and certainly no kind stranger willing  

to make eye-contact, let alone small talk.  The whole experience is just so… mediocre.

And really, that’s the story of most of our days.  What happened? What changed? Is it something you  

did? Maybe the way you did your hair in the  morning? Maybe it’s the faulty coffee machine?  

You try coming up with reasons, but you can’t  seem to figure it out. What prevented you  

from recreating that “perfect” experience?  Statistics might have an answer for you.

You see, whenever we have a random variable  that could be almost anything, a phenomenon  

called “reversion to the mean” tells us that an  “extreme” instance of that random variable will  

be followed up by a less extreme instance  if that measurement were to be taken again.

In the cafe example I just mentioned, how  well your experience in that cafe goes is  

a random variable, or rather a bunch of  random variables. And having the ‘perfect’  

experience entails that all of the random  factors that could have gone your way did.  

It would be an “extreme” instance in that  mornings are rarely as perfect as this one.  

Reversion to the mean suggests that if  this experiment were to be conducted again,  

if you were to go to the cafe again, your  experience would tend more towards the average,  

leading to a mediocre experience. This is because  all those factors being lined up the right way the  

first time was just a result of very good luck,  and it’s not likely to reoccur a second time.

Now, when we’re having average days, or  even bad ones, it’s normal to think that  

there is some negative force of the universe  causing things to be this way. But, in truth,  

there is no such cryptic reason as to why your  morning coffee wasn’t so great. In fact, reversion  

to the mean is a general statistical tendency. It  happens to everyone and everything, all the time.  

Whenever you have a variable whose behaviour  is accounted for partly by randomness,  

you are bound to see reversion to the mean.  If your variable doesn’t depend on randomness,  

well, there you have it, you will have the same  result over and over again - there won’t be any  

variance to begin with. But reality is rarely  like that. In fact, reality is never like that.  

It’s riddled with randomness and factors  that are too complicated for us to predict.  

As a result, we will always see extreme  events from time to time in both directions;  

but over the long run, they seem to cancel each  other out, and the outcomes tend towards a mean.

Now given the universal nature of this phenomenon,  one would expect people to generally have an  

intuition for it. But you’ll be surprised as to  how often reversion to the mean can be overlooked.  

For example, one of the more famous examples  about reversion to the mean is from Daniel  

Kahneman’s book, “Thinking Fast and Slow” where  he details a meeting with Israeli fighter pilots.

Daniel was arguing that positive feedback  was more beneficial to cadets in terms of  

their performance in-flight. However, some of the  commanding officers that were present disagreed.  

They said that Daniel’s rosy take  on feedback wasn’t realistic,  

and that based on their experiences, cadets who  underperformed reacted best to harsh criticism  

and those that performed well ended up  performing worse if they were praised.  

So, clearly, positive feedback  wasn’t good for them, right?

Well, not quite.

You see, someone who performed really well on a  maneuver probably had some luck on their side. Of  

course, these are trained professionals and their  abilities have a part to play in what they do,  

but so does luck. And so, on subsequent attempts,  despite the same ability, they had simply run out  

of luck and ended up performing worse than  before. Their low-performing colleagues,  

however, probably had bad luck the first  time, and lightning never strikes twice.  

Over the next few attempts, they had better luck.  Point is, in both instances, the pilots regressed  

towards the mean. For the pilots that did well the  first time, this meant a decline in performance,  

and for the pilots that did poorly the  first time, this meant an improvement.  

This, of course, implied that  the commanding officers were,  

in truth, overemphasizing the  contribution of their intervention.  

But how do we know that for sure? How do we  know what interventions work and which don’t?

Answering that question becomes particularly  important once you realize the consequences of  

reversion to the mean, especially in healthcare.  How do we know whether any medical intervention  

is effective at all, given that patients with  most illnesses tend to feel better with time?  

How do we know they are responding to treatment  and not simply regressing to the mean?  

That’s where control groups come in. And really,  that’s where the concept of Placebo’s comes in,  

which is a topic I have  covered on this channel before.

Once you realize the effect of reversion to the  mean, it becomes imperative to have control groups  

where the intervention being tested won’t be used  so as to distinguish its effect from that of the  

reversion. If a medicine works better than  reversion to the mean, we can be reasonably  

certain about its effect. This has become a  crucial part of the medical process and the  

greater scientific process, as, again, reversion  to the mean is everywhere and you have to isolate  

its impact in your conclusions. The consequences  of not doing so can be dangerous. It can lead  

to unnecessary suffering from ineffective  interventions, and could even cost lives.

Reversion to the mean is also seen in the  electoral process. More extreme candidates tend to  

be followed by less extreme candidates. Then there  are financial markets, where prices can stall or  

skyrocket, but generally stay in the ballpark  of some market average. In fact, moving averages  

are used by traders around the world every  day. Of course, most of us are naive to this,  

despite its regularity, and routinely extrapolate  the immediate past into the indefinite future.

Reversion to the mean also applies to individual  circumstances. If you did really well on a test,  

chances are, you won’t do so well the next time.  Exemplary performance is rarely sustainable.  

At the same time, if you did poorly the  first time around, you are likely to do  

better the second time around. As I mentioned  with the pilots, your ability definitely has  

a role to play in what you do, but the overall  statistical tendency of ebb and flow remains.

You can see it in sports too. Athletes  who do really well on their rookie season,  

rarely live up to the expectations  in the subsequent seasons.  

In fact, knowingly or unknowingly, there are many  references to this tendency in the form of the  

“Sports Illustrated jinx” or the “commentator’s  curse.” Of course, the commentator isn’t casting  

some dark spell over the players. It’s just that  given the spectacular nature of what a player  

did to earn the adoration, they are simply  not likely to recreate it again. Of course,  

there are exceptions to all rules, and even  with this one. Some people were just too good.

Michael Jordan, for example, rode the covers  of Sports Illustrated over 50 times with no  

significant decline in his performance. But even  this elusive athlete wasn’t entirely free from the  

effects of reversion to the mean. Michael Jordan  was an exceptional basketball player, needless to  

say. And his talents were justifiably extreme.  If reversion toward the mean is indeed correct,  

it would predict that his sons, despite  inheriting some of that talent from their dad,  

were not likely to soar to the heights their  father did. And that’s exactly what happened.  

Even though they had increased attention and  the privilege that came with being Michael  

Jordan’s sons, they never really made it.  They were successful college athletes,  

sure, but they were no NBA players, let alone  one of the greatest athletes to have ever been.

Chances are, they worked really hard. But,  genetics is just a tad too random, and that makes  

the effects of reversion to the mean that much  more pronounced. And that might make Jordan a bit  

sad. But, in some sense, he should thank genetics  for indeed being so random. Afterall, his dad was  

5’9” and his mom was 5’5”. Michael was 6’6”, and  without that seemingly “lucky” boost in height,  

who knows, maybe he would never have become  the greatest basketball player of all time.

So, wait, you’re telling me reality tends towards  mediocrity? And there’s nothing I could do about  

it? Now, I don’t know about you, but I find that  hard to accept. I thought there’s gonna be Hans  

Zimmer playing in the background as my life plays  out. I thought there were gonna be cool cut scenes  

here and there where the wind hits just right so  that it ruffles my hair and makes me look cool.

To me, the concept of reversion to the mean seems  almost boring, and a wee bit disheartening if I’m  

being honest. Because it takes away from the  notion that hard-work is what gets you success,  

not chance events. It further casts a shadow  on the already poorly lit facade of free will.  

If we are all indeed ebbing and  flowing between spectacular and awful,  

only to end up in the mediocre, what control  do we really have? Besides, it seems as though  

statistics is almost forcing us to be mediocre,  to be less than, and nobody likes being mediocre.

As Lieutenant Commander Phillip F Queeg,  captain of the USS Caine in World War II said,  

“Aboard my ship, excellent performance is  standard, standard performance is substandard,  

and substandard performance is not permitted to  exist.” I think a lot of you feel the same way.  

There is something fundamentally nauseating  about “mediocrity.” It’s so… well… mediocre.  

Of course “average” should by  definition be a neutral word,  

but it’s hard to disassociate the  negative connotation it seems to carry.

That is until you fall sick. Or until you lose a  loved one, or don’t have access to food, water,  

or shelter. Then, well, being average is not too  bad. Then, being normally healthy seems priceless.  

Ask someone who has an empty seat  at the dining table - a “normal”  

family is all they could ask for. Being  normal doesn’t seem so bad now does it?  

And yes, while we can’t control the statistical  tendencies that cause reversion to the mean,  

we can, nonetheless, gradually push the  mean closer to where we want it to be.  

With consistent but ever so slight strides, and  with hard work, we can all become better averages.

This is not a denial of how unpredictable life is,  and how little control we have over it sometimes.  

Rather, it is taking control of whatever little we  can control and not getting too preoccupied with  

what we can’t. It also serves as a reminder that  we may not be as responsible for our success as we  

might like to take credit for. Maybe the person  next to me deserved it just as much, but while  

the random variable that is my life was basking  in its fleeting luck, that of someone else's was  

regressing to its unspectacular average. They  were just out of luck. And as far as free will  

is concerned, well, maybe, the height of free  will is to simply be aware of its limitations.

But enough talking, life is finite. You haven’t  got all day, and uh, your coffee’s getting cold.