Levels of variation and intraclass correlation - YouTube

One of the first things that we  analyzed when we start working  

with multiple data is on which level  each individual observation varies.

We also calculate intraclass  correlation to quantify these variances.

Let's take a look at an example to understand  

what levels of variation on an  individual variable level means.

We have here our profitability data or a company.  So we have five observations for a single company  

and average profitability for this company  is about fourteen percent and the individual  

observations vary randomly around the average  profitability because companies sometimes have  

good years sometimes they have bad years so the  performance is not always the same. So there's  

always some year to year variation. But that  doesn't really fully explain why a large data  

set of profitability figures would vary because  there can be also other levels of variation.

For example there can be company level variation.  These red mounds each present one company and  

they are all working within an industry and  this blue area here represents the variation  

of the performance of all companies within that  industry. So we can see that different companies  

vary. Their performance vary within company but  there are also variations between companies. So  

that this company here is consistently  less profitable than this company here.

So we have two levels. We have  the within company level and we  

have the between company level which  is also the within industry level.

We can also add more levels. There is no limit on  

how many levels we can do but  let's go for an industry level.

So we have these blue five different industries  and the industries are different in their  

profitability. Some industries are highly  profitable - others are not so and we can see that  

the individual variation of the data here is a  function of these three sources of variation - the  

between industry level the between company level  and the year-to-year variation within companies.

To understand our data and understand the  phenomenon that the data represent we typically  

need to decomposite variance to understood somehow  come up with percentages or some other statistics  

that quantify how much of the variation is here  and how much of the variation is here in our data.

If our data set is small we typically start with  a graphical analysis. So we can just upload the  

data. This is 25 observations. 5 observation for  each company for 5 companies within one industry.

We can see that there's some patterns for  example this company - there is not much  

variation in performance. This company is  less profitable than that company and so on.

This kind of analysis works well when you have  a small set of observations. If we have a large  

number of observations but still a fairly  manageable number of clusters let's say up  

to 30 companies or 30 industries or whatever  is our level to unit - we can use box plots.

Box plots are graphical presentations  of individual variables and we can do  

box plots by groups. The idea of a box plot  is that we first calculate for a variable  

regulate the median and the median gets this  thick line here. That marks the median. Then  

we calculate the first quartile and the  third quartile of the data. So quartile  

means that below this line lies 25 percent  of our observations and above the line 75  

percent. Median is half-and-half  and third quartile is 75% and 25%.

We draw a box between the first quartile and  the third quartile and half of our data is  

within this box. Then we have these whiskers  that indicate the minimum and maximum and  

sometimes we also have outliers that the  box plot algorithm identifies as circles.

So why is this box plot presentation useful  and how can we analyze the box plots?

We can first of all start to understand the  between and within variance by looking at the  

box plots. We can compare these medians or we  can do box plot with means and we can check how  

much variation there is between these means  or medians and that is our between variation.

We can also take a look at how high the  boxes are and that quantifies the within  

variance and comparing these two dimensions  tell us if the variation in this variable  

is more due to the differences between firms  or is it just random variation or some other  

variation within firms. So is it a within firm  or between variation that explains the data.

We can quantify the level of variation  between two levels also numerically by  

calculating the within variance and between  variance. This is our data and we start by  

calculating group means. So we take each  of these companies and we calculate a mean  

of this. So these are the group means or  cluster means for these five firms. And we  

check how much these means vary. The variation  is quantified here with this statistic and then  

we calculate how much these individual  observations vary from the group mean.

In practice we do group mean centering. So we  take each of these observations. We subtract  

the group mean and that gives us the group mean  standard values. Then we calculate how much the  

group mean standard data varies and this is our  between variation. This is our within variation  

and this is our total variation which is the sum  of the between variation and the within variation.

So the variation of variance is  a statistic that depends on the  

scale. It would be useful to have a  scale-free way to explain on which  

level the data varies and this is where  the intraclass correlation comes to play.

So intraclass correlation is simply calculated  as variance between groups divided by the total  

variance. So it answers the question how much  of the variation in the data is attributed the  

groups and how much is attributed to  the variation within the groups. This  

is called ICC one for reason that there are  many other kinds of intraclass correlations.

So intraclass correlation generally refers  to correlation between observations and  

because there are many this is called the  ICC one. There are like a few others but  

this is the most important one that you need to  understand when you work with multi-level data.

Other inter cross correlations are  mostly about reliability of multiple  

raters but as you see one is - this simple  equation that simply quantifies variation.

When ICC 1 is 0 then that indicates  that there is no variance between  

groups. So the box plots are all on the  same level here. There are no difference  

between means and this case medians  are close as well and all variation  

is simply because there is variation  between these within these groups.

Then when intraclass correlation is 1 then  that means there is no variance within  

clusters at all. All observations equal the  level to means. So this firm's profitability  

is always here. This firm's always here and  so on. So there's no within unit variation.

Why do people calculate intraclass correlation  and how it's typically reported? The role of  

intraclass correlation - the first role is to make  a decision whether something needs to be done for  

the clustering. If all the observations within  cluster have the same then you can just pick  

one observation for each cluster and use those  in regression analysis and it doesn't really  

matter that you have the remaining observations  because they don't provide you any more data.

If ICC 1 is 0 then there is no clustering in  that variable and if your all ICC ones are very  

low then there is no meaningful clustering in  your data and it's possibly safe to go without  

a multi-level modeling. There exceptions to  that rule but generally when ICC 1 is close to  

0 or when ICC 1 is close to 1 then a multi-level  modeling may not be needed but if it's somewhere  

between like it's 50% then you typically need to  take levels into account in your analysis somehow.

Let's take a look at the example of how ICC  1 has been reported in published research. So  

this comes from Hausknecht's paper and this is  a good example because they first explain what  

the statistic is. So quite often people just  report a statistic or report a number without  

explaining what ICC one is. And this study  provides concise description. ICC 1 values  

can be interpreted as the total amount of  variance in the dependent variable that  

is attributable to between unit rather  than within uni differences over time.

So that explains what the statistics  interpretation is and also if their  

values are are high then regression  analysis could be inappropriate and  

then you would have to do something else  or for example use cluster over standard  

errors or multi-level modeling. And then  they go on and they explain what is the  

actual statistic. Abstention point 76 and  then they explain what the statistic means.

So giving this short introduction to your  statistics is very useful for your readers  

because your readers may not be experts in using  multi-level data so make it easier for them.