Event-Study Plots: Suggestions - YouTube

welcome back to our series on  visualization identification and estimation  

in the linear panel events study design in this video we're going to think about  

ways to make event study plots more informative in the previous video in this series on "event  

study plots: basics" jorge introduced you  to the estimating equation that underlies  

our plots which is repeated over here jorge further explained the interpretation  

of this equation and its relationship  to a standard distributed lag model 

if this equation looks new to you i  encourage you to take a look at the video  

"event study plots: basics" before proceeding to quickly recap y is the outcome variable of  

interest here z denotes the policy variable and we  will continue to refer to k as event time and the  

vector delta as the event time path of the outcome what that means is that we can plot delta against  

k and what we get is our typical event study plot  and this typical plot may look something like this 

so today i'm going to talk about a number  of suggestions for the construction of  

these very plots and our hope is that these  make your event study plots more informative 

many of these are common  practice already some less so  

the first thing i want to talk about is that  by construction the z terms on the previous  

estimating equation are going to be collinear  so what that means is that not all deltas are  

going to be identified and we're going to  have to impose some sort of normalization 

so our first suggestion which is already somewhat  common practice is to normalize the coefficient  

immediately preceding any anticipatory effect  and often that means the coefficient at minus one  

so here the treatment happens at event time  zero so this is when the event happens and we  

normalize the coefficient at minus one immediately  preceding the event we think that the first lead  

is a good default because this enables convenient  tests of hypothesis relative to event time minus  

one throughout the next few slides i will use the  two examples depicted here as running examples  

you can think of them as two separate hypothetical  data sets that you might encounter in practice 

on the left hand side over here we have an event  study path that is generally decreasing in event  

time but on the right hand side we have an  identical pattern pre-event so these point  

estimates are exactly the same as on the  left-hand side then a jump at event time  

before this negative trend sort of continues now if i look at these plots i might like to  

know something about the economic as well as  the statistical significance of my estimates 

these plots make that hard to discern partly  because it doesn't convey anything about the  

overall level of the dependent variable and so  what i mean by that is that the policy effect  

might be very small but precisely estimated  here thus it might be statistically significant  

but not economically meaningful this brings us to  our second recommendation which is to add a label  

to your plot that indicates the average value  of the outcome corresponding to the normalized  

coefficient with staggered adoption and no  anticipatory effects this would simply be  

the average value of the outcome of interest  in the period immediately preceding the event 

for example here knowing that the outcome takes  on a value of around 42 as indicated by the  

labels up here might be helpful in deciding  the economic significance of the effect

now our next suggestion addresses the fact that  pointwise confidence intervals are appropriate  

only for testing pre-specified pairwise hypotheses  so for example looking at the two figures up here  

and ex post concluding that there's a significant  pre-trend because the coefficient at minus five  

right here is significantly different from  zero is generally invalid because likely this  

is not the type of pre-specified hypothesis we  were interested in when we ran this event study  

and so more generally since we're depicting  the entire event time path on our event plot  

we might want to also visualize what type of  event time paths are consistent with the data here  

so considering the entire path rather than  a single pre-specified component of it 

in order to do that we suggest to add  uniform confidence bands to your plots. 

in particular we found sup t-bands both  computationally and visually convenient  

now i'm not going to go into too  much detail about these today  

but note that our package xtevent has an option  to simply add them to your plots automatically  

and intuitively the idea is that these  uniform bands give us a way to think about the  

plausibility of the entire event time path so for  instance going back to the specific examples here  

based on the pointwise confidence intervals  you might have concluded that the lead at -5  

again take this coefficient  over here might be significant  

now the correct confidence bands to use here are  probably the uniform ones and they actually cover  

zero in all five pre-periods as indicated by the  sort of outer edges of those lines of the bands

so for example consider an  event time path that is zero  

everywhere leading up to the event  so it would look something like this 

the uniform bands sort of suggest  that you can't actually reject  

that that is sort of a plausible path that  the outcome might take in population here  

so it's not clear this part  is rejected by the data

now there are a few hypothesis tests  that are generally of particular interest  

one is this test for pre-trends that i alluded to  already and relating this back to the estimating  

model again the model if true implies that  leads further than g leads should all be zero  

so our next suggestion is to include  the p-value for this test on the plot  

so of course so this was done over here  and corresponds to this .22 over here and  

corresponds to the p-value that all of the  coefficients pre-event are equal to zero 

of course similarly the model also implies that  lags further out than m lags should be zero  

so we also suggest to include the p-value  for a similar test of where the dynamics  

have leveled off at event time m and this  is indicated by the second p-value down here 

now one of the reasons i think economists  like these type of events study plots  

is that we may have intuitions about what they  look like when the policy is having an effect  

versus what they look like when there  is no policy effect but there's some  

sort of confound going on masquerading as a  statistically significant effect of the policy  

in some form of static summary statistic so for example you might see a significant  

static effect in the table for this figure here  in a specification with a simple post-event  

dummy right because you get something  negative and statistically significant  

but the fact that you can basically fit a  straight line through all of these confidence  

intervals right here likely gives you some pause the uniform confidence intervals are already  

helpful for that but we've developed this idea  a little bit further with our next suggestion

so for this suggestion we suggest to explicitly  show something about what kinds of confounds  

are consistent with the estimated event  time path of the outcome in your plots

in particular we suggest that the least wiggly  confound that is consistent with the null of  

no treatment effect where consistent here  means that we can't reject the confound  

path through a wald test so for instance if we  look at the two concrete examples here again  

on the left hand side we cannot reject  that the true event time path follows this  

essentially straight line from beginning to  end of the figure so therefore we might think  

that this plot doesn't show very strong  evidence of a causal effect of the policy

on the other hand looking at this plot on the  right hand side here it seems much harder to chalk  

up this jump at event time we see to confounding  alone - the smoothest path we can find that can  

fit the data here and that's not rejected by a  wald test still looks pretty wiggly so you have  

to have these kind of compound dynamics going on  to explain this type of observed event type path

of course these type of judgments are  context dependent because in different  

settings different confounds and different  dynamic effects the policy might be plausible  

but either way i think that in a lot of situations  including this type of curve can help your readers  

form useful intuitions about what sort  of confounds are consistent with the data

so to give you some intuition our procedure  essentially works as follows we first try to fit  

a straight line to the estimated coefficients  so roughly you can think of this thing on the  

left here if such a straight line exists where  an f-test cannot reject that in population the  

event time path follows that line we're done  and we add that straight line to the figure

if no such straight line exists  we try to fit a quadratic term  

if that's feasible we use the quadratic term with  the lowest curvature and add that to the figure  

if it's not feasible we add a cubic term and so on

again this is implemented in xtevent  in stata and in fact before i conclude  

this video now let me quickly give you a quick  sense of of how our accompanying stata package  

facilitates the adoption of the suggestions  we just discussed including the smoothest line

so i have preloaded an example data set here  that use the same notation for all variables that  

we've seen in the slides and so to run an event  study here i simply type xtevent i declare the  

outcome variable of interest i declare  the panel variable which is denoted by i  

similar i declare t as the time variable  the policy variable is indicated by z

i have to define the window length

and that runs my my specification

now to look at the corresponding event study plots  i simply type xteventplot and that pulls up this  

figure right here and you note that it includes  uniform bands includes the label indicating the  

value of the outcome variable and it includes the  p-values we discussed earlier below the figure

now we can also add the confound dynamics  

so the smoothest line that we  just discussed to this figure

to do so you simply write xteventplot  and you add smoothest path  

and we're going to do so using a line and you  get this figure over here and so in this case  

what this figure tells you is that you need a very  wiggly confound to explain the event time path of  

the outcome if there was no treatment effect so this is the kind of confound that you would  

need to explain the observed event time path under  the null of no treatment effect and with that  

i'd like to thank you for watching this video and  i hope you find this helpful for your research