I have written about Quasi-experiment designs in my earlier articles. Difference-in-Differences (DiD) is a kind of quasi-experiment design, utilizing time-series data to analyze the change in outcome over time between a population in the treatment group vs a population in the control group. DiD is generally implemented in settings where randomization at an individual level is not possible. DiD also works on longitudinal data and hence we require pre-/post-treatment data for both sets of population. It is useful in removing biases from the post-treatment period comparisons between the treatment and the control group as well as removing biases that might be the result of some local phenomenon.
Some other assumptions of conducting a DiD experiment:
- The allocation of treatment or control to population groups is completely randomized to maintain the statistical significance of the causal effect of the treatment.
- The treatment and control groups have parallel trends in the pre-treatment period. This is important to analyze the change in behavior of the treatment group if treated as compared to the behavior of the treatment group if not treated.
- The assumption of SUTVA should be strictly followed between the population groups, i.e., there should be strictly no spillovers between population groups.
- The population groups remain stable (no flux) to maintain the repeatability of the experiment design.
Linear regression representation of DiD:
Where:
- Time = 1 means post-treatment time period and 0 means pre-treatment time period.
- Treatment = 1 for treatment group and 0 for control group.
- Covariates = all other independent variables and interaction terms.
Interpretation of coefficients (assuming no other covariates for simplicity, the answer remains the same regardless):
Using the values of coefficients above we can easily say that the actual change in behavior of treatment group because of the treatment is the coefficient of the interaction term in linear regression equation.
DiD is very useful in conducting experiments over a large population where we can divide them into groups with minimal spillovers. For example, they are used by TV companies to conduct experiments by dividing faraway but similar performing geographical areas into population groups and then randomly assigning treatment or control. DiD also removes many biases that are the nature of real-world experiments like seasonal effects. However, even with such benefits, DiD is hard to execute primarily because of the lack of baseline conversion data and the stability of defined population groups.
Some of my other article can be found here
You can connect me on linked in here
