Effect Dynamics Over Time

Classic DID models assume an immediate treatment effect, but real-world interventions often exhibit delayed impact patterns. When examining treatment outcome trajectories, the difference between treated and control groups rarely materializes instantaneously. Instead, interventions frequently require time to reach their full potential. This phenomenon is prevalent in marketing campaigns and regional policy implementations alike. Failing to account for this gradual effect buildup can lead to underestimating the true treatment impact by including periods where the effect has not yet matured.

One practical solution involves estimating time-varying Average Treatment Effects on the Treated (ATT). Rather than manually creating dummy variables for each post-treatment period, a more straightforward approach iterates through each time period and runs a separate DID estimation, treating only that specific period as the post-treatment window.

Consider a function that accepts a dataset and a specific date, then performs DID analysis treating that date as the sole post-treatment period:

def compute_period_effect(dataset, analysis_date):
    subset = dataset.copy()
    subset = subset[
        (subset['date'] == analysis_date) | (subset['post'] == 0)
    ]
    subset = subset[subset['date'] <= analysis_date]
    
    subset['treatment_period'] = (subset['date'] == analysis_date).astype(int)
    
    did_model = smf.ols(
        'downloads ~ I(treated * treatment_period) + C(city) + C(date)',
        data=subset
    ).fit(cov_type='cluster', cov_kwds={'groups': subset['city']})
    
    treatment_effect = did_model.params['I(treated * treatment_period)']
    conf_bounds = did_model.conf_int().loc['I(treated * treatment_period)']
    
    return pd.DataFrame({
        'estimated_att': treatment_effect,
        'ci_lower': conf_bounds[0],
        'ci_upper': conf_bounds[1]
    }, index=[analysis_date])

The function first filters the dataset to include only pre-treatment observations alongside the specified date, then restricts the data to dates at or before the analysis date. When the input date is a post-treatment date, this filtering has no effect. However, passing a pre-treatment date triggers a placebo test by simulating a scenario where that earlier date represents the intervention point.

The key step reassigns the post-treatment indicator based on the specified date. This allows running DID estimates for any period, including pre-treatment dates as falsification tests. The function then estimates the DID model, extracts the ATT and its confidence interval, and returns the results in a single-row DataFrame.

To obtain effect estimates across all available dates, iterate through each period while skipping the first date since DID requires at least two time periods:

all_dates = sorted(mkt_data['date'].unique())[1:]

effect_estimates = pd.concat([
    compute_period_effect(mkt_data, period) 
    for period in all_dates
])

effect_estimates.head()

Visualizing these time-varying effects with confidence intervals reveals important dynamics. The plot demonstrates that treatment effects do not surge immediately after intervention. Furthermore, ATT estimates appear higher when excluding early transitional periods. Overlaying the true treatment effect τ shows how well this approach recovers the actual effect magnitude.

The pre-treatment portion of the visualization merits particular attention. During this phase, all estimated effects cluster around zero, indicating no effect existed prior to the intervention. This pattern provides strong evidence supporting the no-anticipation assumption, confirming that treatment effects only materialized after the actual intervention occurred.