Sophisticated matching techniques create analytically equivalent groups to measure program lift more conservatively.
Pre-Join Matching
Members and non-members are matched based on their behavior before the member joined — transactional characteristics, frequency, and order value.
Nearest Neighbor Matching
Individual members are matched to analytically similar non-members, then behavior is compared in the post-period to isolate program effects.
Control Variables
Models control for external factors like location effects and household-level data to isolate the true program lift from confounding variables.
1 Propensity Score Matching
Calculate a single score representing each customer’s probability of joining, then match members to non-members with similar scores.
- Reduces dimensionality — one score vs. many variables
- Creates balanced comparison groups on observable characteristics
- Assumes no unobserved confounders (unlike Heckman)
- Works well when you have rich pre-join behavioral data
2 Difference-in-Differences
Compare the change in spending for members vs. matched non-members from pre- to post-period.
- Controls for time-invariant differences between groups
- Accounts for external trends affecting everyone equally
- True lift = (Member post − Member pre) − (Control post − Control pre)
- Requires parallel trends assumption to hold