Control groups are the gold standard for incrementality. But they're not always possible.
Always-on campaigns don't have an unexposed group, by design, every shopper is targeted. In-store campaigns where screens play to everyone in the store can't cleanly separate exposed and unexposed shoppers in the same location. Small-scale campaigns might not have enough volume to produce statistically significant control group results.
When you can't run a clean test/control experiment, you need a synthetic alternative.
Not as good as the real thing. But far better than no measurement at all.
What synthetic testing means
Synthetic testing constructs a counterfactual, an estimate of what would have happened without the campaign, using historical data and statistical methods rather than a live control group.
The most common approaches:
Matched market testing. Compare stores or regions where the campaign ran to similar stores/regions where it didn't. The "control" isn't a randomly assigned holdout, it's a matched comparison set selected for similarity in sales patterns, demographics, and competitive dynamics.
Time-series modeling. Build a demand forecast based on pre-campaign data, seasonality, trends, promotions, pricing, distribution. Project what should have sold during the campaign period. Compare the projection to actual sales. The gap is the estimated campaign effect.
Synthetic control method. A weighted combination of non-exposed units (stores, regions, or shopper groups) that together approximate the treated group's pre-campaign behavior. The synthetic control acts as a statistically constructed twin.
Each method has trade-offs. Matched markets assume the comparison set is truly comparable. Time-series models assume the future follows the past. Synthetic controls assume the weights estimated from pre-campaign data hold during the campaign.
When to use synthetic methods
Always-on programs where there's no natural unexposed group
In-store campaigns where geographic holdouts aren't practical
Small campaigns where control group sizes would be too small for significance
Retrospective analysis when a control group wasn't designed into the original campaign
The transparency requirement
Synthetic methods introduce assumptions. Honest reporting means stating those assumptions clearly:
"The synthetic control was constructed from 15 non-exposed stores matched on category sales trend and demographic profile. Pre-campaign fit: R² = 0.94."
"The demand forecast uses 52 weeks of historical data, accounting for seasonality, promotion cadence, and distribution changes."
When you state the rules, the brand can evaluate the method. When you hide the method, the brand has to trust you, and trust without transparency doesn't last.
At Footprints AI, when we can't run clean control groups, we use transparent synthetic methods and validate them against campaigns where control groups are available. If the synthetic method produces estimates within 10% of the control group result on validation campaigns, we can use it with confidence on campaigns where control groups aren't feasible.
The bottom line
When you can't run clean test/control, you need a method that's transparent and consistent. Synthetic testing provides that, a statistically constructed counterfactual that estimates what would have happened without the campaign.
It's not as clean as a randomized control group. But it's far better than pre/post comparison, which confounds every external factor, or no measurement at all.
State the method. State the assumptions. Validate against control group results when possible. And always prefer real control groups when they're feasible.
Related Reading
- Minimum Proof Package: The Measurement Bundle That Makes Buyers Renew
- Category Index: How to Tell If You Beat the Category, Not the Calendar
- iROAS: The Retail Media Number That Makes Budget Decisions Easy
- Value Metric: What Retail Media Should Price Against
- Same-SKU ROAS vs Halo ROAS: Two Retail Media Stories, One Truth
Ready to see how this works in practice?
Footprints AI helps brands and retailers measure what matters. See our customer success stories or get in touch to discuss your retail media strategy.
