What Is Incrementality Testing? A Practical Guide
Written by Elias Oender
March 26, 2026 3 min read
The quick answer
Incrementality testing measures the true impact of marketing efforts by isolating their effect. It uses methods like holdout and geo tests to compare outcomes with and without campaigns. This approach often beats trusting platform-reported conversions, especially when accuracy is critical.
What Is Incrementality Testing?
Incrementality testing is a method to measure the true impact of marketing campaigns by isolating their effect. It answers the question: Would conversions have happened without the campaign? Unlike platform-reported metrics, which often attribute conversions broadly, incrementality testing focuses on the actual lift generated. This approach is particularly useful when skepticism about attribution accuracy is high. For example, if you’re unsure whether your marketing attribution model reflects reality, incrementality testing can provide clarity.
How Do Holdout and Geo Tests Work?
Holdout tests divide your audience into two groups: one exposed to the campaign and another held back as a control. By comparing outcomes, you measure incremental lift. Geo tests, on the other hand, target specific geographic areas while leaving others untouched, allowing for a broader comparison. Both methods are powerful but require careful planning. For instance, geo tests work best when markets are similar enough to ensure valid comparisons. If you’re unsure which method suits your needs, consider starting with a free marketing scan.
One account runs geo tests with a ROAS-split budget allocation, ensuring test branches in different regions compete fairly while maintaining a floor spend per branch. This setup allows incremental lift to emerge clearly, even when markets differ slightly. The data feeds into a weekly optimization loop where losing angles lose budget fast, preventing prolonged underperformance.
When Is Incrementality Testing Worth It?
Incrementality testing is worth the effort when platform-reported conversions may be biased or inflated. For example, if your campaigns rely heavily on retargeting, platform metrics might overstate their impact. Incrementality testing can also help you optimize your marketing budget by identifying high-performing strategies. However, it’s not always necessary. For smaller campaigns or when precision is less critical, platform-reported data might suffice. If you’re ready to explore incrementality testing further, book a quick call to discuss your strategy.
Common Pitfalls in Incrementality Testing
Incrementality testing sounds straightforward, but it’s easy to mess up. One of the biggest mistakes is failing to isolate variables. If you’re running multiple campaigns simultaneously, it’s nearly impossible to attribute lift accurately. For example, if you’re testing a Facebook ad campaign but also running Google Ads, the overlap muddies the results. Always ensure your test groups are clean and isolated.
Another pitfall is ignoring external factors. Seasonal events, competitor activity, or even weather can skew your results. If you’re running a geo test during a holiday sale in one region but not another, the lift might not be due to your campaign at all. Always account for these variables in your analysis.
Finally, sample size matters. If your test group is too small, the results won’t be statistically significant. I’ve seen campaigns where the lift looked promising, but the sample size was so small that the data was useless. Aim for a sample size that gives you confidence in the results, even if it means running the test longer.
How to Act on Incrementality Test Results
Once you’ve got your incrementality data, it’s time to act. Here’s a simple framework:
- Identify the winners: Look for campaigns or channels with the highest incremental lift. These are your top performers.
- Kill the losers: Any campaign or channel showing little to no lift should be paused or reworked. Don’t waste budget on what’s not working.
- Reallocate budget: Shift funds from underperforming areas to your winners. This ensures your marketing dollars are spent where they have the most impact.
- Iterate: Use the insights to refine your next test. Incrementality testing isn’t a one-time event, it’s an ongoing process.
Remember, the goal isn’t just to measure lift, but to optimize your marketing spend. If a campaign shows a 20% lift but costs twice as much as another with a 15% lift, the latter might still be the better choice. Always weigh the lift against the cost.
One final tip: document everything. Keep a record of your test setup, results, and actions taken. This helps you build a knowledge base over time and avoids repeating mistakes. Incrementality testing is a powerful tool, but only if you use it wisely.
