Every ad platform reports a return on ad spend, and every one of them counts some sales that would have happened anyway. A customer who already had your store bookmarked clicks a retargeting ad on the way to checkout and the platform takes the credit. Add up the conversions claimed by Meta, Google and your email tool and the total is often larger than your actual order count.
The question that matters for budget is narrower: how many extra sales did this spend cause? That is incrementality, and the reliable way to measure it is an experiment. Ad platforms have made their own lift studies cheaper, but they still have floors that many independent stores do not reach. A geo holdout test, run on your own order data, has no such floor. This guide explains how to design one for a WooCommerce or Magento store.
Why not just use the platform’s lift study?
Sometimes you should. Google says that in 2025 it lowered the cost of an incrementality experiment from a previous high of $100,000 to a minimum of $5,000. When Google opened Conversion Lift to Search and Performance Max in October 2026, eligibility required at least 1,000 observed conversions and a minimum campaign budget of $5,000. A store with 300 orders a month does not qualify, and a platform study only measures that platform. A geo test measures your total sales, whichever channel you change.
How a geo experiment works
The method is old and well documented. In Google’s 2011 paper, non-overlapping geographic regions are randomly assigned to a control or treatment condition, and each region receives its condition through geo-targeted advertising. You change spend in some regions, leave the rest alone, and compare what happens to sales.
Comparing raw sales between regions does not work, because regions differ in size and seasonality. The comparison is instead between what the test regions actually sold and a counterfactual: an estimate of what they would have sold without the change, built from how they tracked the control regions before the test started.
Several open-source tools build that counterfactual. Meta’s GeoLift uses synthetic control methods and includes a market selection step that simulates candidate test regions before you spend anything. Google’s time-based regression was designed for situations with few geographic units, including single test-versus-control market comparisons. These are not interchangeable: a simulation study by Recast that ran 32,000 synthetic experiments found that four common open-source geo tools recover the true effect with different accuracy depending on the scenario.
Choosing a design
| Design | What you change | Good for | Watch out for |
|---|---|---|---|
| Holdout (go dark) | Pause a channel in test regions | Checking whether an existing channel earns its budget | Lost sales during the test; keep the test regions small |
| Scale-up | Raise spend in test regions | Deciding whether to spend more | Diminishing returns; the answer applies to the extra spend only |
| Randomized geo | Randomly assign many regions | U.S. states or DMAs, large countries | Needs enough regions with steady sales |
| Matched market | One or a few test regions against weighted controls | Countries with few regions, such as Canadian provinces | Wider uncertainty; depends heavily on pre-period fit |
Running one on a WooCommerce or Magento store
Step 1. Build a daily sales series by region
Use the shipping address on paid orders, not website analytics, so consent and ad blockers do not affect the outcome. In WooCommerce with High-Performance Order Storage, addresses sit in their own table:
SELECT DATE(o.date_created_gmt) AS day,
a.country,
a.state AS region,
COUNT(*) AS orders,
SUM(o.total_amount) AS revenue
FROM wp_wc_orders o
JOIN wp_wc_order_addresses a
ON a.order_id = o.id AND a.address_type = 'shipping'
WHERE o.type = 'shop_order'
AND o.status IN ('wc-processing', 'wc-completed')
GROUP BY day, a.country, a.state;
In Magento, join sales_order to sales_order_address where address_type = 'shipping' and use region or a postcode prefix. You need history: Recast’s guide to GeoLift suggests at least six months of daily data or a year of weekly data, broken down by geo.
Step 2. Check power before you start
Pretend the test already happened in last year’s data, with no real change, and see how large a fake effect the method would report. That spread is your noise floor. If the channel you plan to pause plausibly drives 3 percent of sales and the noise floor is 8 percent, the test cannot answer the question; lengthen it, enlarge the test regions, or test a bigger change. The same logic as sample size planning for A/B tests applies, which QuantumLayers covers in A/B Testing Statistics: Sample Size, Significance, and the Mistakes That Produce False Winners.
Step 3. Run it cleanly
- Run three to six weeks, covering whole weeks so weekday patterns cancel out.
- Change one thing. No sale, email blast or new product launch that hits test and control regions differently.
- Exclude the regions that hold your warehouse or a physical shop if local effects could distort them.
- Keep a cooldown of one or two weeks after the test and keep measuring; some ad effects arrive late.
Step 4. Turn the result into a budget number
A hypothetical example. A store spends $9,000 a month on Meta ads and pauses them for four weeks in regions that normally produce 30 percent of its revenue. That removes about $2,700 of spend.
| Counterfactual revenue in test regions | $45,800 |
| Actual revenue in test regions | $41,200 |
| Incremental revenue lost by pausing | $4,600 |
| Spend removed | $2,700 |
| Incremental ROAS (iROAS) | 4,600 / 2,700 = 1.70 |
| ROAS reported by the platform for the same period | 3.40 |
| Incrementality factor (iROAS / reported ROAS) | 0.50 |
| Gross profit per ad dollar at a 40% margin | 1.70 x 0.40 = $0.68 |
The platform says every dollar returns $3.40. The experiment says it returns $1.70 in revenue and $0.68 in gross profit, so at current spend the channel loses money. The incrementality factor of 0.5 can then be applied to the platform’s daily reports until the next test, which is far more useful than either number alone. Report the confidence interval with the point estimate; an iROAS of 1.70 with an interval from 0.4 to 3.0 is a reason to run a longer test, not to cut the budget.
Common ways geo tests go wrong
- Spillover. People see ads in one region and ship to another (a gift, an office address). Larger regions reduce this.
- Targeting leaks. Platform geo-targeting uses the viewer’s estimated location, which is not always accurate. Check delivery reports by region during the test.
- One oversized region. A test region much larger than any control is hard to match. Recast’s study includes exactly this scenario because it is common.
- Reading the result as causal for everything. A holdout measures the effect of the spend you removed, at that spend level, in that season. It does not tell you what doubling the budget would do. QuantumLayers’ piece on correlation versus causation is a good reminder of how narrow a causal claim usually is.
Where this leads: calibrating a marketing mix model
One test answers one question. A marketing mix model estimates every channel at once from historical spend and sales, but it needs real experiments to keep it honest. Google released its open-source MMM, Meridian, to all marketers in January 2025, and in September 2026 made Meridian GeoX, which runs geo experiments and feeds their results into the model, generally available. For a store with modest spend, two or three well-run geo tests a year on the largest channels will do more for budget decisions than a model built without them.
Sources
- Google, Marketing experimentation and incrementality testing
- Search Engine Land, Google opens Conversion Lift to Search and Performance Max
- Vaver and Koehler, Measuring Ad Effectiveness Using Geo Experiments, Google (2011)
- Wang and Vaver, Estimating Ad Effectiveness using Geo Experiments in a Time-Based Regression Framework, Google (2017)
- Meta Open Source, GeoLift Walkthrough
- Recast, Should You Be Using the GeoLift Library? and Geo-experiment Simulation Study
- MarTech, Google releases its open source marketing mix model to all
- TechWyse, Google Takes Meridian GeoX to General Availability Globally


