Incrementality Testing Examples: Two Surprising Results That Rewrote the Media Mix

Two real incrementality testing examples that defied the conventional playbook. One proved a “safe” channel was barely working. The other proved an overlooked channel was quietly outperforming the ones marketers obsess over.

Most marketers can recite their platform-reported ROAS. Far fewer can say which of those sales would have happened anyway. That gap is what incrementality testing closes, and the surprise is that it cuts in both directions. Sometimes it exposes a channel you trusted as mostly wasted spend. Just as often, it reveals a channel you under-funded as the best growth engine you have.

Below are two documented incrementality testing examples that landed on opposite ends of that spectrum. The first is the one everyone expects: a “reliable” channel that turned out to be coasting on demand it never created. The second is the one almost nobody expects: a quiet, second-tier social platform beating the hyped names on incremental return.

• Example 1 (the overrated channel): A premium fashion retailer ran a geo holdout test on Google branded search. With ads off in test markets, it lost just seven orders. Platform reporting had overstated impact by up to 5X. The brand cut branded search spend over 84% and reduced total media spend 46% in six months while keeping 99%+ of orders.
• Example 2 (the underrated channel): Across 70+ Pinterest incrementality tests in a 12-month window, Pinterest outperformed other view-based social platforms on incremental ROAS, and not by a small margin. The surprise: the channel marketers treat as secondary beat the ones they hype.
• The pattern: Incrementality testing catches waste and hidden value. Attribution and last-click reporting routinely miss both.
• The lesson: The biggest budget wins come from spending less where ads only intercept existing demand, and more where ads actually create it.

Incrementality testing is a measurement method that isolates the true causal impact of advertising by comparing a group exposed to ads against a held-out group that is not. The difference in outcomes, the incremental sales or conversions, is the value the advertising actually caused, rather than the value a platform claims credit for.

It answers one decisive question: if we turned this spend off, would these sales still happen? Attribution cannot answer that, because attribution assigns credit to the touchpoints it observes and never sees the counterfactual. Incrementality builds the counterfactual on purpose.

The discipline has moved from niche to standard practice. Per a 2026 EMARKETER and TransUnion survey, 27.6% of US brand and agency marketers say expanding incrementality testing is a top measurement priority. And across 225 geo-based tests on one vendor’s platform between August 2024 and December 2025, the median iROAS was 2.31x, with 88.4% of tests reaching statistical significance. In other words, platform-reported numbers and real incremental value diverge constantly, in both directions.

Branded search, a premium fashion retailer

This is the result most people brace for when they finally test: a trusted channel that turns out to be coasting.

The setup. A premium fashion retailer suspected its platform and Google Analytics reporting looked “suspiciously overblown.” Rather than guess, it diagnosed first, pulling every channel into a single cross-channel view to find the worst offender. The prime suspect was branded search, where Google appeared to be claiming credit for high-intent shoppers who were already typing the brand’s name into the search bar.

The execution. The team designed a geo holdout test, splitting markets into matched test and control groups and switching branded search off in the holdout markets. The critical design choice was holdout size: they withheld less than 10% of the channel’s digital spend. That kept revenue at risk to a minimum and let them keep testing other channels at the same time without contaminating results.

The result. Reporting had not just been optimistic, it had been wildly off. The analysis showed Google was over-reporting its impact by as much as 5X. The holdout made it concrete: with branded search switched off in test markets, the brand lost just seven orders. The channel was harvesting demand that already existed.

What went right.

• The team diagnosed before testing, so the experiment was aimed at the highest-value question.
• The sub-10% holdout protected revenue and made concurrent testing safe, so the program moved fast.
• The corrected figure fed straight into the brand’s media mix model, so the insight kept paying off every planning cycle.
• “We lost seven orders” is a number any CFO understands instantly, which made bold reallocation politically possible.

What could be improved.

• Branded search is rarely a clean zero. Fully abandoning brand terms can hand them to competitors who conquest your name in the auction, which is why the brand cut roughly 84% rather than 100%.
• One test is a snapshot, not a law. Auction dynamics and competitor behavior shift, so incrementality should be re-tested on a cadence.
• Seasonality and test duration matter. A test run in an atypical demand window can mislead, so tests should span a full purchase cycle.

How the mix changed and the gain. Armed with the evidence, the brand cut branded search spend over 84%, keeping a thin defensive presence on brand terms, and reallocated the freed budget across the portfolio using its media mix model and a media plan optimizer. Portfolio-wide, media spend dropped 46% over six months while the brand kept more than 99% of its orders. Same demand captured, roughly half the budget. By the brand’s own account, the return on the measurement investment ran into multiples a finance team would notice.

Pinterest, across 70+ incrementality tests

Here is the result almost nobody braces for, and the reason this piece exists. When marketers rank social platforms, the budget and the buzz flow to the usual names. Pinterest tends to sit in the “nice to have” tier, behind feed-based social. The incrementality data tells a different story.

The setup and method. Over a 12-month window, more than 70 Pinterest incrementality tests were run across a portfolio of brands, part of roughly 550 geo tests overall, and paired with causal media mix modeling. Instead of judging a single campaign, results were bucketed by platform so the trendline could be compared across “view-based social” channels on a like-for-like basis.

The surprising finding. Pinterest consistently ranked at the top of view-based social platforms on average incremental ROAS, and not by a small margin. The pattern held not only in individual geo tests but in the MMMs across the portfolio. The channel treated as secondary was beating the channels marketers treat as primary.

Why it happens. Pinterest behaves less like a feed and more like a search engine with commercial intent:

• High-intent environment. Users go to Pinterest to plan and discover, so ads are often welcomed rather than skipped. That dynamic is closer to paid search than feed-based social, which is exactly why it shows up so strongly in incrementality testing.
• Longer consideration cycles. Pinterest content has a longer shelf life than ephemeral Stories or short-form video, creating more chances to influence a purchase over time.
• Commercial intent at search scale. Pinterest now processes roughly 80 billion monthly visual searches, and its positioning has shifted from social discovery to visual product search.

Why it stays undervalued. If the performance is real, why do plans under-fund it?

• Perception lag. It is still seen as niche outside categories like CPG, retail, and home.
• Less hype. Pinterest does not generate the splashy headlines that pull experimental budgets toward TikTok or Snap.
• Misalignment with last-click culture. Its strength is influencing consideration over a slightly longer horizon, which last-click attribution and short attribution windows systematically undercount. Click-based metrics hide exactly the value incrementality surfaces.

How the mix should change, and what to watch. The implication is direct: brands have room to increase Pinterest’s share of the social budget to match its measured performance. But spend strategically, not blindly. The clearest cautionary signal from recent tests involves Pinterest’s automated Performance+ product. When Performance+ and traditional campaigns overlap in the same funnel stage beyond roughly 20 to 30%, incrementality suffers, because Performance+ is not reaching fundamentally different inventory or audiences. The fix is to run Performance+ as an independent A/B test against traditional campaigns to avoid cannibalizing your own audiences. Pinterest’s momentum only sharpens the point: it posted its first billion-dollar quarter in Q1 2026 at $1.008B in revenue, up 18% year over year, with 631 million monthly active users.

On the surface they are opposites. One channel got cut by 84%, the other earned an argument for more budget. But they make the same point.

Platform reporting and last-click attribution are biased in predictable ways. They over-credit channels that intercept existing demand (branded search catching someone already searching your name) and under-credit channels that build demand over a longer horizon (Pinterest influencing a purchase weeks later). Incrementality testing corrects both errors at once, because it ignores who clicked and measures only what actually moved the business.

That is why the real prize is not “spend less” or “spend more.” It is reallocation: pulling dollars out of spend that only looked productive and pushing them into spend that was quietly doing the work. The fashion retailer found half its budget was redundant. The Pinterest pattern shows the other half of the story, growth sitting unfunded because the wrong yardstick kept it invisible.

1. Pick the channel that matters most. Start where platform-reported ROAS feels too good to be true (branded search, retargeting) or where a doubted channel might be a sleeper (Pinterest, CTV, a second-tier social platform).
2. Choose a geo holdout for omnichannel impact. Geo tests suit any channel where user-level holdouts are impractical, including TV, radio, retail media, and anything ending in an offline sale.
3. Keep the holdout small. A holdout under roughly 10% protects revenue and lets you test multiple channels at once.
4. Run it long enough. Cover a full purchase cycle, generally two to four weeks minimum for retail, longer for considered purchases.
5. Feed results into your MMM. Use the corrected incrementality figure to recalibrate the model and guide ongoing budget decisions, not a one-time cut.
6. Re-test on a cadence. Treat incrementality as a living input. Algorithms, competitors, and seasons change, so your measurement should keep up.

What is an example of incrementality testing? A premium fashion retailer ran a geo holdout test that switched off Google branded search in test markets. It lost only seven orders, revealing platform reporting had overstated the channel by up to 5X. The brand cut branded search spend over 84% and reduced total media spend 46% while keeping 99%+ of orders.

Can incrementality testing show a channel is better than expected? Yes, and that is one of its most valuable uses. Across more than 70 Pinterest incrementality tests in a 12-month period, Pinterest outperformed other view-based social platforms on incremental ROAS by a wide margin, even though many media plans still treat it as secondary. Channels that build demand over a longer horizon are routinely undercounted by last-click attribution.

Why does last-click attribution undervalue channels like Pinterest? Last-click credits the final touch before a purchase. Channels that influence consideration earlier, or over a longer window, rarely get that final click, so their contribution stays hidden. Incrementality and triangulated measurement reveal it because they measure business outcomes rather than clicks.

What is the difference between incrementality and attribution? Attribution assigns credit to observed touchpoints along a path to conversion. Incrementality measures whether the conversion would have happened at all without the ad, by comparing exposed and held-out groups. Attribution can both over-credit and under-credit, while incrementality measures true causal lift.

How long should an incrementality test run? Long enough to capture a full purchase cycle, typically a minimum of two to four weeks for retail, and longer for higher-consideration categories. Tests that are too short risk underpowered, misleading results.

What is a good incremental ROAS (iROAS)? It depends on margin and payback goals, but benchmarks help. Across 225 geo-based tests in 2024 and 2025, the median iROAS was about 2.31x. Compare iROAS to your breakeven target rather than to platform-reported ROAS.