How do I measure incrementality on display advertising?

Nick Stoltz, Expert in Cross-Channel Measurement Strategy and Adoption

Published 04/13/2022

Measuring display advertising can be a challenge. It’s often difficult to account for views and ad impressions which can contribute to conversion goals driving your business. The most common & easiest method of measuring display advertising is through view-through and click-through measurements provided by the publishers. There are pros and cons to using publisher provided view-through and click-through data:

Advantages of Publisher Provided Display Advertising Measurement

  1. The first obvious “pro” is that the measurement is provided by the publisher so there is no extra work required to measure the ad campaign.
  2. You can use this data to optimize campaigns/audiences with that publisher, but it has its limitations.

Disadvantages of Publisher Provided Display Advertising Measurement

  1. The main “con” of using publisher provided view-through and click-through conversions is that you will be double counting across other media channels. For example, if someone sees your display ad on site X and sees an ad on Facebook, how do you attribute the conversion?
  2. If your media portfolio consists of more than 1 channel (which is most certainly the case) you can’t compare the results in an apples to apples way with other publisher measurement results.

In order to measure true contribution of the display advertising, marketers need to measure the incrementality of the display ads. The incremental contribution of display advertising can be measured by using audience holdouts, serving the held out audience a placebo ad, and comparing the measured conversion rate of the held-out audience versus the campaign (or exposed) audience.

Design of Experiments (DOE) Example: Segmenting Control Group and Reporting on Cohort Incrementality

This process is called Design of Experiments (DoE). When expertly designed, it has the ability to deliver on the promise of incrementality measurement at the vendor, campaign and audience level in a way that MMM cannot due to practical limits on data granularity and degrees of freedom.

Incrementality Testing for Retargeting Tactics

For heavily biased tactics like retargeting, DoE incrementality results can be actively incorporated into MMM as Bayesian Priors to improve MMM models across the board. For retargeting tactics, DoE offers the most unbiased measurement approach, as it randomly selects a subset of website visitors for exclusion from retargeting impressions, both in total and at the vendor level, in order to measure true incrementality of these tactics on a customer group that has already established interest and intent.

Design of experiment incrementality measurement for retargeting marketing example showing audiences segmented by platforms: Facebook, Criteo, Pinterest, All three vendors and control population.

Incrementality Testing for Prospecting Tactics

For prospecting tactics such as Facebook, DoE randomly selects a subset of prospects to serve as the control group. One approach to capturing a control audience is to show them a placebo such as a PSA advertisement (charity ad) which has nothing to do with the brand, but serves as a way to initiate tracking and thus segmenting the user away from the exposed cohort. Because this is designed at the group level, DoEs are not subject to all of the user level data challenges encountered by MTA requiring only that campaigns exhibit enough reach to establish statistical significance at the group level. For most advertisers this statistical significance is achieved in a matter of weeks and can be meaningfully updated afterwards on a weekly basis to inform tactical campaign optimization.

Incrementality Testing for Prospecting Tactics such as Facebook


Want to learn more about how to reveal incrementality on all your advertising channels? Read the guide!

Want to Learn More About
Incrementality and Attribution?