How Can I Measure Incrementality on Facebook/Meta?

Terence Einhorn, Sr. Solutions Consultant in Sales

Published 10/01/2022

4 Advanced Marketing Measurement Techniques That Work

It is well known that basic on-platform and site analytics (e.g., last touch) metrics are insufficient for performance measurement on Meta due to their inability to measure incrementality and their limited tracking capabilities, respectively.

To solve for these shortcomings, there are four main techniques marketers can use to measure the incremental effectiveness of Facebook/Meta ads:

Below, we’ll review all four and discuss their benefits and challenges.

On-Platform Studies

The goal of Meta platform studies is to measure the number of platform-tracked conversions that “would have happened anyway” had those converting customers not been served an ad. 

There are two methodologies that measure this “counterfactual” number:

  • Conversion Lift Study

 Meta takes your campaign audience and carves out a “test group” and a “control group.” The control group is then withheld from receiving that campaign, and their conversion behavior is compared with the test group to assess incremental lift, usually via Conversions API.

  • PSA Testing

Similar to Conversion lift, PSA testing entails carving out a portion of your audience and serving them a “PSA” ad or an advertisement that generally has no connection to your business (for example, a Smokey the Bear PSA).

Though there are some nuances between these two methodologies in terms of conversion tracking, their main goal is the same: to measure how many of the non-exposed customers end up converting anyway in the absence of your campaign.

The Pros and Cons of On-Platform Studies 

The main benefit of platform studies is that they are basically free (although, in some cases, they can be fairly time-consuming to implement).

However, due to the tracking limitations brought on by the current data privacy landscape, platform studies can yield highly inaccurate results due to low “event match quality” (the ability for Meta to pair an eventual conversion with an unexposed user), as well as the ability for Meta to identify a truly representative control group.

While some marketers still believe that on-platform studies can be effective, the reality is that in today’s marketing landscape, they are not. In fact, some platforms have moved away from first-party conversion lift tests altogether. 

As such, while platform studies may be tempting to a brand with limited budget and time, they are not recommended as a single source of truth for Meta effectiveness.

Media Mix Models (MMM)

Media Mix Modeling, or MMM, is a method of observing how week-to-week (or day-to-day) variation in media exposure is associated with variation in sales to estimate media’s impact on sales. The association (or correlation) of different media drivers to sales is used to calculate the relative impact of each.

Using MMM to measure Meta/Facebook requires an extensive set of historical data beyond Meta, including other media’s spend/impressions/clicks, as well as non-media factors such as economic conditions, the weather, competitive conquesting, pricing, and operational data. 

The Pros and Cons of Media Mix Modeling (MMM)

The main benefit of measuring Meta performance with MMM is that MMM can be deployed against third-party sales (such as wholesale, Amazon, etc.) in a way that’s difficult to do with lift studies, MTA, or incrementality.

The main drawbacks of MMM are that it is a regression and, therefore, is difficult to tease out causal relationships without advanced modifications, combining it with incrementality testing, and/or intentionally varying spend.  These models are also resource-intensive, require a significant amount of data collection and processing/data science resources, and can often take months to complete.

Incrementality Experiments

Similar to platform studies, incrementality experiments divide a Meta audience into Test vs. Control groups to measure lift vs. a “counterfactual.” 

However, incrementality experiments typically divide audiences based on markets (e.g., state, DMA) and measures lift against first-party transaction data in these markets, as opposed to Meta’s conversion tracking. Unlike a platform study, these tests can be managed independently by marketers, or using a tool like Measured.

The key advantage of this method is that it is not reliant on the tracking ability or event match quality of the Meta platform.

The Pros and Cons of Incrementality Experiments

The main benefit of incrementality experimentation is that it accurately measures the causal impact between Meta and Sales, including Meta’s interactive effect on other Media Channels (discussed further below).

Additionally, because results are based on first-party transaction data, incrementality experimentation can identify whether Meta is underreporting their own conversion volume (a common case for brands with extended consideration cycles).

Also, unlike MMM and MTA, incrementality experiments are episodic in nature and can, therefore, measure how meaningful changes to your Meta execution or the landscape overall (i.e., iOS14) impact Meta performance immediately. In contrast, MMM and MTA can only deliver an average impact over a longer period of time.

The main drawback of incrementality experimentation is its inherent business disruption (holding Meta out from a few markets may have a net negative impact on sales).

Additionally, if a tool like Measured isn’t used to administer these tests, a robust data science effort is required to execute an accurate market selection/test design to ensure results are reliable while also minimizing the aforementioned business risk.

Multi-touch Attribution (MTA)

Multi-touch attribution, or MTA for short, is the approach of using user data to track consumer journeys across digital media touchpoints to ascertain which touchpoints are most present in converting vs. non-converting pathways. These touchpoints are then attributed a fraction of each conversion based on this correlation.
The Pros and Cons of Multi-touch Attribution (MTA):
MTA’s hypothetical benefit is the ability to measure incrementality at a highly granular level (i.e., Adset, Ad, etc.) due to its bottoms-up nature. 

While the ability to parse out different user pathways is attractive, the pixel and cookie requirements inevitably result in significant data limitations in today’s privacy-centric digital marketing world, rendering MTA results inaccurate at best - and actively harmful in many cases.

In reality, MTA is not a scientific measurement, and even with the perfect data necessary, the results would likely be inaccurate and misleading.

Why incrementality is the gold standard for Facebook/Meta Measurement

The main benefit of Incrementality vs. media mix modeling or multi-touch attribution is that it measures the causal impact of Meta on overall sales, whereas MMM and MTA simply measure the correlative impact.

For example, Instagram retargeting may be highly correlated with your sales in that its execution is driven by customers visiting a site (already a signal of purchase intent). However, this does not mean that the retargeting ads are driving significant incremental conversions if most of these customers are likely to convert anyway.

Compared to on-platform studies, the main benefit of incrementality is that it measures the net impact of a Meta program on Sales, inclusive of the interactive effects Meta may have with other channels.

For example, in the absence of Facebook Prospecting, your affiliate program may not be as productive. This means the causal impact of Facebook Prospecting on the business is greater than just what one can see on the platform. 

Additionally, incrementality experimentation can identify whether Meta is underreporting their own conversion volume due to tracking limitations. It can also identify the incremental lift of Meta against non-tracked conversions (such as in-store conversions).

As such, incrementality experimentation is generally considered the best option for measuring Meta effectiveness out of the four techniques summarized above.

Video: Facebook Incrementality Measurement

Facebook Incrementality Measurement

Want to Learn More About
Incrementality and Attribution?