The Difference Between Incrementality, MMM, and MTA
Incrementality experimentation, media mix modeling (MMM), and multi-touch attribution (MTA) are all measurement methodologies that the modern marketer might take into consideration when trying to prove the value of their media spend. Let’s take a deep dive into what each methodology entails, what the differences and similarities are between them, and the evolution and future of media measurement.
Understanding the Core Concepts
What is Incrementality Experimentation?
Incrementality experimentation is the use of systematic in-market test vs. control groups to measure media’s impact on sales.
Incrementality experimentation is unique because it measures the causal relationship between media and sales by inherently controlling for all other external factors that could impact either. By applying experiment results to a reporting and optimization framework, brands are able to cut through the bias present in media platform reporting, as well as more observational, model-based measurement techniques.
What is Media Mix Modeling (MMM)?
Media Mix Modeling, or MMM, is a method of observing how week-to-week (or day-to-day) variation in media exposure is associated with variation in sales to estimate media’s impact on sales. The association (or correlation) of different media drivers to sales is used to calculate the relative impact of each.
MMM models require extensive sets of historical media data, including spend, impressions, and clicks, as well as non-media factors such as economic conditions, the weather, competitive conquesting, pricing, and operational data. While laborious and time-consuming to implement, MMM has a unique advantage: It can address and measure basically any marketing tactic for which you have historical data.
What is Multi-Touch Attribution (MTA)?
Multi-touch attribution, or MTA for short, is the approach of using user data to track consumer journeys across digital media touchpoints to ascertain which touchpoints are most present in the pathway of those who convert. These touchpoints are then attributed a percentage value of the conversion.
MTA is the most bottoms-up attribution approach because it collects customer journey data at the user level and then aggregates these pathways to create a modeling data set. While the ability to parse out different user pathways is attractive, the pixel and cookie requirements inevitably result in significant data limitations in today’s privacy-centric digital marketing world, rendering MTA results inaccurate at best but possibly reckless in many scenarios.
Comparative Analysis: Incrementality Testing vs MMM vs MTA
|- Based on causal inference
- Not modeled on historical data, so it lets you measure how disruptions to the media landscape impact media performance quickly
|- Measures all media with historical data available, including offline channels and non-addressable media
|- Ability to address digital media at a granular level and reveal insights about particular user pathways from a customer’s PoV
|- Some inherent business disruption
- Cannot test all media tactics simultaneously and continuously
|- Often takes months to deploy
- Relies on correlative inference
|- Hardly considered a feasible measurement approach due to the evolution of data privacy and “walled garden” platforms
What are the Pros and Cons of Incrementality Testing?
Incrementality is the most accurate form of measurement because it is based on causal inference (only a test vs. control experiment can reveal the true causal, not correlative, impact of media on sales). It also allows marketers to measure how disruptions to the media landscape (such as the COVID-19 pandemic or the release of iOS14 and the loss of the IDFA, etc.) impact media performance far more quickly than other techniques, as it is not modeled on historical data.
Incrementality tests do require some inherent business disruption, as they are typically run over a period of 14-30 days or more, depending on the brand’s consumer consideration cycle, and only a few tactics can be tested simultaneously. As such, incrementality itself must be applied to an existing reporting/attribution framework, as you cannot test all media tactics simultaneously and continuously. Additionally, Incrementality cannot be deployed on media channels where geo-targeting is unavailable (for example, by itself, incrementality tests cannot be deployed on influencer or affiliate marketing channels).
Usage in Strategic Planning and Tactical Shifts
Here’s an example of incrementality testing in use: Brand A wants to know the impact of its Facebook Prospecting campaign on sales.
- Design: Select five states that are statistically representative of the overall business while being relatively minor in volume
- Execution: Facebook Prospecting is withheld only from those five states for 30 days
- Result: Compare the transaction volume in those five states to the rest of the country (this is your control group). The amount of sales “lost” when Facebook Prospecting is removed is considered the “contribution” of Facebook Prospecting to the business, all else equal.
What are the Pros and Cons of Media Mix Modeling?
MMM’s strength is that it can measure all media for which historical data is available, including offline channels and non-addressable media. It also excels at capturing the longer-term impacts of media on sales and calculating the diminishing relationship between spend and ROI for a given media tactic.
As MMM relies on correlative, not causal, inference, MMM needs to be supplemented with Incrementality experimentation to produce credible results. Additionally, traditional MMM often takes months to deploy, given the laborious data lift, and is typically updated only a few times per year. This makes it a poor method for measuring media at a highly dynamic and granular level.
Usage in Strategic Planning and Tactical Shifts
Here’s an example of MMM in use: Brand B wants to know the impact of linear TV on sales.
- Strategy: Collect weekly historical data for all various marketing efforts, sales, and a few key external factors (such as the economy, weather, and seasonality)
- Execution: This data is consolidated into a single source, and using modeling software, a regression algorithm is run to determine the relative association of each marketing variable to sales, all else being equal
- Results: This association (or coefficient) is then used to calculate how many sales linear TV was responsible for within a given time period
What are the Pros and Cons of Multi-Touch Attribution?
MTA’s strength is in its ability to address digital media at a granular level and reveal insights about particular user pathways from a customer’s point of view.
However, the evolution of data privacy and “walled garden” media platforms have created challenges in data collection that far outweigh MTA’s utility. As such, MTA is not generally recommended as a feasible measurement approach.
Usage in Strategic Planning and Tactical Shifts
Here’s an example of MMM in use: Brand C wants to know the impact of brand search on sales.
- Strategy: Build a bottom-up data set compiling user pathways that include the various media users “touched” along the way to converting in the store
- Execution: A modeling software is used to run a Regression, which determines the relative likelihood of a Brand Search click to be present in a converting versus non-converting customer journey, all else being equal
- Results: This likelihood (or coefficient) is then used to determine the total number of sales that Brand Search was responsible for within a given time period
Evolution of Measurement Methods
The Journey Towards Automated Experimentation
Prior to the digital advertising revolution, MMM was the primary form of media measurement as it did not rely on granular tracking or targeting ability and was particularly useful for measuring more above-the-line, traditional upper funnel tactics such as linear TV, radio, sponsorships, and print.
With the rise in digital advertising, along with detailed user tracking, MTA became prevalent thanks to the specificity and granularity of its reporting and insights. However, doubts remained in the honesty of this technique as it tended to be very favorable to lower funnel, click-based channels due to their inherent correlation with demand, as well as the difficulty of collecting data on more upper funnel, view-based channels. These are all challenges to the effectiveness of MTA.
This led to the rise of incrementality experimentation, which was able to adequately assess the impact of lower-funnel channels based on causal inference while at the same time solving for the addressability issues of view-based media. However, as mentioned above, Incrementality experimentation still had shortcomings - it was not available for channels lacking audience or geo-targeting capabilities.
What is the Future of Media Measurement?
Clearly, each of these three measurement techniques faces challenges on its own.
As a result, the most comprehensive and accurate form of media measurement in the coming era will be MMM-supported incrementality measurement, applied to a granular reporting and optimization framework.
Such a solution would provide:
- Causal inference derived from incrementality experimentation
- Coverage across entire media portfolios via MMM
- Granular reporting and optimization capabilities enabling real-time decision-making
If such a solution sounds like something your brand would be interested in learning more about, check out our page on the Measured Incrementality Platform today.