Measured Incrementality Model: High-Level Explainer

The Measured Incrementality Model (MIM) is built to quantify the incremental impact of media, support budget planning, and drive business growth for clients. Designed with configurability, flexibility, and agility at its core, the model is causally anchored by geo-study priors, incorporates configurable non-media variables, and recommends either spend or platform order inputs at the tactic-level based on data quality. Users can also incorporate brand-specific marketing performance information (using CPO values) at both the total- and tactic-levels. We are committed to continuously evolving the model to ensure it remains a powerful and reliable decision-support tool.

The MIM is a regression-based marketing mix model (MMM) which incorporates Bayesian-style elements. While not fully Bayesian, it ingests several types of priors, such as geo-study causal results—without requiring specification of the full data-generating process. At a high-level, the regression is defined as:

The regression is implemented in two components:

1. A non-mediacomponent ( ) that:
   1. Can be thought of as a feature engineering / dimensionality reduction step because it generates a single aggregated variable representing the non-media performance.
   2. Has a non-linear, multiplicative structure where the dependent variable is de-seasoned and then log-transformed prior to model fitting.
   3. Has intercept, seasonality, and holiday terms included by default with macroeconomic indexes, price and promotion indexes, trend terms, and more available via configuration (and assuming data availability). The price and promotion indexes are log-transformed, which creates a partial log-log model behavior and an elasticity coefficient interpretation.
   4. Is regularized (Ridge) to enable the model to ingest many non-media drivers without performance degradation.
2. A media component () that
   1. Incorporates all tactic-level media inputs.
   2. Supports modeling by either spend or platform order signals depending on data quality. Coefficients are constrained to be non-negative and lightweight regularization is applied.
   3. Has adstocking applied to select tactics. (More below.)
   4. Generates response curves for every tactic when spend is available. (More below.)

Splitting into two components (1) ensures that non-media transformations do not obscure media result interpretations, (2) improves model scalability, and (3) mitigates issues related to high dimensionality.

The dependent variable is defined by the client and derived from the transaction data. Examples include orders, activations, and traffic, which can be subset by sales channel, customer type, or geography.

The model is fit using a customized-MLE (maximum likelihood estimation) procedure—rather than MCMC sampling—which supports constraints, regularization, observation weighting, and prior information while still ensuring efficient and reliable convergence at scale.

Regarding the data itself:

• Three years of data is required, but models can, in some cases, be built with less.
• The model operates on weekly data spanning Sunday through Saturday.
• The model inputs are weighted to make the model more responsive to recent media performance and strategic changes. The most recent year gets the most weight, the previous year less, and the earliest least.

There are three different types of priors that feed the model: geo priors, CPO priors, and base priors.

Geo Priors

• Any geo holdout test can be used for calibration.
• The model uses the study contribution to compute the prior value. We transform the contribution so that the resulting prior has the same interpretation as the regression coefficient. E.g., If the tactic is modeled using spend, then the coefficient is orders (dependent variable) per unit spend.
• Paired with the prior value is a weight. The weight is a function of several inputs, including the recency and confidence of the study. A higher weight means more influence on the model.
• If the study involved multiple tactics, then the study contribution is decomposed before model fitting to create a unique prior for each tactic included in the study. The contribution is divided amongst the tactics using the proportion of spend belonging to each.
• If the study result is flat or negative, we set a very small (near-zero) contribution, since model coefficients are constrained to be non-negative. This adjusted contribution is defined as half the distance between 0 and the upper bound of the confidence interval, with additional safeguards to handle all-negative intervals and to avoid pushing the prior too far into positive territory. The prior weight is then scaled down proportional to the size of the adjustment.
• There is no limit to the number of geo studies that can be used for calibration, but any geo study can be excluded from the model if no longer relevant.
• Since geo studies are causal results, the model is constructed so that the coefficients align fairly closely with the geo study results.

CPO Priors

• These priors are secondary to the geo priors and carry much less weight. They are used when we want to (a) give the model freedom to move with divergent evidence, while (b) still providing a reasonable starting value or incorporating additional outside information.
• It is through these priors that clients can provide their own priors based on existing measurement models, including in-house MMMs, or previous experiments.
• CPO priors are provided to the model in two ways:
  • A total media contribution value that is converted to a portfolio CPO value and used as the prior for all modeled tactics.
  • A manual specification unique to a single tactic. These CPO values can come from non-Measured experiments, third-party MMMs, and/or other analyses.

Base Priors

• These priors are generated using the total media contribution value mentioned above.
• Like CPO priors, the goal of the base prior is to give the model a reasonable value with which it can initialize the regression.

The weights assigned to both the CPO and base priors are scaled according to model stability and data availability. This is one way we help mitigate omitted-variable bias. For example, if we have limited access to non-media variables, the priors can have a slightly higher weight, so that the media attribution isn’t artificially inflated.

Adstocking

Adstocking is selectively applied to tactics that are likely to have a delayed effect, which mainly includes offline, digital video, and upper funnel and awareness tactics. The MMM considers two shapes of lagged effect: (1) geometric (with penalization) and (2) Weibull. For the geometric specification, the spend week receives the highest share of the effect with each subsequent week getting a reduced amount. For the Weibull specification, the peak effect week may be delayed (i.e., not the spend week), which introduces an effect ramp up and ramp down. The adstock functional form and its parameters are optimized (data-driven) for each individual tactic. The model tests numerous possible parameter combinations and selects the optimal value based on correlation to the dependent variable.

Response Curves

Tactic-level response (i.e., diminishing returns) curves are modeled as negative-exponential functions (formula: ), where the and   parameters are fit separately for each tactic. There are two approaches to building the curves depending on the metric (spend or platform orders) used as the tactic input:

• Platform orders: The curve shape is derived from the spend and platform orders.
• Spend: The curve shape is learned as a part of the regression.

Prior to fitting a response curve, the data goes through several preparatory steps:

• Weighting: The most recent quarter gets the highest weight and every previous quarter slightly less, so that the optimized curve reflects and is responsive to the most recent performance.
• De-seasoning: Remove seasonality from the data used to build the curve, so that the curve is modeling the unobscured media impact.
• Removing outliers: Identify and remove outliers so that the response curve represents the average return at each spend level and isn’t skewed by anomalous business outcomes.

Response curves are used, in conjunction with the regression coefficients, to calculate tactic-level contribution. They are also a primary input into the Media Plan Optimizer (MPO), Measured’s media budget planning tool.

Required Client Provided Data

• Transaction data: The dependent variable of the model. It can have a flexible definition. For example, the dependent variable could be online orders, online orders from new customers, retail orders, Amazon orders, etc.
• Vendor data: The media data, including platform orders and spend, split by tactic. Both metrics are recommended for maximum configuration flexibility, but not required.
• This data, routinely sourced from multiple systems, is compiled by Measured through client-constructed data drops or API pulls.

Optional Client Provided Data

• Price and promotion indexes: We use line item level data, either pulled from an API (for Shopify-based businesses) or provided directly by the client, to build a custom index that is a weekly weighted average price representing the whole product catalog that factors in discounts and accounts for individual product purchase frequency.
• Custom variables: Any non-media variable(s), beyond those included natively, provided by the client for inclusion in the MMM. Example custom variables include brand indexes, owned social media, and competitive spend.

Specifications for the above inputs are available upon request.

Measured Provided Data

• Holidays: A list of standard national and cultural holidays for each country. Automatically tested for inclusion in every model.
• Seasonality index: The seasonality index is a blend of a client-specific index derived from the dependent variable and an industry index. If a full 3 years of data is available, the industry component has minimal impact, but when data is limited the industry index carries more weight and provides necessary stability. No additional data is required. Automatically included in every model.

Macroeconomic indexes: The default indexes are unemployment rate and consumer price (CPI), but additional indexes can be included as needed. Indexes are pulled into the model natively from FRED, Statistics Canada, and OECD.

Model-level

To achieve the best results, the daily model run, used to produce the media insights, leverages all available data (up to 3 years). Separately, but also daily, we run an iterative back-testing procedure to quantify model quality and stability. For each back-test iteration, we compute out-of-sample normalized root mean squared error (NRMSE) and mean absolute percentage error (MAPE) metrics on the observations not included in model training. This back-testing procedure is executed across 3 different holdout lengths: 8, 12, and 16 weeks, to ensure a comprehensive assessment.

Tactic-level

Since the model-level evaluation is based on predictive, and not inferential, accuracy, we evaluate each media tactic individually using six key factors:

• P-value: The probability of observing the inputted data—or data more extreme—assuming the tactic has no true effect on business performance.
• Relative confidence interval width: A measure of the precision and reliability of each coefficient estimate.
• VIF: A measure of the multicollinearity associated with each tactic (how strongly it overlaps with the other predictors).
• Sufficiency: The percentage of weeks for which each tactic has data.
• Alignment: Relevant only when the tactic is modeled using platform orders. It is the ratio of weeks with platform orders to weeks with spend. Identifies potential tracking issues.
• Calibrating geo study: The risks associated with the previous metrics can be mitigated by a calibrating geo study.

Qualitative

Along with the quantitative model evaluation, the model results are rigorously reviewed and validated by experts. This adds another important dimension to the review process. While predictive accuracy is important, it does not guarantee that the results fully capture business context or reflect media strategy realities. That’s where the expert validation comes in. Our team routinely reviews model results, especially at initial setup and after major test results, to ensure they make sense within the context of the client’s business and relative to industry benchmarks.

• MIM is run daily.
• Those daily results are available for review on the Measured Incrementality Model UI.
• Results are applied to the product more broadly (e.g., Cross-Channel Dashboard and MPO) on a client-specified cadence. This gives users the ability to review every model run without immediately committing each run to downstream reporting and planning tools.
• Much of the configurability mentioned throughout this document, including incorporating non-media variables, including / excluding geo study results, and selecting tactic-specific modeling inputs, is done via the UI. Any configuration not yet available in the UI can be implemented by request through the customer success (CS) team.