What’s the Best Approach for Media Performance Measurement?

Best Approach

Clay Cohen, VP of Marketing

Published 04/02/2024

In a recent blog, we demonstrated the necessity and value of aligning your marketing testing practice with your broader business strategy and goals. But what if you don’t have a testing practice already in place? Now, we want to move on to the next step: How do you know if you should build a media measurement and testing practice with in-house resources, find a measurement partner, or land somewhere in between?

There are four common approaches to measurement tooling we see in the market today:

1) Do nothing: rely on ad platform conversion and attribution data from each vendor
2) Start from scratch and build something internally 
3) “Frankenstein it” – partially build & buy
4) Find a measurement partner

Option One: Ad Platform Data

Pros: Low Resources and Cost

As any advertiser knows, every platform (Google, Meta, TikTok, etc.) has its own dashboard that provides performance data, and relying on these platform-provided metrics is easy. Due to the simplicity of this method, it could be a viable option for brands with small marketing teams or limited resources to assess the efficacy of their ads. 

But let’s be clear - this is our least recommended option. 

Cons: High Risk

Letting the platforms grade their own homework does not come without risk to your business. The metrics surfaced in ad platforms are based on a correlative methodology that doesn’t accurately reflect the true performance of campaigns - it’s designed to assign maximum attribution credit to the ads you run on their platforms. Reported data from media platforms are based on post-exposure attribution, typically some combination of post-click and post-view with attribution “windows” like 7/1.

This approach rewards campaigns that can show ads to individuals with a high intrinsic propensity to convert – typically after they view lower-funnel ads that are more easily tracked within the short post-click and post-view windows. This approach is not optimal for advertisers who seek to acquire new, incremental customers who would not have otherwise converted. Platform measurement most often leads to brands undervaluing upper-funnel awareness media because the platforms are not able to track clicks and views earlier in the consumer journey outside of their attribution windows.

It’s all too common for us to encounter brands that have been allocating spend down-funnel as a result of decisions based on platform-reported data. Ad campaigns such as retargeting tend to look favorable on platform dashboards, slowly absorbing essential spend from upper-funnel awareness channels, ultimately shrinking the brand’s ability to acquire new customers. Neutral, incremental measurement is the only way to accurately measure ad spend across the entire funnel, often demonstrating the real value of upper-funnel media.

Additionally, most marketers are aware that post-exposure, click/view-based attribution numbers are not apples to apples and never seem to align from platform to platform. In a recent case study, one of our customers found a large discrepancy between Meta’s in-platform metrics and the value that Google Analytics was assigning to the same Meta campaigns. After measuring the actual contribution using incrementality tests, they found that both ad platforms were misreporting the value of the campaigns. 

While there are minimal technical resources required to utilize platform metrics (beyond passing the data through an API, pixel, or SDK), this option puts full faith in the ad platforms’ ability to accurately self-report campaign performance, using the ever-shrinking pool of user identifiers, and often requires marketers to make guesses about which numbers they should be trusting.

Even presuming the best of intentions on their parts, platform metrics rely on a correlative methodology - just because someone saw an ad and made a purchase doesn’t mean the ad caused the purchase - and furthermore, are extremely sensitive to the loss of identifiers like IDFA and cookies. Thus, relying on platform metrics will likely lead to both wasted time and suboptimal ad spend down the road for most brands.

Option Two: Building Internally

The thought of building a truly bespoke solution tailored to the nuances of your organization can be very tempting. However, before you embark on building your own measurement tooling, take a minute to consider the following.

Pros: Customization

Let’s start with the upsides: building internally means your solution is 100% tailored to your business. If your organization has unique requirements regarding experimentation methodology, needs fine control over MMM models, or has a very non-standard data pipeline, building bespoke could be a path to consider.

As a foundation, this option requires a significant data investment. This includes tracking, ingesting, and storing your own business outcomes (i.e., sales, lead, conversion, etc.) data from your CRM, site, or storefront, as well as ad exposure data from your media channels. Then on top of this unified data set, the real work begins - developing a measurement system to reveal the causal relationship between advertising and sales.

Cons: High Cost, Difficulty, and Risk

The first challenge is that data from ad platforms like Google and Meta is not standardized and isn’t necessarily formatted to suit marketing experimentation or MMM. Because of this, in addition to building the basic API connections to your ad platforms, you will likely also need to construct an ETL (Extract, Transform, Load) process to standardize the data. 

Further, you will need to build an appropriate internal or cloud storage system for your data in a manner that enables all of the required statistical calculations to carry out your actual measurement. We recently calculated all the costs a brand would endure when building a comprehensive data ingestion framework to organize all media, CRM, and sales data. It comes to nearly $500,000 in start-up costs for a typical mid-size brand.

Finally, you must build and deploy the actual measurement models (e.g., synthetic control for geo testing, Bayesian MMM with a process for updating priors, etc.).  While plenty of open-source tools and academic papers explain these processes, the difficulty of standing up something automated and scalable should not be underestimated. 

In parallel to the above processes, you must also develop a procedure for spec’ing and deploying your measurement solution, likely via a geo test.  You must pull from historical data (that you’ve extracted above) to determine the correct setup for your tests through standard power calculation or more sophisticated simulation-based market selection. And finally, you must deploy the test.

That is, you must actually ensure that the appropriate geo-targeting is applied to the correct channels so that ads are deployed where they should be. This can be done manually, but that’s prone to human error – especially in a scenario where there are many channels to manage and simultaneous tests being deployed. Building automated deployment requires integration with the various ad APIs (where available) and a deep “ad tech” understanding, as you’d essentially be building a lightweight API-based DSP.  To learn more and compare different levels of geo-testing practices, read our guide here.

Building all of this infrastructure from the ground up can incur a certain degree of business risk because you would be deploying live campaigns and pausing others, ultimately resulting in some loss in sales. Even a fully built-out solution with a dedicated support team should expect quarters - or even years - of iteration to iron out the kinks of your internal measurement practice because, at the end of the day, even once you’ve established a practice, you have to derive useful, actionable insights that instill confidence across your team, from marketing, to finance, to the C-suite.

Cost is another factor to consider; while this will be different for every company according to their resource availability, there are a few things you will definitely need. 

Companies should plan on a team of dedicated engineers and data scientists, at least one to three PMs specialized in marketing analytics and ad tech (depending on how fully built out your solution may be), and a deep cross-functional interaction program with your agency or internal media buying group.

This isn’t something that can be “dabbled” into either - it’s an ‘all-or-nothing’ investment, or you’ll be left with a half-baked solution that requires a ton of manual support, possibly sewing mistrust between marketing and finance.

In short, this route is very difficult and resource-intensive, both technically and operationally, and poses a higher level of risk than some other options.

data infrastructure cost

Option Three: “Frankenstein” it

Pros: Medium Difficulty

There is a compromise to build and buy that many companies choose to pursue, as it’s easier than building from scratch; you can outsource some of the more technical pieces, and build other components internally. 

For instance, there are services that can extract ad data from various platforms and provide it to you. There are even other buying tools that can make deploying the measurement tests a little bit easier on your team. Additionally, some partners, like Measured, can offer sophisticated geo-testing tools to supplement, calibrate, or validate your existing mix modeling practice.

Cons: Unknown Cost

But of course, many of the tools and services required are not built bespoke for the purpose of measurement, and some may pose integration challenges as well. Even if you outsource these components of your measurement stack, maintenance remains very difficult and requires a dedicated internal team to document, manage, and produce results.

Similar to option two, hidden costs should be considered when going this route. In this case, you are highly dependent on the expertise that you pay for, and additional resources may be needed at the last minute. You will likely also need some dedicated internal resources (like a PM) to oversee the whole system. Essentially, it’s very difficult to predict what this option will cost you in terms of time and money. 

Option Four: Partner Up!

Pros: Minimal Resources and Risk, 3rd-Party Neutrality, Fast Time-to-Value

Found an experienced measurement partner? Great - you’ve minimized the learning curve and can be on your way to comprehensive measurement in a very short amount of time. 

With the right partner, you integrate, and you’re ready to go; a sophisticated partner will have meaningful automation built into the measurement and optimization technology, which cuts down on the time it takes to start getting results and optimizing your media spend. 

Cons: High Cost

Your costs with this ‘buy’ option are constant, so you don’t have to worry about hidden costs or unforeseen circumstances disrupting your budget. However, this can be an expensive option up front: finding a measurement partner who can extract ad data reliably, deploy and monitor your tests, and offer strategic insights to help you take action on all the data can cost a significant chunk of your marketing budget.

The downside of a full third-party solution is that you may have somewhat less control and flexibility, though a good partner should be able to provide enough customization to meet your business needs. The experimental methodology and models will be recommended to you by the measurement experts to suit your needs, and you’ll have to build organizational alignment to align with their methodology. Still, if you have a specific style of experimentation you simply must run or want to pick and choose a part of it, this option may not be flexible enough to accommodate your business requirements.

Starting Your Measurement Journey

If you’re here to determine which of these paths is best for you, excellent! This means you already understand the importance of experimentation and incrementality measurement and have started the journey.

The question now is: what's the easiest, quickest, and most effective way for your business to actually start measuring? Our view is that unless you're a really big performance marketing brand with a clear internal team designed to build out a bespoke measurement suite (think of the world’s largest growth marketers like Netflix and Priceline), chances are that option four is your best bet to minimize risk and maximize results.

With a partner like Measured, you know you’re getting expertise built from over seven years of experience and over 25,000 tests.  We offer you a partner who can handle the data collection, testing, and even the MMM. We've seen the challenges that come up with incrementality measurement, and we’ve overcome them. We’re here to make sure you can get up and running with high-quality, scientific measurement and start maximizing your media budget ASAP - book a demo today.