Geo-Testing Series Part 2: Test Management

Hello Growth Marketers.  I’m Andrew Covato, and I’ve spent my career building and deploying incrementality-based measurement and optimization systems for major ad buyers and sellers.  I now work with performance marketers, helping them build out scientific growth programs.  I also work with measurement platforms, but exclusively with ones doing smart things – namely, helping scale incrementality measurement – just like Measured.  Check out a bit more about me at growthbyscience.com.

Geo Test Management Overview

This is part two of a three-part blog series on geo-testing. In part one, I wrote about test setup.  Today, I’ll discuss test management, which is arguably the most crucial component of any type of marketing testing – even outside geo-testing.  Test management encompasses the following tasks:

1. Planning - what tests should be run, when, etc.
2. Deployment - ensuring that campaigns are set up with the right targeting, treatment, etc. and keeping track of this across multiple platforms
3. Storage - building a repository of past tests and their results

In my experience helping marketers of all sizes – startups with 6-figure budgets to publicly traded behemoths with growth budgets in the billions – I have seen one commonality: What can make or break a test program is a company’s ability to actually plan, execute, and store the tests.  

Challenges of Geo Test Planning

For example, an organization may be trying to understand how much budget to spend on TikTok vs. Instagram vs. YouTube. The best way to do this would be to run a multi-celled experiment where you compare the relative performance of each of the channels.   The only way to do this in a geo-testing context would be to apply geographic targeting on all campaigns across each of the channels to ensure the correct test deployment.  There is a lot of opportunity for error in doing this manually, as you’d have to do without a dedicated test platform or in-house tool. Planning a test program can be hugely challenging.  Organizations will either have no testing ideas or will have so many test ideas that it can be overwhelming.  In either case, it’s hard to take that first step.  This is why test recommendations, previewing, and planning – ideally with test spec calculation (e.g., where to run the test, how long, etc.) can be huge.   By far, the most onerous part of test management is actually setting up the campaigns.  There are many reasons for this: 

1. Geo tests require extremely precise targeting to be overlaid on go-live campaigns.  This is a very time-intensive task to do manually.
2. Usually, marketing managers are the only folks with hands-on knowledge of how to set campaigns up across the various platforms…and these are very busy folks.  They have to handle business-as-usual campaigns in addition to testing.
3. Tests have to be monitored to ensure they are capturing the information needed correctly. A botched test means lost time, money, and information; often, tests are run to guide a larger campaign.

I’ve seen beautiful test agendas pared down into much more high-level and rudimentary initiatives because they have been way too hard to set up and manage.  Also, there is real overhead required in long-term test planning and storing results, and often this exists in a messy spreadsheet buried in someone’s cloud drive or laptop with a not-so-descriptive name like “mrkt_test_plan02042022_full(1)(2).xlsx”.  

Setting Up for Success

As your growth organization scales and as your strategies become more complex, the very process of running a test agenda can be extremely resource-intensive if done manually.  This is where platforms like Measured can be real time-savers.  When evaluating a testing platform, make sure you look for easy test design, automated API-based test deployment (i.e., a product that plugs into major ad platforms), and automated results repository. TL/DR Here is what test management looks like as an organization scales.

Next time we’ll talk about the cornerstone of testing: analytics & reporting.  We’ll dive into the guts of how testing is conducted and what types of outputs can be generated. Stay tuned for part three!  Geo-testing is a powerful tool for revealing the true impact of your media investments, but not all tests are created equal. How mature is your brand’s testing practice? Download the geo-testing scorecard here to find out.

Interested in learning more? Download our comprehensive ebook, 'The 3 Essential Steps to Geo-Testing Like a Pro'.