Testing Times: Retesting Incrementality Under New Economic Conditions
Michael Taylor, Founder, Vexpower Attribution Expert
Published 08/15/2023
When economic conditions change, so does your marketing performance. When we’re in times of uncertainty, like now, it pays to retest the incrementality of your channels. We see similar case studies to this all the time at Measured, where a previously non-incremental channel flips in changing economic conditions (or vice versa).
As marketers, we can’t assume our strategies or results will remain static. Consumer behavior is constantly evolving along with the economy, and our marketing performance metrics change with it.
A key tool for navigating these shifts is incrementality testing: determining what sales would have been had you not spent money on advertising. By more frequently retesting incrementality, we gain insight into how our marketing channels are truly performing and can adapt our strategies accordingly. This allows us to optimize our budget and ensure strong ROI regardless of market conditions.
When Markets Change, So Should Marketing
In times of inflation, recession, or market volatility, consumer spending and behavior fluctuate. When money is tight, people scrutinize their budgets and reevaluate what they buy and how much they spend. And as their behavior and priorities change, so does our marketing incrementality.
https://en.wikipedia.org/wiki/United_States_Consumer_Price_Index
For example, say I had been investing heavily in paid search ads for a luxury product, and initial incrementality tests showed these ads were driving strong performance. But if inflation rises sharply and people cut back on discretionary spending, incrementality on those paid search ads may decrease significantly as consideration cycles increase. Upper funnel channels that generate demand where there is none, like YouTube, TikTok, or influencer campaigns, may be more incremental than usual.
On the other hand, if I market essential goods and services, incrementality of lower funnel channels like paid search could increase during economic downturns as consumers become more responsive to promotional offers and hunt for the best value. They may have less patience for ‘brand’ campaigns and want to cut straight to the chase, so they can get onto their bigger problems rather than vacillating over what brand of toilet roll to buy.
The lesson here is that as market conditions change, we must retest incrementality to understand how these changes are impacting our marketing channels. What was once working and driving strong results may no longer be effective. By identifying these shifts early through fresh incrementality testing, we can make data-driven decisions to reallocate budget to the highest-performing channels.
A Case Study: How Unemployment Rates Affected Brand Search Incrementality
https://tinuiti.com/blog/paid-search/paid-branded-search/
To provide a concrete example of why retesting incrementality is so crucial, here’s a real case study from my experience the last time we had a recession, in 2009. With a travel client we switched brand ads off and on over four weeks and found that brand search ads were driving 20% incrementality. That means whenever Google reported $1 in revenue for someone searching for our brand keyword and clicking through to buy, we would have still gotten 80 cents of that revenue if we weren’t running ads. We turned off brand ads and redistributed the budget.
However, over the next 12 months, unemployment rates in the target market halved as the country recovered from the global financial crisis. Given this significant economic change, we had to retest incrementality. Actually, I can’t claim foresight here; what actually happened is the CEO called me up and demanded we re-test it because he wanted to push more budget to make his end-of-quarter numbers. This time when we switched on brand search, we found incrementality had increased to 40%, making the campaign economically viable again!
With more people employed and earning an income, a higher percentage of visitors coming to the website were ready to book vacations. At the same time, competitors had intensified their efforts to capture the growing customer base. This meant that if our ad didn’t capture the click immediately, potential customers were more likely to turn to a competitor's site or offer. Our theory at the time was that consumers were more prone to nicer imagery and more evocative ad copy of our competitors when making instant gratification purchases.
Whatever the particulars of your business, it’s clear to see how changes in economic conditions can have a sizable impact on marketing performance and incrementality. By retesting incrementality, we were able to catch this significant shift in performance early and adjust the strategy to drive better results within the new economic climate. The lesson here is to test often, especially during times of economic uncertainty or change.
When to Increase the Frequency of Incrementality Testing
Under normal conditions, for companies with an annual marketing budget in the $1M to $10M range, which is where my experience lies, I recommend testing the incrementality of primary marketing channels at least quarterly and secondary channels annually. For example, if you spend $2M a year on ads as a travel company, you might find 65% of that budget goes to Google Ads, with 25% to Facebook Ads, 5% to Bing, and another 5% to other small bets – a ‘pareto’ distribution.
In normal times, I wouldn’t test Bing or small bets at all, but Facebook I’d test annually and Google quarterly. During periods of economic volatility or recession, however, testing frequency should roughly double, so once every 45 days for a primary channel and twice a year for secondary channels. It may even be important to run a test on smaller channels as well as individual campaigns if you suspect changes in incrementality like we did with brand search ads. It’s especially important to ramp up incrementality testing when:
- You notice lower overall revenue or conversion rates with stable or higher ad spend. This could signal decreasing incrementality, meaning ads are less effective, and you’re wasting budget.
- There is a sudden surge in view-through conversions. While view-throughs indicate your ads are still reaching prospects, a big jump could mean you’re reaching more low-quality traffic. Additional incrementality tests will provide clarity.
- There is a high level of uncertainty in your marketing mix models. Incrementality testing can provide concrete insights into how channels are performing so you can optimize your budget and improve modeling.
By ramping up incrementality testing when key marketing metrics change significantly or uncertainty increases you'll gain the insights needed to understand how changing economic conditions are truly impacting performance. With this knowledge comes the power to make data-driven optimization decisions and allocate budget to the most effective channels.
Conclusion
In both marketing and commerce, few things remain static. As the economy fluctuates, so do consumer priorities, habits, and your results. But by making incrementality testing a consistent practice and increasing frequency during times of change, you can navigate uncertainty more effectively.
To learn more about incrementality testing, schedule a demo today.
