Cal Berkeley Study Validates a Learning Approach to Ad Success

Study proves an ‘always-on’ learning approach, rather than algorithms, is the biggest driver of success in advertising.

A wide-reaching academic study with strong implications for paid media professionals was just released.  If you’d rather not wade through 30+ pages of the actual study, then hopefully, a (somewhat) TL/DR will suffice.

Background of the Study

The study is based on the premise that more skillful advertising should not be excluded as a driver of alpha in business.  A number of similarly extensive studies prove that there are, in fact, stark differences in productivity levels (2x to 5x) between corporations in the same business category.  Human talent/capital, management practices, incentive structures and more have all been studied as productivity variable drivers, and not surprisingly, the more productive firms survive at significantly higher rates than their less productive counterparts.  But we haven’t seen a comprehensive study analyzing advertising and its potential impact on productivity & profitability - until now.

The study tracked 200,000 advertisers across 25 industries, deploying a total of 700,000 campaigns on Meta.  They pinned the study on website-based sales performance, and importantly (especially for us incrementality-minded), they applied Facebook’s lift testing method to the entirety of the database.  Why?  In their words:

"Most online advertisers use observational, non-experimental methods to determine the effectiveness of their advertisements, a practice plagued by endogeneity (correlation vs causation) issues, and that can only be corrected for using well-designed experiments."

So, we’re grounded in causal adjusted sales performance as our KPI.  The study's key takeaway is that we can see significant variance in how well companies buy advertising.  This means apparel brand A is seeing far greater incremental sales via Meta ads than apparel brand B, with the former recognizing 22% to 122% higher returns than the latter.

How?

The aforementioned productivity-focused studies fairly unanimously proved that firms learned by doing/experimenting, and those learnings directly equated to higher levels of future productivity.  The trend holds here.  

Sales performance was higher for the advertisers who were more engaged, active and data savvy than their competition, who were solely reliant on the Meta algorithm to do the work for them.  You can read about how they concluded an advertiser was more leaned into learning & more data-savvy within the study, but in short, they used history of ad spend, number of campaigns & adjustments, number of ads, number of embedded website pixels and advertiser age as proximities.    

The single largest driver of causal adjusted sales performance was the number of campaign updates made, and this is a powerful insight for us and worth injecting into the ongoing AI conversation.  Campaigns with few or no updates saw an initial upward trend that wasn’t sustained as the campaign aged.  The study was able to rule out consumer familiarity with the brand and algorithmic learning as drivers of long-term performance improvement.  Rather…it was good old-fashioned humans, learning & getting better with time.

Does It Matter?

Turns out yes…though admittedly, this is where the study makes its largest assumptions as they weren’t able to track external business metrics of survival.

As a proxy, they measured ad spend eight months after the study under the assumption that those firms still advertising were more likely to be succeeding than those who’d cut their ad spend entirely.  The results are as we’d suspect - the advertisers who over-indexed on greater learning & data sophistication had a higher, statistically significant probability of still advertising eight months later.  

While the study needs to make some assumptions to reach its conclusion, the findings are nonetheless powerful, especially in light of the current AI zeitgeist.  Here are the key takeaways for ad practitioners and their executive leadership teams:

1. Advertising matters - it’s a proven driver of sales growth.
2. The skill with which you place ads matters…you can outperform (or be outperformed by) your competition quite significantly along this business variable.   
3. The key to winning this competition is not to be reliant on algorithms for success but rather for the human capital in control of the ad budget to test, learn, optimize, rinse & repeat in conjunction with the algorithms.

Take that AI. 

For a deeper dive into optimizing your ad spend, schedule a demo today.