Understanding Correlation vs Causation: Why ChatGPT and AI Algorithms Alone Are Not Enough

By now, chances are good that you have not only heard of but also likely tried out ChatGPT.

While unquestionably an advance that stands to shake up search – no small feat – it’s worth leveling up understanding about what exactly this technology does and what it does not. Much of the commentary following its release suggesting the singularity is nigh is off-base.

Fundamentally, the program still uses tried-and-true algorithmic practices of pattern-matching big data and predicting outcomes from it. What ChatGPT and other "Narrow AI" programs lack is the ability to understand correlation versus causation, and no incremental amount of data fed into a model will give the machine this ability.

As it relates back to our world of paid media, it's worth noting that ChatGPT functions similarly in nature to how your audience targeting algorithms do. They build up a data pattern of your customer profile and then look into their big data set to identify a similar pattern/customer. Because causation is missing from the algorithm, the model can never understand why a certain ad, time-of-day delivery, creative format used, or type of consumer served drove a conversion for you. Some argue, compellingly, we may never understand the inner workings of these black-box models: "Is Artificial Intelligence Permanently Inscrutable?" - Nautilus

“What machines are picking up on are not facts about the world,” Dhruv Batra, assistant professor at Virginia tech, says. “They’re facts about the dataset.” That the machines are so tightly tuned to the data they are fed makes it difficult to extract general rules about how they work.

Instead of identifying root causes that drove customers to buy, an ad algorithm is entirely focused on identifying the pattern of data that best matches with your conversion profile. What that ends up looking like in practice is identifying existing or in-market demand (see: Parachute Home Finds They Can Cut Tactic Spend With No Impact). Furthermore, when given a CPA goal, these algorithms will purchase ads toward this audience typically with the lowest cost (and thus lower quality) ads. These kinds of ads (low cost/quality), to these kinds of customers (already en route to buy), represent the best (but fictional) CPA result.

Historic advertising attribution models (MTA, Last Touch, Platform Reporting) unfortunately reinforce this algorithmic behavior and create a negative feedback loop, rewarding, and in turn encouraging, the algorithm to deliver the least causal results. Guiding investment from these attribution methods, you’ll be pushing toward less and less desirable outcomes for your business and, in time, will be left wondering why ad performance reports look so terrific while the top-line revenue of the business is suffering. Rewarding correlation-driven algorithms focused solely on a data pattern, rather than a composite of causal factors, is why there’s a disconnect.

And this is in line with wider AI use-cases. Some may recall a lot of hype generated back in 2016 when DeepMind’s AlphaGo beat several master-level Go players. AlphaGo had been widely considered the most advanced form of narrow AI. While it didn’t receive nearly as much attention, we recently discovered AlphaGo is penetrable after all: "Amateur Beats AlphaGo." How did it lose? Namely, an inability to generalize in a way that humans find easy. The system can understand only very specific situations its dataset has been exposed to in the past. Said another way: it’s fantastic at correlation and completely unaware that causation even exists.

The wider takeaway here is that Terminators aren’t quite yet a threat, and current "AI" is still very much technology that will be job-enhancing at best, rather than eliminating. The more narrow takeaway, for those of us in paid media, is that you have to push back against the tide of correlational targeting algorithms.

Measured is a company founded in 2017 with the belief that test vs control experimentation is the way forward in attribution to solve for precisely the causation question: did the ad cause the sale, or would the sale have happened anyway? The company has put years of work into building a platform that automates and expedites a once laborious experimentation process. Measured's platform allows its customers to deploy media experiments faster and more accurately than their competition. Thanks to Measured, companies can make data-driven decisions to optimize their ad spend and maximize their return on investment. In fact, Measured's customers have reported significant success, with some achieving sales growth while cutting spend by as much as 30%. To learn more about Measured and how its platform can help your business, schedule a personalized demo today.