The Magic of Google Analytics
When I first got into digital marketing around 2009, I was blown away by this rather novel (at the time) tool called “Google Analytics,” or “GA” as is the industry shorthand. This remarkable software allowed you to understand what people were doing on your website and what drove them there. And…it was free. I would argue that one would be hard-pressed to find a more influential piece of software that costs zero dollars.
But what’s the old adage? How about: “If it’s too good to be true, it probably is.” I believe this applies quite aptly to GA, and I’ll explain why shortly.
But first, if anyone thinks this is going to be a teardown of Google Analytics, they would be only 50% correct. I do think GA is a wonderful, incredible, brilliant piece of software – for web analytics.
However, GA is also probably singularly responsible for billions and billions of dollars of wasted ad spend when used (as it is by far too many companies) as the single source of truth for marketing analytics. While Google is the beneficiary of some of the misallocated ad dollars, they have not intentionally designed the system this way. Rather, marketers have been seduced by the seemingly simple and clean way that GA presents marketing analytics and have closed their eyes and plugged their ears against the clear message: GA (and similar tools) are simply not scientific and are prone to misrepresenting marketing performance – often vastly. They must be complemented by other tools like incrementality testing and MMM.
Download our incrementality guide for marketers here to learn more.
Web Analytics vs. Marketing Analytics
Let’s take a minute to quickly break down the difference between web and marketing analytics.
Web analytics refers to understanding users’ behaviors on a website - where they spend the most/least time, what causes them to “bounce” (leave), how various UX treatments influence the actions users take or don’t take, etc.
By the way, it goes so much deeper than this. As I mentioned before, there really isn’t another piece of software that comes close to the value/money proposition offered by GA in regard to web analytics. There’s no denying that Google Analytics excels in providing deep insights into web traffic and user-website interactions.
Marketing analytics, on the other hand, refers to the analysis of one’s marketing campaigns across channels like Instagram, Google Search, YouTube, TikTok, affiliates, etc. Google Analytics does do this…kind of. And this is where things get tricky. GA does a great job of simplifying marketing performance assessment, and the insights displayed are easy to understand (at least at face value). Take a look at an example report from GA here to see this in action:
- Asymmetric data availability
- False sense of security in measurement methodology
The Limitations of GA for Marketing Analytics
Asymmetric Data Availability
Google Analytics is owned and operated by…you guessed it - Google. This means that if you use ONLY advertising on Google properties, marketing analytics in GA will work really, really well...as far as non-causal methodologies go (which we’ll touch on in the next section). You’ll have essentially a full purview on all clicks and views of ads within the Google ecosystem.
But naturally, many marketers spend lots of money buying ads on non-Google walled gardens. Whether you opt to spend on Meta, TikTok, or another platform, the fact is that, in most cases, you’re unable to get view-only ad exposures into GA from such sources.
Consider an example: Assume a click-through rate of 1-3% on ads. Not seeing views in GA means you’d be missing 97%-99% of your ad exposures! You’re not mistaken if you think that seems like a lot…
Let’s go further. Imagine spending $10,000 at a $5 CPM. This is 2,000,000 impressions. Say your frequency is 4 ads per person, so that’s 500,000 people. If we estimate that 2% of people who see the ad click on it, GA would not have data for 490,000 people that your ads reached. You would also be unable to assess the impact of ~$9,800/$10,000 of your ad spend on that channel.
Essentially, Google Analytics has full visibility into Google ads, which is fantastic for Google ads. However, since they only provide visibility into clicks on ads on most walled gardens, this means relying on just GA results in you, as a marketer, missing out on the impact of large swaths of ad spend.
False Sense of Security in Measurement Methodology
Historically, attribution across touchpoints occurred by looking at all ad interactions that preceded a conversion and fractionally distributing the credit for that conversion over those touchpoints according to some business logic. Common examples of this are below (each bar represents the credit assigned to an ad interaction):
- Data-driven attribution
- Paid and organic last-click
- Google paid channels last-click
In other blog posts and podcasts, I have explained the failures of last-click. For this writing, I’d like to focus on Data-driven Attribution (DDA). According to the GA help center, DDA works as follows:
“Attribution uses machine learning algorithms to evaluate both converting and non-converting paths. The resulting Data-driven model learns how different touchpoints impact key event outcomes. The model incorporates factors such as time from key event [Google’s new name for “Conversion”], device type, number of ad interactions, the order of ad exposure, and the type of creative assets. Using a counterfactual approach, the model contrasts what happened with what could have occurred to determine which touchpoints are most likely to drive key events. The model attributes credit to these touchpoints based on this likelihood.”
As someone who is deeply passionate about applying science to marketing, this initially gave me some hope, especially after reading the next paragraphs:
“Data-driven attribution uses path data, including data from both converting and non-converting users, to understand how the presence and timing of particular marketing touchpoints may impact your users’ probability of key event. The resulting models assess how likely a user is to interact with a key event on the path to a key event at any particular point in the path, given exposure to a particular ad interaction.
The models compare the key event probability of users who were exposed to the ad, to the key event probability of similar users in a holdback group. (In more technical terms, the models compute the counterfactual gains of Google ad exposures by training on data from randomized controlled trials.)”
However, there isn’t much more readily available that describes this in more detail. How are the “randomized controlled trials” constructed? How well-powered are these tests, and how does GA handle statistical significance?
Upon further research, I came across this paper from Google, which appears to describe the methodology that DDA employs. If this is what DDA in GA employs, then it does not use randomized controlled trials. Rather, it uses somewhat of a “natural experiment,” which is defined by Google’s AI as follows:
“A naturalistic experiment, also known as a natural experiment or naturalistic design, is a research method that involves observing a situation without influencing it, and then comparing the current condition to another condition. Researchers often use naturalistic experiments when controlled experimentation isn't possible, such as when there are policy changes, weather events, or natural disasters.”
From the paper cited above, we see how this may be applied in GA’s DDA. Essentially, GA looks for groups of identical conversion paths, and compares conversion behavior to other groups with that exact same path, but without a particular marketing touchpoint.
In the paper mentioned previously, Google calls out three main issues with DDA that impact the algorithm’s ability to accurately estimate incrementality. Those are:
- Inconsistent targeting across marketing channels (the issue I highlighted above)
- No visibility into conversions from folks who have no ad interactions.
- No ability to control for different browsing behaviors and opportunity to see ads.
All of these make DDA a directional tool at best. This methodology would not cut the mustard at some of the top-performance marketers I’ve worked with. As such, using DDA comes at a peril to marketers. As simulations in the paper show, the algorithm will likely consistently over-/under-estimate the true efficacy of a channel. However, the situation becomes more complex when there are biases/errors across multiple channels.
What could this mean for marketers? DDA could lead to misallocating spend across channels and, additionally, could call for drastically over- or under-spending in aggregate relative to the correct spend level to achieve an organization's specific profitability and growth goals.
Regarding Google's measurement methodology, you have two choices: Wrong and honest about it (last click) or Wrong and hiding it (DDA).
How To Use Google Analytics Effectively
Google Analytics is a wonderful tool. It has helped analyze millions of websites and likely (without exaggeration) plays a meaningful role in bolstering the global economy in this regard. However, when relied upon as a marketing analytics tool, it may be responsible for the equally meaningful deterioration of the global economy. This isn’t GA’s fault, nor is it some nefarious plot by Google. The responsibility for GA’s misuse as the infallible source of truth for marketing analytics lies squarely on the shoulders of marketers.
I’ve said it before, and I’ll repeat it again: marketing analytics isn’t tidy; if you try to make it so, you’re probably doing it wrong.
Google Analytics does a great job of making marketing efficacy analysis look deceptively tidy. While this is a great way to get new, smaller marketers off the ground and possibly can provide directional guidance on performance, as a marketing organization’s sophistication increases, so too does the need for greater precision.
Most marketers I speak to today that are GA zealots are well past the point of being able to use this tool as their source of ad efficacy truth. Alas, these folks suffer from a cocktail of lack of understanding, corporate inertia, and laziness. Don’t be one of them - if you use GA, ensure you understand its limitations, and make sure you’re using real science to understand your marketing efforts.