The Biggest Mistake Digital Marketers Ever Made: Claiming to Measure Everything

Posted by willcritchlow

Digital marketing is measurable.

It’s probably the single most common claim everyone hears about digital, and I can’t count the number of times I’ve seen conference speakers talk about it (heck, I’ve even done it myself).

I mean, look at those offline dinosaurs, the argument goes. They all know that half their spend is wasted — they just don’t know which half.

Maybe the joke’s on us digital marketers though, who garnered only 41% of global ad spend even in 2017 after years of strong growth.

Unfortunately, while we were geeking out about attribution models and cross-device tracking, we were accidentally triggering a common human cognitive bias that kept us anchored on small amounts, leaving buckets of money on the table and fundamentally reducing our impact and access to the C-suite.

And what’s worse is that we have convinced ourselves that it’s a critical part of what makes digital marketing great. The simplest way to see this is to realize that, for most of us, I very much doubt that if you removed all our measurement ability we’d reduce our digital marketing investment to nothing.

In truth, of course, we’re nowhere close to measuring all the benefits of most of the things we do. We certainly track the last clicks, and we’re not bad at tracking any clicks on the path to conversion on the same device, but we generally suck at capturing:

• Anything that happens on a different device
• Brand awareness impacts that lead to much later improvements in conversion rate, average order value, or lifetime value
• Benefits of visibility or impressions that aren’t clicked
• Brand affinity generally

The cognitive bias that leads us astray

All of this means that the returns we report on tend to be just the most direct returns. This should be fine — it’s just a floor on the true value (“this activity has generated at least this much value for the brand”) — but the “anchoring” cognitive bias means that it messes with our minds and our clients’ minds. Anchoring is the process whereby we fixate on the first number we hear and subsequently estimate unknowns closer to the anchoring number than we should. Famous experiments have shown that even showing people a totally random number can drag their subsequent estimates up or down.

So even if the true value of our activity was 10x the measured value, we’d be stuck on estimating the true value as very close to the single concrete, exact number we heard along the way.

This tends to result in the measured value being seen as a ceiling on the true value. Other biases like the availability heuristic (which results in us overstating the likelihood of things that are easy to remember) tend to mean that we tend to want to factor in obvious ways that the direct value measurement could be overstating things, and leave to one side all the unmeasured extra value.

The mistake became a really big one because fortunately/unfortunately, the measured return in digital has often been enough to justify at least a reasonable level of the activity. If it hadn’t been (think the vanishingly small number of people who see a billboard and immediately buy a car within the next week when they weren’t otherwise going to do so) we’d have been forced to talk more about the other benefits. But we weren’t. So we lazily talked about the measured value, and about the measurability as a benefit and a differentiator.

The threats of relying on exact measurement

Not only do we leave a whole load of credit (read: cash) on the table, but it also leads to threats to measurability being seen as existential threats to digital marketing activity as a whole. We know that there are growing threats to measuring accurately, including regulatory, technological, and user-behavior shifts:

• GDPR and other privacy regulations are limiting what we are allowed to do (and, as platforms catch up, what we can do)
• Privacy features are being included in more products, added on by savvy consumers, or simply being set to be on by default more often, with even the biggest company in the world touting privacy as a core differentiator
• Users continue to increase the extent to which they research and buy across multiple devices
• Compared to early in Google’s rise, the lack of keyword-level analytics data and the rise of (not provided) means that we have far less visibility into the details than we used to when the narrative of measurability was being written

Now, imagine that the combination of these trends meant that you lost 100% of your analytics and data. Would it mean that your leads stopped? Would you immediately turn your website off? Stop marketing?

I suggest that the answer to all of that is “no.” There's a ton of value to digital marketing beyond the ability to track specific interactions.

We’re obviously not going to see our measurable insights disappear to zero, but for all the reasons I outlined above, it’s worth thinking about all the ways that our activities add value, how that value manifests, and some ways of proving it exists even if you can’t measure it.

How should we talk about value?

There are two pieces to the brand value puzzle:

1. Figuring out the value of increasing brand awareness or affinity
2. Understanding how our digital activities are changing said awareness or affinity

There's obviously a lot of research into brand valuations generally, and while it’s outside the scope of this piece to think about total brand value, it’s worth noting that some methodologies place as much as 75% of the enterprise value of even some large companies in the value of their brands:

Image source

My colleague Tom Capper has written about a variety of ways to measure changes in brand awareness, which attacks a good chunk of the second challenge. But challenge #1 remains: how do we figure out what it’s worth to carry out some marketing activity that changes brand awareness or affinity?

In a recent post, I discussed different ways of building marketing models and one of the methodologies I described might be useful for this - namely so-called “top-down” modelling which I defined as being about percentages and trends (as opposed to raw numbers and units of production).

The top-down approach

I’ve come up with two possible ways of modelling brand value in a transactional sense:

1. The Sherlock approach

“When you have eliminated the impossible, whatever remains, however improbable, must be the truth."
- Sherlock Holmes

The outline would be to take the total new revenue acquired in a period. Subtract from this any elements that can be attributed to specific acquisition channels; whatever remains must be brand. If this is in any way stable or predictable over multiple periods, you can use it as a baseline value from which to apply the methodologies outlined above for measuring changes in brand awareness and affinity.

2. Aggressive attribution

If you run normal first-touch attribution reports, the limitations of measurement (clearing cookies, multiple devices etc) mean that you will show first-touch revenue that seems somewhat implausible (e.g. email; email surely can’t be a first-touch source — how did they get on your email list in the first place?):

Click for a larger version

In this screenshot we see that although first-touch dramatically reduces the influence of direct, for instance, it still accounts for more than 15% of new revenue.

The aggressive attribution model takes total revenue and splits it between the acquisition channels (unbranded search, paid social, referral). A first pass on this would simply split it in the relative proportion to the size of each of those channels, effectively normalizing them, though you could build more sophisticated models.

Note that there is no way of perfectly identifying branded/unbranded organic search since (not provided) and so you’ll have to use a proxy like homepage search vs. non-homepage search.

But fundamentally, the argument here would be that any revenue coming from a “first touch” of:

• Branded search
• Direct
• Organic social
• Email

...was actually acquired previously via one of the acquisition channels and so we attempt to attribute it to those channels.

Even this under-represents brand value

Both of those methodologies are pretty aggressive — but they might still under-represent brand value. Here are two additional mechanics where brand drives organic search volume in ways I haven’t figured out how to measure yet:

Trusting Amazon to rank

I like reading on the Kindle. If I hear of a book I’d like to read, I’ll often Google the name of the book on its own and trust that Amazon will rank first or second so I can get to the Kindle page to buy it. This is effectively a branded search for Amazon (and if it doesn’t rank, I’ll likely follow up with a [book name amazon] search or head on over to Amazon to search there directly).

But because all I’ve appeared to do is search [book name] on Google and then click through to Amazon, there is nothing to differentiate this from an unbranded search.

Spotting brands you trust in the SERPs

I imagine we all have anecdotal experience of doing this: you do a search and you spot a website you know and trust (or where you have an account) ranking somewhere other than #1 and click on it regardless of position.

One time that I can specifically recall noticing this tendency growing in myself was when I started doing tons more baby-related searches after my first child was born. Up until that point, I had effectively zero brand affinity with anyone in the space, but I quickly grew to rate the content put out by babycentre (babycenter in the US) and I found myself often clicking on their result in position 3 or 4 even when I hadn’t set out to look for them, e.g. in results like this one:

It was fascinating to me to observe this behavior in myself because I had no real interaction with babycentre outside of search, and yet, by consistently ranking well across tons of long-tail queries and providing consistently good content and user experience I came to know and trust them and click on them even when they were outranked. I find this to be a great example because it is entirely self-contained within organic search. They built a brand effect through organic search and reaped the reward in increased organic search.

I have essentially no ideas on how to measure either of these effects. If you have any bright ideas, do let me know in the comments.

Budgets will come under pressure

My belief is that total digital budgets will continue to grow (especially as TV continues to fragment), but I also believe that individual budgets are going to come under scrutiny and pressure making this kind of thinking increasingly important.

We know that there is going to be pressure on referral traffic from Facebook following the recent news feed announcements, but there is also pressure on trust in Google:

• Before the recent news feed changes, slightly misleading stories had implied that Google had lost the top spot as the largest referrer of traffic (whereas in fact this was only briefly true in media)
• The growth of the mobile-first card view and richer and richer SERPs has led to declines in outbound CTR in some areas
• The increasingly black-box nature of Google’s algorithm and an increasing use of ML make the algorithm increasingly impenetrable and mean that we are having to do more testing on individual sites to understand what works

While I believe that the opportunity is large and still growing (see, for example, this slide showing Google growing as a referrer of traffic even as CTR has declined in some areas), it’s clear that the narrative is going to lead to more challenging conversations and budgets under increased scrutiny.

Can you justify your SEO investment?

What do you say when your CMO asks what you’re getting for your SEO investment?

What do you say when she asks whether the organic search opportunity is tapped out?

I’ll probably explore the answers to both these questions more in another post, but suffice it to say that I do a lot of thinking about these kinds of questions.

The first is why we have built our split-testing platform to make organic SEO investments measurable, quantifiable and accountable.

The second is why I think it’s super important to remember the big picture while the media is running around with hair on fire. Media companies saw Facebook overtake Google as a traffic channel (and then are likely seeing that reverse right now), but most of the web has Google as the largest and growing source of traffic and value.

The reality (from clickstream data) is that it's really easy to forget how long the long-tail is and how sparse search features and ads are on the extreme long-tail:

1. Only 3–4% of all searches result in a click on an ad, for example. Google's incredible (and still growing) business is based on a small subset of commercial searches
2. Google's share of all outbound referral traffic across the web is growing (and Facebook's is shrinking as they increasingly wall off their garden)

The opportunity is for smart brands to capitalize on a growing opportunity while their competitors sink time and money into a social space that is increasingly all about Facebook, and increasingly pay-to-play.

What do you think? Are you having these hard conversations with leadership? How are you measuring your digital brand’s value?

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!