All measurements are wrong, some are useful

Key Takeaways

• Reality check: It’s not about third-party cookies, it’s about Privacy First
• All measurements are wrong, some are useful – but you already knew this
• Winners will adjust strategies, losers will adjust tactics

Reality Check

We’re in the midst of tremendous upheaval in marketing. If you think it has something to do with third-party cookies (3PC) or iOS 14, you’re suffering from what I call operational myopia. It’s what happens when you’re down in the weeds when you need to be up in the sky. When you see the trees and miss the forest. 

If you’re like most marketers, you probably spend most of your workday worrying about budgets and audiences. Asking yourself how to squeeze every last drop out of your marketing spend by measuring the benefit of channel A vs channel B, or the wording of your latest ad.

It’s not surprising that most of the articles on FLoC, Turtledove, iOS 14, etc talk of the headaches this will cause marketers. How will you measure your ads? What will Facebook do next? I’ve read several articles and watched many Youtube videos describing the changes. Many discuss, with sometimes emotional language, the devastating effects these changes will have on themselves and their clients.

You can take some solace in the fact that you’re not the only ones to experience an industry in upheaval, and certainly won’t be the last.

The trading pits of Chicago

The documentary FLOORED (now available on Youtube) describes in emotional detail the digitalisation of the Chicago trading floors and the death of the floor trader.

Once, these traders filled the trading floor with pandemonium shouting and signalling their orders to buy and sell at each other. Some traders made a good career not out of reading the markets, but the other floor traders around them.

When those traders moved out of the pit and behind computer screens across the country, they lost their edge. Some traders adjusted and found new success. Others found bitterness and calls of conspiracy.

Significant upheavals in an industry create new winners and losers. The question isn’t how to adjust tactics, but how to adjust strategies. The old floor traders didn’t find success by simply peeking in the windows of other traders, but by getting better with the computers at the core of the change.

Right now in marketing, a similar upheaval is happening. That presents a tremendous opportunity.

If you’re asking yourself how do I keep using the same tactics and tools, then prepare to be left behind. This change isn’t about cookies or mobile advertising IDs. This is about putting the customer’s Privacy First and letting them be in the driver’s seat.

More fundamentally, it’s about the survival of targeted marketing. If the industry doesn’t start putting the customer’s right to privacy front and centre, we in the industry will be legislated out of existence. Many countries give their citizens the right to privacy. I’m not aware of one country that guarantees a marketer’s right to target.

The mistake I made when I first heard about many of these changes was to equate Privacy First with an inability to perform targeted marketing. After reading some of the proposals like FLoC it’s clear to me that’s not the case. In fact, after some consideration, I’ve started to formulate some principles that I believe will separate the Privacy First winners from the losers.

All measurements are wrong, some measurements are useful

Let’s face it, you probably already knew that all your measurements were wrong.

Often this was based on just a feeling or because two measurement platforms might disagree with each other. This uncertainty allowed us to act as though our measurements were correct or at least put faith in one platform or another.

Under Privacy First, uncertainty is baked into the system and it means that you now know that your measurements are wrong, but don’t worry! All measurements are at best an approximation of reality and not necessarily correct. You already knew this implicitly, but that doesn’t mean an approximate answer isn’t useful. An approximately correct answer is much better than a precisely wrong answer.

We’ve all seen someone draw a conclusion from a measurement in an AB test with results that couldn’t possibly be significant, or apply qualitative research on 30 people to assume how 300,000 people will act.

My rule of thumb has always been that the more precise the measurement, the more likely it is to be incorrect. You could generalize this to say: the more something provides you with a sense of control over your day-to-day work, the more likely that this thing is wrong.

I even question how useful most measurements are. Covid managed to improve itself, creating several new and more effective variants, without precise population estimates, without knowing anything about planes or flight plans. Without any sort of plan really. So why can’t a marketer improve what they do without knowing that it was exactly Sally Smith from Sheffield who added the red shoes to her basket 28 days ago?

The Privacy First thrivers will be comfortable with approximate answers. They will model uncertainty in their work, and spend more time on our next topic.

Experiment wisely

If there is one area I think marketers could improve upon it’s experimentation.

I have to admit that I could also improve on this.

One of the most common mistakes I see is impatience. It often causes issues like experimenting too much or ending experiments too soon. Some of the proposals from Google add noise to the measurement process. The results of your experiments were approximate before but the systems we use allowed us to labour under the impression that they were precise.

The new systems are built on approximations. The platforms gave you the impression that their answers were exact even if we secretly knew that wasn’t the case. Now we’ll have to directly acknowledge this fact. With “noisy answers”, you’ll have to experiment less to separate signals from noise. Probably something you should have been doing already. 

Perhaps most importantly, make sure you have management support. I’ve often seen marketing teams generating action just to please senior management or to support an executive’s pet idea. In the future, you’re less likely to be successful as you’ll need more time to test ideas and find what works.

Testing a bad idea for political reasons will only ensure that your good ideas never get a chance to show their true benefit.

Embrace the change

Privacy First is an inflection point for targeted marketers. One that needs to be embraced for the industry to survive. It doesn’t matter what Facebook will do next.

They may try to get around the iOS 14 changes by, say connecting more people via social logins. What’s to stop the EU from creating a law stating that social logins need to get explicit opt-in to do anything more than authenticating a user? 

The cat and mouse games that the industry has been playing for a while now will only serve to fragment the industry and make your job as a marketer more difficult. This serves neither you nor the customer’s privacy well.

What we need now are common standards that can be applied across platforms. This is the only way to ensure the longevity of the industry.

Keep positive!

Privacy First represents a tremendous opportunity for those willing to embrace it.

Many of your competitors will struggle to adapt and will waste time, money, and energy playing the short game. It’s up to you to look beyond today’s tactics and develop tomorrow’s Privacy First strategies.