Get comfortable with being uncomfortable
Experimentation is big business. And one that pays to be different – especially, if you know that those with the deepest pockets are most likely to win.
Facebook Marketing Science Research explored the impact of digital experimentation on ad performance. This landmark report analysed 10,000 experiments over a 12-month period across all verticals.
The conclusion was clear. E-commerce companies saw a 2-3% uplift in performance per experiment while advertisers that ran 15 in the year reported a 30% total increase.
In our sample, an advertiser that ran 15 experiments (versus none) in a given year sees about a 30% higher ad performance that year; those that ran 15 experiments in the prior year see about a 45% increase in performance, highlighting the positive longer-term impact of this strategy.
So, why is it that only 12.8% of the companies studied were active in their experimentation?
In this article, we’re going to explore why experimentation is one of the most untapped tools for progressive marketers today. Read on for a six-step framework designed to incrementally drive your ad performance and revenue in the long-term.
Running order:
- Why marketers are slow to experiment
- The exploration-exploitation trade-off
- Six-step framework to performance marketing experimentation
- Three key takeaways
Why are so many marketers slow to experiment?
Across every industry, 70% of advertisers don’t explore by means of experimentation. Now, we know that experimentation yields better results. So why are so many marketers still slow to adopt?
In the Harvard Business Review, Julian Runge puts forward four common challenges:
- Organisational inertia: this may hold true if your company shows scepticism over the data or has a well-established marketing mix model already in place.
- Reduction of campaign reach: running an experiment means creating a ‘holdout’ group: a control group that receives no messaging. It’s essential for unbiased results, yet inevitably reduces your campaign reach.
- Lack of inter-company alignment: if you have multiple stakeholders and external partnerships, collaboration and alignment are no mean feats.
- Established decision-making: it’s common to overestimate the complexity of setting up an experiment, or focus on the short-term expense.
Introducing the exploration-exploitation trade-off
This brings us to ‘the exploration-exploitation trade-off’: the concept of choosing between something you know, and getting something close to what you expect. That’s your ‘exploitation’. If you choose something that you’re unfamiliar with and get something new, that’s exploration.
Successful experimentation relies on both.
If we keep this push and pull between the familiar and the unknown in mind, here’s what we know.
- On new channels, the old marketing strategies no longer cut it. If you’re used to non-digital channels where experimentation is expensive or practically impossible, digital experimentation could be a completely new approach. And that’s okay. Getting buy-in takes time.
- Creativity and intuition will always have a place at the table. In fact, they’re demanded. They work hand in hand with your data analysis and data-driven decision making.
Average marketers stick to the status quo because it makes their numbers look good on the surface. If you care about the truth and seek to prove it, this framework is for you.
The six-step framework to successful performance marketing experimentation
Step 1: Build a culture of experimentation.
This first step is arguably the most important. And also the most challenging.
- To foster a culture of creativity and experimentation, you need to tap into your senior stakeholders. Get executive sponsorship if you can.
- Depending on the size of your business, you or the CEO may be your ultimate advocate for breaking down broader business silos and creating that all-important urgency.
Here, persistence is key. Pitch your experimentation as a strategic pillar of decision making above a short-term tactic.
Step 2: Identify your KPI.
Bring stakeholders across the business together to identify a key performance indicator (KPI) for your advertising to drive. For example, let’s imagine you sell online meal kits. Your KPI could be the number of new subscriptions.
Step 3: Compare available advertising approaches.
Review the tactics at your disposal. Then, test your current digital approach against a control population (your ‘holdout group’) who receive no messaging. Measure, evaluate, then define a clear time period for testing your exploratory campaign.
Step 4: Earmark 10% of your budget against your exploratory approach.
Spending 10% on exploration is the sweet spot. In a survey among top managers, advertisers spent an average of 9.4% of their overall spend on exploration. Spending more doesn’t necessarily mean winning more. Indeed, Runge found that advertisers saw their performance % change decrease with increased spend.
Step 5: Identify your highest performing approach.
Then repeat, learn and adjust to influence your defined metric. Let’s imagine you run an exploratory digital campaign for meat-free meals kits, which results in a 10% drop in subscriptions. You can either drop or adjust your messaging and rerun the experiment. And if it increases subscriptions? Scale it up and add to your existing activity.
Step 6: Reinforce.
The more experiments you run, the more you’ll be able to pick out your strongest players and run ads with confidence. If you decide to transfer production to a creative management platform (CMP), you can build multiple ad-sets at scale.
Three key takeaways…
- The earlier you adopt, the higher your advertising performance. For brands that ran 15 experiments in the previous year, the increasing ad performance grew from 30% to 40%.
- It’s the insights that will drive your revenue, not the experiment itself. Performance increases are unlikely to materialise in the experiments but in your wider advertising.
- And while it pays to be uncertain, spending more on experimentation doesn’t mean winning more. If you’re an underdog, this is how you win against the bigger players.
At Programmai, we help you cut through the noise on the major ad platforms – Google, Facebook, Instagram, Snapchat and TikTok. By giving you a predictive, first-party data edge over your competitors.
Rather than your traditional grouping by demographic or segment, we supply the people that you really want to be talking to. Instead of measuring their buying potential on a website visit, we leverage your first-party data in its entirety to look to the future.
We accurately predict short and long-term customer outcomes, so that you can invest the right amount based on their predicted customer lifetime value and propensity to purchase.
Uncertainty = experimentation.
Certainty = results.
It certainly pays to be uncertain.
It's simple. If we don't experiment with lift tests — we lose money. Programmai helps us to improve the incrementality & lift of our acquisition campaigns on Facebook et al. as proven by control-holdout experiments. Their predictive lookalikes outperform anything we've tried ourselves using other means of building audiences.
