For every marketing team, experimentation is essential to understanding the efficacy of our marketing. The concept of ‘test and learn’—the process of conducting various experiments you can monitor, measure, and evaluate—helps create the data insights necessary to make decisions about where and how we spend our marketing dollars.
Sometimes, there’s a misconception that test and learn is something you should only prioritize at the very beginning of building your marketing tactics. The logic is that once you find your winning channels that deliver the desired outcome, you can very much go into auto-pilot mode and turn your attention to other focus areas. While there’s a degree of rationale to that, it doesn’t factor a host of market conditions, or where you might be in the evolution of your marketing.
Why do we need experimentation?
It’s not uncommon for companies in scaling mode to have changing priorities. If you’re a software-as-a-service (SaaS) company, marketing may need to pivot from brand awareness, to audience acquisition strategies that drive SaaS growth. As a retailer, you may find yourself needing to build a contacts database through broad marketing tactics first, before focussing on subscribers with strong buyer intent. The same applies to how you optimize your owned assets like your website, constantly iterating the user experience with new wireframes, button placement, or calls to action to see what creates engagement.
I would advocate for experimentation to become a staple of your marketing strategy. It’s how you find new channels, engage net new audiences, minimize spend risk, and maximize your return on investment (ROI). As part of a common framework, it shouldn’t be confused with optimization. You need ongoing experimentation to run in parallel with any tweaks you make to improve results. Analyzing how these changes impact the performance of your efforts are the insights you’ll rely on to make ongoing, data-driven optimizations.
A framework for Test and Learn Marketing
Testing different marketing tactics and measuring their effectiveness can be as simple as brainstorming a few ideas and putting these into practice to see what sticks. I prefer a more systematic approach that involves structuring actions around key motions.
You can follow this framework to create a test and learn pathway:
1. Identify what you’re trying to achieve:
Setting clearly defined goals and objectives are important for how you intend to evaluate the success of your marketing experiment. These could include increasing subscribers to your newsletter, improving email engagement rates, increasing active time on your website, or any other specific outcome you want to achieve.
Example: Let’s assume you’re an online retailer who sells vintage fashion on your Shopify store. You’ve recently launched a loyalty program and the key objective of your experiment is to drive traffic to your loyalty program sign-up page from social media.
2. Create a theory around your intent:
Formulating a hypotheses about the marketing tactics or strategies you want to experiment with, is important for creating a baseline of what you think you can achieve. These hypotheses should be specific and measurable.
You’ve researched that Instagram is a good social media platform to find users interested in vintage fashion, and want to start advertising on this platform in addition to the advertising you already have running on Facebook. You’re making the assumption that if you increase your advertising channels from one to two social platforms, you will double inbound traffic to your sign-up page.
3. Design your experiment:
Develop a structured approach to testing your hypotheses. It could be something as simple as designing an A/B test, where you compare two versions of a marketing component (e.g. ad copy headline, email subject line, landing page layout) and measure performance against your objective.
To keep your experiment simple, you run exactly the same graphics, copy, and headlines across Facebook and Instagram, and track your traffic hits via the in-platform reporting tools in Shopify, or use a third-party application like Google Analytics.
4. Implement further experiments:
Once you have a proof of concept, you can enhance your experiments further by making room for variations of the initial variant implemented. For instance, if you’re testing different channels with the same assets, start creating multiple versions of those assets (e.g. different copy, graphical elements, colour choices, etc.) and run them simultaneously to gather further data on ad performance.
For your Facebook ads, you decide to run ad imagery that only includes a close up of a pair of cowboy boots. On Instagram, your visual is a model actually wearing the boots instead. You also decide to version your headline, calls-to-action, and the product type featured in the ad to better align to your target demographic.
5. Measure everything important
Collecting data through-out the experiments you conduct are imperative to your analysis of the results. Monitoring key performance indicators (KPIs) such as engagement click-through rates, sign-up rates, ad performance metrics, and/or revenue generated are just a few of the KPIs worth considering. Use statistical analysis to look for repeat patterns and bechmark the significance of the results.
The Instagram ads outperform Facebook ads. The inbound traffic from Instagram is 30% higher than the traffic from Facebook. Furthermore, users who arrived at your sign-up page from Instagram outperformed Facebook traffic when it came to actual sign-ups by a ratio of 2:1 respectively. The best performing creative were the ads where the model was wearing products from the store.
6. Draw meaningful conclusions:
Based on the data you’ve analyzed, you can now start to evaluate the performance of the tested elements, drawing conclusions about their impact on your desired objectives. Determine which versions/variants performed better and whether the results support or refute your initial theories.
With the results you were able to achieve on Instagram, you decide to continue experimenting with advertising on other social platforms, taking the learnings from the Facebook vs. Instagram trial.
7. Iterate your learnings, launch, and re-optimize:
Apply the insights gained from the experiments to start refining your marketing strategies. Take a holistic view of what you’ve learnt, implement the tactics that were proved to be successful, and discard or modify those that did not perform well against your objectives. Repeat the test and learn process continuously to refine your marketing efforts further.
You drop Facebook and decide to test Pinterest advertising by re-channeling your advertising dollars to this new social platform. You keep running your Instagram ads, using the best performing ad format and benchmark against how Pinterest performs.
Building a culture of innovation
For test and learn marketing to succeed, you need to have learnings to guide you into what should come next. This is only made possible when you have outcomes-based data to draw upon, which means you must monitor and measure everything you’re doing.
Applying an iterative outlook to testing and learning, enables you to make data-driven decisions and identify the most effective marketing strategies. Don’t assume you need to achieve benchmark-beating results every time you experiment. Sometimes, small yet incremental improvements across multiple areas can be equally as important when you put all the outcomes together. Keep it simple, and keep it focussed.
Perhaps most importantly, ‘Test and Learn’ as a methodology is a great starting point for embedding agile thinking into your Marketing mindset. It helps build a culture of curiosity and innovation, helping teams reduce guesswork and achieve better ROI outcomes using empirical evidence.
Need some help setting-up some Marketing experiments?
Book a no-obligations consultation, and we can chat.