The big bad wolf won’t be blowing these bees’ house down any time soon.
Unlike other species of excavating bees who build their homes in wood or soil, Anthophora pueblo carves a home out rock in the Utah desert, creating nests that can withstand the tests of time and elements. The species was discovered almost 40 years ago by a USDA entomologist named Frank Parker who found dozens of new species of insects in the region.
He took samples of the unique hives, but never formally published
During the 2016 Rio Olympic Games, Mahe Drysdale rowed 2,000 meters (1.24 miles) in just 6 minutes and 41 seconds.
However, despite his impressive performance, the world record-holder nearly lost the race.
In one of the closest finishes in Olympic history, Drysdale won by mere millimeters.
Have you ever seen anything like this!?
— BBC Sport (@BBCSport) August 13, 2016
In contrast, Great Britain’s Men’s Eight team rowed the same distance in just 5 minutes and 29 seconds—over 70 seconds faster than Drysdale’s time!
What’s more, the Brits won by more than a half second.
That might not seem like a huge margin, but in the Olympics, a half second is a big deal.
So, why was Britain’s team so much faster than Drysdale?
The answer is simple: they had more oars in the water.
Now, at this point, you might be thinking, This is all well and good, Jake, but what does rowing have to do with online marketing?
Well, it turns out that conversion rate optimization (CRO) is a lot like rowing.
The more oars you have in the water, the faster you’ll make it to your goal and the more likely you are to beat out the competition.
The Secret is Testing Multiple Variants
Over the years, CRO seems to have become synonymous with A/B testing in the minds of many marketers.
Now, there’s nothing inherently wrong with this. A/B testing is a form of conversion rate optimization. You have a page and you want it to perform better, so you change something and see if it improves your results.
But here’s the thing, A/B testing isn’t the only way to do CRO.
It might not roll off the tongue as nicely as “A/B testing”, but if you’ve got enough traffic, A/B/C/D/etc testing can allow you to produce meaningful results much more quickly.
For example, Optimizely recently studied and reported on the factors that defined the world’s best testing companies.
Guess what the 4 biggest factors were?
- Testing the things that drive the most revenue
- Testing every change
- Testing to solve real problems
- Testing multiple variants simultaneously
Does #4 surprise you?
Apparently, the most effective CRO doesn’t come from A/B testing—it comes from testing multiple variants.
Essentially, A/B testing is like the Mahe Drysdale of CRO. It works and it can even deliver amazing results.
But, it’s only two oars in the water—there’s no way it can compete with an 8-man team.
To put this in more concrete terms, according to Optimizely, just 14% of A/B tests significantly improve conversion rates. On the other hand, tests with 4 variants improve conversion rates 27% of the time.
So, if you test 4 variants, you are 90% more likely to improve your conversion rate than if you just ran an A/B test. However, 65% of CRO tests are—you guessed it—A/B tests!
Why Testing Multiple Variants Works Better
Basically, there are two reasons why multiple variant testing outperforms A/B testing: 1) it’s faster and 2) it allows you to test more variants under the same testing conditions.
Multiple Variant Testing is Faster
Sure, you can test the same things with a series of A/B tests as you can with a multiple variant test—it just takes a lot longer.
When you run an A/B test, you can really only learn one thing from your test. Your variant will either perform better, the same or worse than your original.
And that’s it, that’s all you can learn.
Now, if you’re smart about your A/B testing strategy, your results can teach you a lot about your audience and make your future tests smarter, but you’re still only learning one thing from each test.
On the other hand, with multiple variant testing, you can try out several ideas at the same time. That means you can simultaneously test multiple hypotheses.
So, instead of just learning that a hero shot with a smiling woman outperforms a shot of a grumpy man, you can also see if a grumpy woman image drives more results than the grumpy man pic or if a happy man outshines them all.
Or, you can try multiple combinations, like a new headline or CTA in combination with either the smiling woman or the grumpy man.
Running all of these tests simultaneously will allow you to optimize your page or site much more quickly than you could with a long series of A/B tests.
Plus, running a test with multiple variants will greatly improve the odds that a single test will deliver at least one positive result, allowing you to start getting more from your website sooner.
Multiple Variant Testing is More Reliable
Another problem with successive A/B tests stems from the fact that the world changes over time.
For example, if you are in eCommerce and run your first A/B test during October and your second test during November, how do you know if your results aren’t being skewed by Black Friday?
Even if your business isn’t seasonal, things like differences in your competitors marketing strategies, political change or a variety of other variables can make it difficult to directly compare the results of A/B tests.
As a result, sometimes it can be hard to know if a particular A/B testing variant succeeded (or failed) because of factors outside of your control or even knowledge. The more tests you run, the murkier your results may become.
However, with a multiple variant test, you are testing all of your variants under the same conditions. That makes it easy to compare apples-to-apples and draw valid, reliable conclusions from your tests.
What Does Testing Multiple Variants Look Like in Real Life?
To show you just how testing multiple variants can improve your CRO results, let me share an experience we recently had with one of our clients.
The client wanted to get site traffic to their “Find Your Local Chapter” page, so we decided to add a “Find Your Local Chapter” link to the client’s footer. That way, the link would be seen by as many people as possible.
Makes sense, right?
So, we put together something that looked like this:
At first, we figured we would just put the link in the footer and run a test to see if the link made a difference.
But then, we started wondering if there was a way to make the link even more noticeable. After all, getting traffic to this page was a big deal to the client, so it made sense to emphasize the link.
With that in mind, we added color to the link:
Now, this idea seemed logical, but at Disruptive, we believe in testing, not gut instinct, so we figured, “Hey, we’ve got enough traffic to test 3 variants, let’s take this even further!”
The problem was, the client’s site was a designer’s dream—modern and seamlessly designed. To be honest, we had a bit of trouble selling them on the idea that creating a page element that interrupted their seamless flow was worth testing.
But, eventually, we convinced them to try the following:
It was very different from anything the client had tried on the page before, but we decided to run with the idea and include it in our test.
Not surprisingly, adding the “Find Your Local Chapter” link increased page visits by over 60% for every variant—that’s an awesome win, right?
But here’s the thing. With our original, strict A/B test, we would only have discovered that adding the link increased traffic by 63%.
On the other hand, by including a couple of extra variants, we were able with the same test to discover that—contrary to the client’s belief—the more our link “interrupted” the site experience, the more traffic it drove to the chapter page.
Sure, we might have reached the same conclusion with several more tests, but we achieved these results much more quickly and reliably than we would have with an A/B testing series.
Should You Test Multiple Variants?
When it comes to testing multiple variants, there’s only one real reason not to use it: your boat is too small.
Think about it: if the entire British Eight Man team had tried to cram onto Mahe Drysdale’s boat, they never would have made any forward progress.
The same idea applies to CRO.
As great as multiple variant testing is, if you don’t have enough traffic, a test could take months or years to complete.
In fact, in true multivariate testing—where you test to see how a large number of subtle changes interact to generate your conversion rate—you want at least 100,000 unique visitors per month (for more information on multivariate testing, check out this great article).
On the other hand, you need far less traffic to simultaneously test multiple page variants.
To see how long a multiple variant test will take on your site, try out this VWO has a free sample size and test duration calculator from VWO. If the time frame makes sense for your business, go for it!
Whether it’s Olympic rowing or CRO, the more oars you have in the water, the better your results will be.
Although it may be tempting to limit CRO to A/B testing, testing multiple variants will allow you to improve your conversion rates more quickly and reliably than you could with a series of A/B tests.
You’ve heard my two cents, now it’s your turn.
Have you tried multiple variant testing? What was your experience like? Did any of the data in this article surprise you?
About the Author: Jacob Baadsgaard is the CEO and fearless leader of Disruptive Advertising, an online marketing agency dedicated to using PPC advertising and website optimization to drive sales. His face is as big as his heart and he loves to help businesses achieve their online potential. Connect with him on LinkedIn or Twitter.
Hubble hasn’t found aliens on Europa, but it may have found new evidence that plumes of salt water from the moon’s globe-spanning salty ocean can escape through cracks in its icy shell.
Using its Space Telescope Imaging Spectrograph (STIS) instrument, Hubble captured far-ultraviolet images of what could be geysers of water from beneath the surface, erupting in Europa’s southern hemisphere. If the features in those images are really geysers, that could be very good news for future missions
We’ve talked a lot about data quality in the past – including the cost of bad data. But despite a basic understanding of data quality, many people still don’t quite grasp what exactly is meant by “quality”.
For example, is there a way to measure that quality, and if so, how do you do it? In this article, we’ll be looking to answer those questions and much more. But first…
Dispelling Data Quality Myths
The foundation for ensuring data quality starts when basic requirements are created
One of the biggest myths about data quality is that it has to be completely error-free. With websites and other campaigns collecting so much data, getting zero errors is next to impossible. Instead, the data only needs to conform to the standards that have been set for it. In order to determine what “quality” is, we first need to know three things:
- Who creates the requirements
- How are the requirements created, and
- What degree of latitude do we have in terms of meeting those requirements
Many businesses have a singular “data steward” who understands and sets these requirements, as well as being the person who determines the tolerance levels for errors. If there is no data steward, IT often plays the role in making sure those in charge of the data understand any shortcomings that may affect it.
You Can Have It Good, Fast or Cheap – Pick Two
Everything from collecting the data to making it fit the company’s needs open it up to potential errors. Having data that’s 100% complete and 100% accurate is not only prohibitively expensive, but time consuming and barely nudging the ROI needle.
With so much data coming in, decisions have to be made and quickly. That’s why data quality is very much a delicate balancing act – juggling and judging accuracy and completeness. If it sounds like a tall order to fill, you’ll be glad to know that there is a method to the madness, and the first step is data profiling.
What is Data Profiling?
Data profiling involves looking at all the information in your database to determine if it is accurate and/or complete, and what to do with entries that are not. It’s fairly straightforward to, for instance, import a database of products that your company manufactures and make sure all the information is exact, but it’s a different story when you’re importing details about competitor’s products or other related details.
With data profiling, you’re also looking at how accurate the data is. If you’ve launched on 7/1/16, does the system record that as 1916 or 2016? It’s possible that you may even uncover duplicates and other issues in combing through the information you’ve obtained. Profiling the data in this way gives us a starting point – a springboard to jump from in making sure the information we’re using is of the best possible quality.
Determining Data Quality
So now that we have a starting point from which to determine if our information is complete and accurate, the next question becomes – what do we do when we find errors or issues? Typically, you can do one of four things:
- Accept the Error – If it falls within an acceptable standard (i.e. Main Street instead of Main St) you can decide to accept it and move on to the next entry.
- Reject the Error – Sometimes, particularly with data imports, the information is so severely damaged or incorrect that it would be better to simply delete the entry altogether than try to correct it.
- Correct the Error – Misspellings of customer names are a common error that can easily be corrected. If there are variations on a name, you can set one as the “Master” and keep the data consolidated and correct across all the databases.
- Create a Default Value – If you don’t know the value, it can be better to have something there (unknown or n/a) than nothing at all.
Integrating the Data
When you have the same data across different databases, the opportunity is ripe for errors and duplicates. The first step toward successful integration is seeing where the data is and then combining that data in a way that’s consistent. Here it can be extremely worthwhile to invest in proven data quality and accuracy tools to help coordinate and sync information across databases.
Your Data Quality Checklist
Finally, because you’re dealing with so much data across so many different areas, it’s helpful to have a checklist to determine that you’re working with the highest quality of data possible. DAMA UK has created an excellent guide on “data dimensions” that can be used to better get the full picture on how data quality is decided.
Their data quality dimensions include:
Completeness – a percentage of data that includes one or more values. It’s important that critical data (such as customer names, phone numbers, email addresses, etc.) be completed first since completeness doesn’t impact non-critical data that much.
Uniqueness – When measured against other data sets, there is only one entry of its kind.
Timeliness – How much of an impact does date and time have on the data? This could be previous sales, product launches or any information that is relied on over a period of time to be accurate.
Validity – Does the data conform to the respective standards set for it?
Accuracy – How well does the data reflect the real-world person or thing that is identified by it?
Consistency – How well does the data align with a preconceived pattern? Birth dates share a common consistency issue, since in the U.S., the standard is MM/DD/YYYY, whereas in Europe and other areas, the usage of DD/MM/YYYY is standard.
The Big Picture on Data Quality
As you can see, there’s no “one size fits all” approach to maintaining accuracy and completeness on every type of data for every business. And with big data’s appetite for information growing more and more every day, it is becoming more important than ever to tackle data quality issues head-on. Although it can seem overwhelming, it’s worth enlisting data hygiene tools to let computers do what they do best – crunch numbers.
The most important step you can take is simply getting started. The data is always going to grow as more prospects come on board and new markets are discovered, so there’s never going to be a “best time” to tackle data quality issues. Taking the time now to map out what data quality means to your company or organization can create a ripple-effect of improved customer service, a better customer experience, a higher conversion rate and longer customer retention – and those are the kinds of returns on investment that any business will wholeheartedly embrace!
About the Author: Sherice Jacob helps business owners improve website design and increase conversion rates through compelling copywriting, user-friendly design and smart analytics analysis. Learn more at iElectrify.com and download your free web copy tune-up and conversion checklist today!
A draft article due to appear in APS Observer caused widespread outrage this week. Susan Fiske, the former president of the Association for Psychological Science (APS), writes that bloggers and other online critics of psychology papers are running wild:
New media (e.g., blogs, twitter, Facebook posts) are encouraging uncurated, unfiltered trash-talk. In the most extreme examples, online vigilantes are attacking individuals, their research programs, and their careers. Self-appointed data pol
(This post originally appeared in the online anthropology magazine SAPIENS. Follow @SAPIENS_org on Twitter to discover more of their work.)
Joshua Hinson’s first biological son was born in 2000. His son’s birth marked the start of the sixth generation that would grow up speaking English instead of Chickasaw, which was the primary language his ancestors had spoken for hundreds of years. Hinson was born in Memphis, Tennessee, and grew up in Texas. Other than a small handful of words, he kn