A/B Testing vs Multivariate Testing: When and how are you choosing?

Have you ever? Change the website ... but don't know if it really improved or just "think by yourself"

Believe that every market, marketer, or business owner, must have encountered this uncomfortable situation: You are dedicated to both physical, mental and budget. Whether changing pictures, promotions, writing new Headline, attractive, or even new layout design ... But then the result is "Employment"

Sales do not increase, click less, or the worst is "Worse than before" but you don't know what it is. Feeling like walking in the dark Cannot find a way to continue I don't know if what I have done is "yes" or "no". This is a classic problem that kills conversion and causing us to lose business opportunities unfortunately.

Prompt for illustrations: The picture of marketers or business owners are sitting in the temples in front of the computer that shows the agreed sales graph. With a question mark (?) Floating around to convey confusion and uncertainty.

Why is "guessing" as an ingenious enemy of sales?

The source of most problems comes from the decision that is based on "feelings" or "instinct" alone. We think "Headline like this should be more liked customers" or "This color button should be more outstanding", but we have nothing to confirm that what "we think" will be what "customers think".

In the world of digital marketing that is driven by data (Data-DriVen), guessing is like having to dissolve the money. Because every change that does not pass the test is "risk" that is unnecessary. This is the point where powerful tools such as A/B Testing and Multivariate Testing play an important role. It is a scientific process that turns "guessing" into a "proof truth", helping us to know exactly what changes that really have our business. Not just thinking

Prompt for illustrations: Comparison images between the left brain with symbols "Instinct/feeling" with the right brain with symbols "Graph/Data" with an arrow pointing from the feeling to the data side Media that we have to change from guessing to data

Let the website "Dae" because of the next guess? The results are more terrifying than you think.

Ignoring the test and continues to change the website according to the feelings. May lead to negative effects that are more intense than just "Sales do not add". Try to imagine:

• A wasteful marketing budget: You may throw an enormous shooting money to pull people into the Landing Page page, but if that face is not effective enough People came in and closed. The money you paid is equal to zero.
-

While you are still the right way by guessing Your competitors may be using Data from A/B Testing to develop their websites to leave you every day.
• Conversion rate falls down: some changes that you think "a little" may cause confusion or "Friction" for users without knowing and gradually eating your conversion rate by little.
-

The website that changed has changed direction. Makes users feel that the brand is not professional and lack of attention to their experience

Letting the web continue like this It is no different from letting the boat leak without thinking of that leakage. Understanding the basic Cro (Conversion Rate Optimization) on Shopify or other platforms Therefore is the armor to prevent these problems the best

Prompt for illustrations: The paper boat with a name tag "My Website" is having a small leak. Many points called "Wasterated Budget", "Lost Customers" and slowly sink into the sea.

Stop guessing! Let's get to know A/B Testing and Multivariat Testing.

It's time to change uncertainty into a tangible information. The solution to this problem has 2 main types that are highly popular and effective, that is A/B Testing and Multivariat Testing (MVT), which are clearly different and the right situations in use.

A/B Testing (Winner Test)

Imagine that A/B Testing is a fight on the boxing stage. We have "old champions" (current version or Control) to fight with the "challenger" (new version or Variation).

• What is it: is comparing 2 versions of the web page (or more in A/B/N Test) that is clearly different. To see which version is better (such as Conversion Rate)
• When used: suitable for testing "big" and "high impact" such as changing all Headline, new layout design, Hero Image change, or adjusting the payment process
• Example: Test the version A page ("Green" button) with the version B ("Orange" button) to see which color makes more people click.

Multivariate Testing (MVT - The best mixing set test)

If A/B Testing is the MVT boxing, the laboratory of scientists. We are not just looking for a winner. But we want to know what the best "ingredients" consist of By testing many small changes on the same page at the same time

• What is it: Multiple Elements and "multiple variations" of those components at the same time to find out which "combination" (Combination) that works together, providing the best results.
• When used: suitable for "tuning" or "Optimize", a webpage that has a high and effective admission To be even better Used to understand that small elements How does each part affect each other?
• Example: on the same page We want to test Headline (2 types) x images (2 types) x text on the button (2 types) at the same time, which will create a set of 2x2x2 = 8 versions to find which combination sets are the highest conversion.

What kind of selection Start by asking yourself "We want to change the change (Revolution) or gradually develop (Evolution)?" If it is the first thing, start by doing A/B Testing on the webflow or your form. But if it is like MVT, it is the answer.

Prompt for illustrations: 2 -sided images, 2 pairs of boxing gloves (red vs blue) with "A/B Testing" and the right side is a scientific trial bean. The leaves with different liquids are being mixed together with the "Multivariate Testing" label.

An example case: from ordinary coffee shops to conversion champions with tests

In order to be clearer Let's take a look at the story of the shop "The Grind House", which is a shop selling online coffee beans. They have a beautiful landing page, but Conversion Rate does not move anywhere. So they decided to use Data to help decide.

Step 1: Big structure adjustment with A/B Testing

• Problem: The original page (version A) has a long information, but the button "Watch all coffee beans" is almost the bottom of the page.
-

If we move the CTA button to the top (Above the Fold) and change the Headline to be more interesting. Should increase clicks -

They created a new version of the new Headline, "Experience the unique coffee flavor. Delivered directly to your home "and put the button of" Choose your coffee beans ", a large orange color underneath -

After running A/B Test for 2 weeks

Step 2: Adjust the sub -element with multivariate testing

• Next problem: Now they have a winning layout (version B), but want to Optimize better.
  • Headline: (Probably the same)
  • HERO IMAGE: A (Coffee Farm) VS (Barista is drip coffee)
  -
  Button Text:
  Message A ("Select your coffee beans") vs text B ("Shop now")
-

Changing the Hero Image and the text on the CTA button may help to add more conversion -

On the Layout of version B, they test the MVT as follows:-

MVT found that the winning combination set is

Which helps to increase the conversion rate. Overall, another 18% from the original!

It can be seen that they use A/B Testing to find a "great change" first and then use the MVT to "adjust the details", which is a very powerful strategy. And is a guideline that Cro experts are always recommended.

Prompt for illustrations: Infographic images tell the story of The Grind House, with Step 1 (A/B Test) and Step 2 (Multivariate Test) and clearly display the results.

Want to start testing? Checklist 6 simple steps that can follow immediately.

The beginning may look scary, but actually there are clear processes. Whether you choose to do A/B Testing or MVT, you can follow this checklist.

1. Set goals that can be measured (Define Your Goal): What do you want to improve? Do not say widely, "want to increase sales" but to be specific, such as "wanting to increase the number of people, press the add to car on the product page" or "want to reduce the Bounce Rate on the Landing Page page by 10%"
2. Form a hypothsis: Create clear sentences in this structure: "I believe that changes (x) will result in results (Y)" such as "I believe that the color change button 'is orange (x) will increase the click (Y) because it is more outstanding than the original and creates a rushing feeling (Z)".
3. Choose the tool and method of testing (Choose your Tool & Method): From your goal and assumption. Do you want a big change (A/B) or MVT? Then choose tools to use such as VWO or Optimizly.
4. Create a Create Variation: Constructing the new version (Variation) according to your assumptions in your tools. Make sure everything works correctly.
5. Start testing and waiting (Run The Test & Be Patient): Let the system start to divide the traffic to different versions. The important thing is "patience" should be so long enough to get a statistical Significance, which usually takes at least 1-2 weeks to reduce discrepancy.
6. Analyze the results and do (Analyze & Implement): When the test ends See the results of which version is the winner. If there is a clear winner Then use that version to actually use with 100% traffic if the result is no different It means that what you change doesn't have any effect. And it's time to make a new assumption to continue testing

Starting from the A/B Testing process for new E-Commerce is the best point to create a test culture in your organization.

Prompt for illustrations: 6 checklist images with icons in each item (Article 1: Archery, Article 2: Labox, Article 3: Tool Box, Article 4: Sutan Set/Design Tools, Article 5: Watch/Calendar, Article 6: Graph

Questions that people tend to wonder (FAQ) about A/B and Multivarlate Testing.

I have compiled a popular question that is often encountered. With a clear answer and easy to understand

Q1: How much does it have to have a traffic to get to the website?
A: No fixed numbers. But it depends on your current conversion rate and the size of the results you expect. But the general principle is that multivariate testing requires a lot more traffic than A/B Testing because it has to divide the traffic into small parts. A lot of versions for each version to receive enough information. If the website has a very high traffic, starting with A/B Testing will be easier to see.

Q2: How long should I let the test run?
A: The important thing is not "the number of days", but "Statistical Significance", which will tell you that the results are reliable at a level of 95% or?

Q3: Which between A/B Testing and MVT? "Better"?
A: There is no better one. It's like asking "Which screwdriver and screwdriver is better?" It depends on the "work" that you want to do. If you want to nail (Looking for a clear winner from a big change), it must use a hammer (A/B Testing), but if you want to screw (Adjust a small element To find the best combination set), you must use screwdriver (MVT)

Q4: If the results come out that "No winner" (Inconclusive), what should I do?
A: This is a valuable lesson! This kind of result tells us that "What we change Does not affect the behavior of users at all. "Which is better than guessing and switching to a new version, which is not really better. What should be done is to return to the new assumption that "brave" and "different" than before and then start testing again. Or conclude that the original element is already good and go to another point to test instead

Prompt for illustrations: People are asking questions. With a text box with the word Q1, Q2, Q3, Q4 and short answers Easy to understand

Summary: Stop guessing and start "test" to create real growth.

At this point, I hope everyone will see the differences and benefits of A/B Testing and Multivariet Testing.

Simple summary to remember by heart:

• A/B Testing: Used for "winners" that are clear between very different ideas. Suitable for "Revolutionary"
-

Used for finding the "best mixed set" of small elements Suitable for changes in the "Evolutionary" for optimize.

The key to conversion rate optimization is not the most expensive tool. But in the creation "Testing culture" in the organization, changing methods from "I think ..." is a "information saying ..." is the first step that is the most important to unlock the growth of your business online.

Do not let your success depend on the random. Starting from a small thing today, try to assume your most important page. And start testing it Traveling to the real growth ... Starting with your first test!

Ready to change the guess into a tangible sales yet? Let the Vision X Brain experts help you set the strategy and perform the most effective test. Consult us for free today!

Prompt for illustrations: The graph of the line that rises from the beginning with the word "Guesswork" to the highest point with the word "Data-Driven Success" with A/B logo on the graph.

‍