Nabler.com

A/B Testing Tool: Comparing Google Optimize vs Optimizely

Understanding the difference between Google Optimize and Optimizely

6 min readJul 23, 2020

How can you get the conversation rolling through tough times like changing consumer behavior and increasing market competition? How to hold the viewer’s attention for the fleeting seconds they are on your website? Challenging right? AB testing is by far the best recommended option for being able to determine what the user likes and can lead to a conversion.

When comparing A/B testing tools, the focus is on the features that are most important for your experimentation. Google Optimize and Optimizely are both solid choices for experimentation and personalization tools, though each has unique capabilities and use cases.

They are tools for developing experiments, running them, and collecting data about user activity in both controlled and variable situations. You can run the experiments and use results for CRO improvement and learn about your user behavior. It can also help you identify problems and generate new hypotheses and come up with new experiments.

If you are a marketing manager or a CRO expert, conversion optimization should be on top of your to-do lists! We’ll try to cover the pros and cons of Google Optimize and Optimizely. In a way, this comparison might not seem fair since we’re comparing a free tool such as Google Optimize and the premium Optimizely, but the article might help you get an idea of what both tools offer and which one is more suitable for your business and experiments. In this article, we’ll try to compare these two optimization tools as we as will try to cover the pros and cons of both Google Optimize and Optimizely.

We’ll make a comparison in terms of the following steps in the process of running an experiment:

Setting up the tool
Development of variation
Tracing conversion goals
Defining the audience
Test and run the experiment
Reporting

Setting up the tool

To implement Optimizely you need to insert an Optimizely code snippet in the head of your HTML file and you are ready to go.

Whereas for installing Google Optimize, you first should have to have a Google Analytics account and then add Optimize snippet code inside Google Analytics tag. You will have to link your Google Optimize project to the one in Google Analytics and install the Chrome extension to start developing variations. Google also recommends implementing an optional anti-flicker snippet to avoid A/B testing flicker that sometimes occurs in many tools. This flickering occurs when the original content is visible while the variant loads. To prevent this from happening simply add the additional script to temporarily hide the page while the Optimize container fully loads.

Development of variation

The Optimizely visual editor allows you to make changes to the layout and appearance of your website or pages. This includes editing visual elements, its position, modification of text, images, background styles, and borders. In addition to the WYSIWYG editor (Google Optimize), you can insert HTML, CSS, JavaScript, and use jQuery selectors.

Whereas creating variants in Google Optimize is simple with its built-in WYSIWYG visual editor. It allows you to simply click the elements you would like to add or make any changes you want. In the event where you need to make a more advanced change, you also have options like HTML, CSS, and custom Javascript to fine-tune your variant.

Tracking conversion goals

To track visitor’s interface with both control and your variation, you need to set up tracking of conversion goals. Goals can be clicks, landing on a specific page, or some custom goal like the completion of your form, etc.

Optimizely offers an easy interface for tracking of conversion goals like:

The custom goal is goal triggered by the code, but page view and click is something that a user can set up within a minute without writing any code.

Whereas, conversion goals in Google Optimize are called objectives. You can set up:

objectives based on your goal that is defined in Google Analytics
bounces
page-views
session analysis
custom objective (defining an event like in Google Analytics)

Optimizely allows you to have as many goals as you want, Google Optimize has the limitation of 3 goals per experiment.

Defining your audience

After you are done with the segmentation of your results, often you will need to run experiments targeting only a specific set of audience (such as desktop visitors, users coming from UTM campaign, based on user’s location, etc.), so for your A/B testing experiment, you will need to define your target audience. (List all audiences from Optimize and Optimizely).

Optimize 360 (premium Google Optimize version) has a feature to set audiences based on GA data. That enables you to target a more specific group of people. For example, you can target only visitors that in the past have purchased some item which can be convenient. Whereas in Optimizely a solution won’t be so straightforward.

Test and running an experiment

Before setting your experiment live, it is always important to test it out to be sure that everything is working as expected. If you are setting up a web page for different devices, make sure it looks and works well on all screen sizes. Also, it is important to test if your goals and objectives are firing correctly.

Test using the Preview mode

Both the tools come with Preview mode and give you an option to share a preview with other people involved in the experiment. Preview mode helps you test out your variations across browsers and devices.

Optimizely’s preview mode also logs goals fired during testing. Preview mode won’t be effective if the experiment is shared with users who don’t have access to Optimizely or Google Optimize projects.

Test using test cookie

You can also test out your experiment by setting up a test cookie and targeting only an audience that has a cookie that you defined as the test cookies. In that case, you start your experiment, you can check if goals are fired if results are logged the way you expected.

In Optimizely, after you have tested out, you can pause the test, reset the results, remove the test cookie, and set the experiment live — show it to “real” users.

After you pause the experiment in Google Optimize there is no option to reset results and start it again. Once you are unable to edit the experiment, suppose if you spot some bugs during testing you can’t pause the test and fix, you will need to copy the experiment, fix the issues, and start it again. If you are testing the firing of goals (objectives) the whole process will last more than it should because the results are generated with a few hours delay. So, you won’t be able to tell immediately if your experiment works well.

Reporting

Optimizely and Google Optimize both have a different approach when it comes to counting conversions. Google’s reports are session-based, meaning it might count the same visitor twice if they started two different sessions.

Optimizely, on the other hand, is a unique visitor based, so it won’t count the same visitor and conversion twice.

They also differ in the statistical method they use when determining the engaging variation. Google Optimize uses Bayesian methods and Optimizely uses the Frequentist approach.

Reporting in Optimizely is almost live, meaning you can see visitor’s conversions immediately or with a few minutes delay. Google Optimize on the other hand has some limitations in this regard. It has up to 12 hours of delay and this can be very frustrating if you are testing out experiments before setting them live. You are not able to see results on the same workday if something needs to be fixed.

Conclusion

This article is not about highlighting on which tool is best for you. The decision about best depends on the nature of the project. Google Optimize being linked to Google Analytics can be both an advantage and a disadvantage. If your project type is not complex then Google Optimize should be a very good solution considering it’s a free tool and has a fair amount of features and options. You can also check out our in detailed comparison sheet below:

Originally published at https://www.nabler.com.