An A/B test compares two versions of a page through random assignment of visitors, to evaluate whether a change helps achieve an objective. Google Analytics 4 can support measurement, but creating and delivering variations requires an experimentation tool or a custom implementation.
Note about this guide: the original article, published in 2019, showed the old Experiments section of Google Analytics. That workflow does not apply to GA4. In addition, Google Optimize and Optimize 360 became unavailable on September 30, 2023. Below you will find criteria for designing a test today and, at the end, the original screenshots identified as a historical archive.
What is an A/B test and what is it used for?
In an A/B test, some participants see version A and others see version B. You can compare a headline, a form, or even two complete designs. The important difference from comparing one month with another is that experimental assignment helps reduce the influence of external factors, provided the design and measurement are appropriate.
The objective can be to complete a request, subscribe to a newsletter, purchase, or perform another relevant action. Watching a video or pressing a button can also be measured, but those interactions should be distinguished from the final outcome the business needs.
Multivariate tests compare combinations of several elements to study their effects and relationships. They require a design and observation volume compatible with the number of combinations. They are not an automatic solution for every large redesign: an A/B test of two complete designs can evaluate which works better, although it does not identify which element explains the difference.
Does an A/B test guarantee more conversions?
No. It may find an improvement, show that a change harms the journey, or fail to provide sufficient evidence to decide. Its value lies in testing a hypothesis before extending it. No percentage increase can be promised simply by using this technique.
In paid campaigns, improving the proportion of visits that end in a valuable action can affect cost per acquisition. To interpret it, also review traffic costs, request quality, and subsequent sales. More forms do not always mean more customers.
How to prepare a useful A/B test
1. Identify a specific difficulty
Start with observable questions: why does a page receive visits but few completed forms? At which step do people leave? Is the button understandable? First check for broken links, form errors, or measurement problems. A known fault should be corrected, not maintained solely for experimentation.
2. Research before proposing changes
Use analytics to explore navigation, interaction, and relevant actions. In GA4, interpret metrics according to their current definitions; do not directly transfer conclusions from bounce rate or time on page in the old Analytics. Supplement data with usability tests, interviews, heatmaps, and recordings when you meet applicable privacy and consent conditions.
Heatmaps and recordings show behavioral signals; they do not reveal on their own what a person thinks. Gaze measurement, or eye tracking , can study where someone looks, but does not automatically demonstrate understanding or purchase intent either.
The original Agencia Cero article mentioned a collaboration with Agencia Mars for eye-tracking studies. We retain the historical reference; it does not imply that the collaboration or its services remain current.
If a page with a form has visits but few requests, investigate possible explanations: unnecessary fields, an unclear offer, doubts about data use, or people who are not yet ready to make contact. Observation guides questions; it is not enough to establish the cause.
3. Formulate a testable hypothesis
For example: “If we remove the fields we do not need to answer an inquiry, more people will complete the form without reducing request quality.” Define the main action and a control measure, such as the proportion of contacts the team can serve. This avoids celebrating an increase in submissions that only creates unproductive work.
4. Decide which difference you will evaluate
To isolate an element’s effect, change one main variable and keep other conditions comparable. If you need to compare two complete proposals, document that you will evaluate the whole. Avoid changing the campaign, offer, and form halfway through the test without a record: mixing changes complicates interpretation.
5. Define the sample, duration, and decision rule
There is no universal minimum number of days that guarantees certainty. Duration depends on eligible traffic, action frequency, the difference worth detecting, the statistical method, and the business cycle. Define these criteria before starting and use the sample calculation or decision method of the chosen tool.
More visits do not correct biased assignment or duplicate events. Include comparable activity periods and avoid ending the test at the first favorable result. A statistical confidence level does not mean absolute certainty about the outcome, and interpretation depends on the method used.
6. Choose elements with a reason to test them
- Headlines and subheadings that explain the offer.
- Paragraphs that answer purchasing questions.
- Authentic, authorized testimonials.
- Calls to action and links.
- Images relevant to the decision.
- Form fields, instructions, and error messages.
Prioritize changes that could resolve an identified difficulty. Testing colors without a clear question can consume time and traffic without providing a useful decision.
7. Choose tools and check measurement
For a website test, you need to create variations, assign participants consistently, and record which version they saw and what they did afterward. Google mentions integrations with AB Tasty, Optimizely, and VWO in its documentation about the closure of Optimize. Evaluate features, compatibility, privacy, and costs before choosing; it is not an automatic free replacement.
Unbounce, mentioned in the original article, also documents A/B testing for its landing pages. Review the product scope and the plan available for your case. Having Analytics installed is not equivalent to having a complete experimentation system.
How to measure a website test with GA4 today
The Google documentation for integrating experiments distinguishes between external tools and custom developments. The tool runs the variations; Analytics receives information allowing their results to be analyzed.
- Define the outcome. Decide which relevant action will be recorded and check that the event occurs correctly. A confirmed submission is more useful than counting every attempt to press the button.
- Prepare variations and assignment. Document what changes and how each participant’s experience is maintained within the experiment.
- Connect exposure and outcome. In a compatible integration, the exposure event and its variation identifier distinguish groups. In a custom implementation, define experiment and variation parameters and the necessary dimensions, following the documentation.
- Run technical tests. Verify variations, forms, and events on mobile and desktop. Review consent, duplicates, internal traffic, and differences between tools.
- Analyze according to the plan. Compare experimental groups and periods with the same criteria. GA4 helps observe results; statistical validity requires reviewing the method and data quality.
Do not send names, email addresses, or personal form content in analytics parameters. Use experiment identifiers that describe the test, not the person.
Historical archive: Experiments in the old Google Analytics
The following screenshots belong to the original 2019 tutorial. They are retained to explain how the exercise was structured then. They are not operating instructions for GA4, and you should not install the old code they show today.
Preparing versions and accessing Experiments
The exercise started with two ready pages and a previously configured goal. In that interface, the workflow began in the “Behavior” menu and continued to “Experiments.”
That submenu opened a configuration screen. Its conceptual usefulness was organizing the test before starting it; the location of those controls does not correspond to the current version of Analytics.
Historical step 1: experiment configuration
Experiment name
This identified the test. Today, assigning an understandable name that identifies the hypothesis and version evaluated remains useful.
Experiment objective
In the example, the objective was for the person to reach a thank-you page after requesting an Analytics review. For a current test, verify that the measured action represents an actual request rather than a reload or accidental visit.
Traffic percentage and distribution
The historical tool allowed choosing the proportion of traffic included and offered an equal-distribution option. Initial exposure can be limited for technical risk, but the experiment’s assignment and distribution must be defined and verified with the current tool.
Duration and confidence threshold
The interface included controls for minimum duration and confidence. They are shown as part of the archive, not as recommended values. No chosen duration or threshold replaces experimental design or sample review.
Historical step 2: variation addresses
The URLs of the original page and the second version were entered. That approach compared separate pages; current tools may offer other ways to present variations.
Historical step 3: code and validation
After the URLs were configured, the tool provided code that the tutorial instructed users to install only on the original page. The configuration was then validated and the experiment started. This procedure belongs to the old product; use only the current instructions for the solution you choose.
Historical interpretation of results
The example sought to distribute visitors between versions and compare goal completion. The following screenshot illustrates that tool’s results presentation, not a current Agencia Cero result.
Data appearing after a few hours or days does not mean they are sufficient for a conclusion. First check that the test works; make the decision when the defined criteria are met. The old recommendation to wait three days or use twenty-one days as a statistical guarantee is not a valid rule for every site.
What to do after an A/B test
If the evidence supports the variation and the control measures hold, you can assess implementing it and observe subsequent behavior. If the result is inconclusive, retain the learning without inventing a winner. If the hypothesis is not supported, review what you observed and what the next useful question is.
Save the hypothesis, versions, dates, eligible population, measurement, and decision. That record helps avoid repeated tests and conclusions taken out of context. If you need to organize your indicators and understand what to change, explore the approach of Agencia Cero’s digital analytics; tools and scope are defined according to the project.
