Personalization: show a change to one audience

A personalization shows one change to one group of visitors, for good. For example:

• returning visitors see "Welcome back" above the products;
• visitors from one Google Ads campaign see a headline that matches the ad;
• mobile visitors from Belgium see "Free shipping to Belgium".

Everyone else sees your site as it is.

Create a personalization

1. Go to Optimize → Personalization and click New personalization.
2. Give it a name, and paste the page you want to change. Choose On this page or On every page of the site (for a bar in the header, for example).
3. Under Who sees it, pick a saved audience or choose the visitors there: device, new or returning, traffic source, or rules such as country, a UTM campaign or a page. A personalization is always for one audience: "everyone" is not an option.
4. Choose the Holdout (see below). 10% keep the original is selected.
5. Click Create personalization.

Varly takes you to Build & launch. Make the change with the visual editor, blocks or custom code, exactly as in a test. Check it with the preview link or the QR code on your phone; see Preview and QA.

Launch it

The launch checklist is the one from a test, with four steps: what counts as a conversion, Who sees it and the holdout, How long Varly measures, and when it starts. You can start with a dry run, launch now or schedule a start. Click Launch personalization.

Varly checks how many visitors your audience gets and plans how long it needs to measure, the same way as for a test. With a 10% holdout that takes longer than a 50/50 test, because the holdout is small.

The holdout: how Varly knows whether it helps

Part of the audience keeps seeing the original, at the same time. Who is in the holdout is random and stays the same on every visit. Varly compares the two groups with the same statistics as an A/B test, including guardrails such as revenue and JavaScript errors.

• 10% keep the original (recommended): enough to measure, while 90% of the audience sees the change.
• 5%: more visitors see the change, but measuring takes about twice as long.
• 20%: measures faster, but a fifth of the audience keeps the original until the result is in.
• No holdout: everyone in the audience sees the change. Varly then can't measure whether it helps or hurts. It only shows how many visitors saw it.

Varly never compares "before" with "after". A week with the change against a week without it mixes in everything else that changed, such as a sale, the season or a campaign. Only two groups at the same time give a fair answer.

The result

On the Results tab, Varly shows the personalization against the holdout. As with a test, it gives the result once, at the planned end, and not before:

• It helps: visitors who see it convert better than the holdout.
• It hurts: they convert worse. Stop it, or change it and measure again.
• No clear difference: the personalization may not be worth keeping, or the audience is too small to tell.

If you set up notifications in Slack or by webhook, Varly tells you when the result is in.

After the result, the personalization keeps running with its holdout. When you're happy with it, click Show it to everyone in this audience: the holdout sees it too from then on. Varly keeps the result from the planned end, frozen, and stops measuring. You need the right to launch tests for this.

Two personalizations on the same thing

You can run several personalizations side by side. When they change different parts of a page, a visitor who fits both sees both.

When two of them change the same part of a page, the order on the Personalization page decides: the one higher in the list wins. A visitor who fits both sees only the first one. The other one skips that visitor and doesn't count them, so each result is only about its own visitors. Varly shows this under each personalization: "Changes the same part of a page as …, which comes first". For a personalization on several pages, Varly compares page by page: a headline it changes on the product page doesn’t clash with a headline another personalization changes on a category page. Use Move up and Move down to change the order; it applies within a minute.

• Custom code counts as changing the whole page, because Varly can't tell what it touches.
• A visitor in the holdout of the first personalization doesn't get the second one either. That keeps the first result fair: its holdout sees the original.
• A personalization in a dry run doesn't change the page, so it never pushes another one aside.

This is different from Keep tests apart, which splits the visitors between tests. For personalizations that would show each one to only part of its audience, so Varly uses the order instead.

A/B tests on the same page run on top of personalizations: a test compares its versions against what the visitor sees now, including any personalization.

Pause, resume or stop

On the Personalization page or the Results tab:

• Pause: everyone sees the original until you click Resume. Paused days don't count toward the measurement.
• Stop: everyone in the audience sees the original again, within a minute, and Varly freezes the numbers. A stopped personalization can't start again; make a new one instead.

Share and download

• Download CSV on the Personalization page lists every personalization with its audience, page, holdout, status and result.
• On the Results tab you can download the numbers or share a read-only link, as for a test.
• AI assistants connected to Varly can read them with list_personalizations and get_personalization. They can't create or launch one.
• The monthly client report has a short Personalization part when one ran that month: who sees it, the holdout, the result against the holdout and the status. It never counts as a test or a winner.

Common problems

I don't see the personalization on my site. Check that you fit the audience (device, new or returning, rules) and open the page in the preview first. Maybe you're in the holdout: open the preview link to see both versions. See My test doesn't show.

It says "Changes the same part of a page as …". Another personalization higher in the list changes the same element. Visitors who fit both see only that one. Move this one up if it should win.

The result is taking long. A small audience or a small holdout needs more weeks. A bigger holdout measures faster.