Run a server-side test

Some changes can't be made in the browser: a new checkout flow, other prices or shipping rules, a different search, or a page your server builds. In a server-side test your developer builds every version in your own code. Your server asks Varly which version each visitor gets and shows it. Varly keeps the split fair and counts goals, orders and revenue the same way as in tests you make with the visual editor.

Create the test

1. Open Optimize → Tests and click New test.
2. Under What do you want to test?, choose Change it in your own code (server-side).
3. Fill in Name of the test.
4. Optionally, fill in Page it changes, for example https://yourshop.com/checkout. Varly uses it to warn about other tests on the same page in the calendar. Leave it empty when the change is on many pages, such as prices.
5. Under Versions, give each version a name your team understands, for example Current checkout and One-page checkout. Each version has a fixed Key (control, b, c, d) that your developer uses in the code. Click + Add variant for up to three variants.
6. Under Traffic split, keep Even (recommended), or choose Custom to give each version its own share, for example 80/20 to show a risky change to fewer visitors. A custom split needs more time.
7. Click Create server-side test.

The test opens on the Set up in your code tab. It is ready to launch right away: there is nothing for Varly to build.

Set up in your code

Send this tab to your developer. It shows:

• the keys and names of the versions, and the traffic share of each;
• code to copy for Node.js / TypeScript, WordPress (with or without WooCommerce), PHP, Python and Any language (HTTP), already filled in with your site and this test;
• what works on Shopify (see below);
• the Server key card. An owner clicks Create server key, names it (for example "Production server") and copies the key. Varly shows it only once. Your developer stores it on the server as VARLY_SERVER_KEY. One key works for every server-side test on the site.

Check the setup with a dry run

Until you launch, your server gives everyone the original and sends Varly nothing. To check the setup before anyone sees a change:

1. Launch the test and choose Dry run first (recommended) in step 4. See Launch, pause, resume and end a test.
2. Visit the page yourself with cookies accepted, or wait for some visitors.
3. On Set up in your code, click Check my setup. Varly shows when your server last reached Varly, and per version how many visitors your server reported and when it last did.
4. When every version has visitors and Split check on the Results tab says OK, click Go live: let your code show the new version.

Which visitors take part

• Varly recognizes a visitor by the _varly_id cookie the snippet sets after consent. That is why the same visitor always gets the same version, on your server and in the snippet, and why goals, orders and revenue are linked to the right version.
• A visitor without that cookie gets the original and is not counted: a first visit before the snippet ran, or someone who didn't accept cookies. Your developer can include first visits after consent; see the developer reference.
• With privacy mode on, visitors who haven't accepted cookies can take part anonymously too, but only when your page sends their anonymous ID along with the request, and never with Do Not Track or Global Privacy Control. See the developer reference.
• Who takes part works with devices, new or returning visitors, traffic sources and a share of visitors. For devices, your server uses the browser's user agent. New or returning visitors and traffic sources only work when your developer passes them; otherwise those visitors are left out.
• Your server decides who takes part, so the rules (page URL, UTM tag, country, cookie, …) and Keep tests apart work only in tests that Varly shows on the page.

Pause or end the test

Pause and End test work as in any test. Your server checks Varly every 30 seconds, so within a minute every visitor gets the original again, without a deploy. When you also turn Tracking on off for the site, your server gets no tests at all.

When a variant wins, have your developer make it the normal version in your code, then remove the test code.

Shopify

• Theme (Liquid): Shopify runs your theme on its own servers, so there is no place for your own server code. For changes in your theme, choose Change a page and use the visual editor.
• Hydrogen or another headless storefront: that is your own server code. Use the Node.js code.
• An app with an app proxy: Shopify doesn't forward cookies to an app proxy. Your developer sends the visitor ID from the page with Varly.ids() and decides in the app.
• Checkout: Shopify's own checkout can't run your code, so it can't be tested this way.

Common problems

Check my setup shows no visitors. The test must be running (a dry run is fine). Check that the code runs on the page, that the server key is set, and that you visit with cookies accepted.

Visitors in one version only. The code handles only one key. Check that every key (control, b, …) shows its version.

Everyone sees the same version. A page cache (WP Rocket, LiteSpeed Cache, Cloudflare, Varnish) serves the same page to everyone. Exclude the test page from the cache, or let the cache vary on the _varly_id cookie.

The key doesn't work. It was revoked, belongs to another site, or is the public site key (starts with pk_). Server keys start with sk_srv_.

For developers

The libraries download the site's tests once and decide on your server, without a network call per page view. They send which visitor saw which version in small batches. The full reference with every endpoint is in the developer reference.

Node.js / TypeScript (varly-server.mjs):

import { createVarly } from './varly-server.mjs'

const varly = createVarly({ siteKey: 'pk_…', serverKey: process.env.VARLY_SERVER_KEY })

const { variant } = varly.decide('<test ID>', {
  cookies: req.headers.cookie,
  userAgent: req.headers['user-agent'],
})
if (variant === 'b') {
  // the new version
}

PHP, for WordPress (with or without WooCommerce), Magento or plain PHP (Varly.php):

require_once __DIR__ . '/Varly.php';
$varly = new \Varly\Client(['siteKey' => 'pk_…', 'serverKey' => getenv('VARLY_SERVER_KEY')]);
$decision = $varly->decide('<test ID>'); // reads $_COOKIE['_varly_id'] and the user agent
if ($decision['variant'] === 'b') {
    // the new version
}

Python (varly_server.py):

from varly_server import Varly
varly = Varly(site_key="pk_…", server_key=os.environ["VARLY_SERVER_KEY"])
d = varly.decide("<test ID>", cookies=request.cookies, user_agent=request.headers.get("User-Agent"))

Any language, one request per page view:

curl -G https://cdn.varly.ai/v1/assign \
  -H "Authorization: Bearer $VARLY_SERVER_KEY" \
  --data-urlencode "site=pk_…" --data-urlencode "experiment=<test ID>" \
  --data-urlencode "visitor=<value of the _varly_id cookie>"
# {"experiment":"<test ID>","variant":"b","inTest":true}