Glossary

A

A/B test: a test where visitors are split between the original page and a changed version, the variant. Varly then compares how often each group orders. Each visitor keeps seeing the same version. See Create an A/B test.

A/B/n test: an A/B test with more than one variant. Visitors are split evenly over the original and all variants. A winner needs stronger proof, so more variants don't mean more false winners.

Attention map: the Attention view of a heatmap. It shows how long each part of the page was on screen. Warmer means more time spent there. See Heatmaps.

B

Bayesian statistics: the default way Varly judges tests. It shows the Chance to beat original for each variant. You can switch to frequentist under Settings → Experiments. See Read your test results.

C

Chance it's real: in click impact, how likely it is that the difference between clickers and non-clickers is real and not chance. It uses the same Bayesian method as A/B tests. See Click impact.

Chance to beat original: in a Bayesian test, how likely it is that a variant converts better than the original. With one variant, a winner needs at least 95%. This is often called "chance to win".

Click impact: a report that compares visitors who clicked an element with visitors on the same page who didn't, by orders and revenue. It shows a connection, not the cause. See Click impact.

Confidence: in Prioritization, how strong the research behind an idea is. It grows with more and harder research findings. See Prioritize ideas.

Consent: the visitor's choice in your cookie banner. Varly only measures, tests and surveys visitors who accept. See Cookie consent.

Control: the unchanged page in a test. Varly calls it the Original.

Conversion: in Varly, an order. In a test, a visitor converts when they place an order after first seeing the test during its live period, also if they ordered before. The Conversion rate is the share of visitors who did.

D

Dead click: a click after which nothing on the page changes within a second. Clicks on form fields and real links don't count. Dead clicks often mean visitors expected something to happen. See Heatmaps.

Dry run: a way to launch a test where visitors are split but the page doesn't change. You can check the split before anyone sees the variant. Each test has its own dry run, even when another test is live. Go live: show the new version to visitors ends the dry run and starts the count again. See Launch a test.

E

Ease: in an idea, how easy it is to build and test, from 1 to 5.

Error click: a click followed by a JavaScript error within 3 seconds. Something may be broken there. Shown under Before an error in heatmaps.

Evidence: see Research findings.

Expected change: Varly's estimate of how much better or worse the best variant converts than the original, in percent, with a 95% interval. Also called uplift.

Exposure: the moment a visitor on the test page is assigned a version. A visitor counts once, in the version they first saw.

G

GA4 (Google Analytics 4): Google's analytics tool. Varly can send it which test version each visitor saw, with the event experience_impression. See See your A/B tests in Google Analytics 4.

F

Favorite: a recording your team added with Add to favorites. Favorites are kept for 13 months instead of 30 days. See Watch session recordings.

First clicks and Last clicks: heatmap views of what visitors clicked first on a page, and the last click before they left or moved on.

Frequentist statistics: the classic method, which shows a p-value. Choose Frequentist (p-value) under Settings → Experiments. Varly uses a two-sided test that compares the two conversion rates.

H

Heatmap: a picture of a page that shows where visitors click, how far they scroll or where they look. Red means a lot of activity, blue means little. See Heatmaps.

I

Idea: a test idea built on research, written as "If we see… then we change… because we expect…". Ideas move from New to Shortlist, In test and Done. See Ideas.

Impact: in an idea, how much it could move conversion, from 1 to 5.

M

Masking: hiding text in recordings, set under Hide text in recordings. It happens in the visitor's browser, before anything is sent, and also applies to the texts of clicked elements in heatmaps. Choose Strict, Balanced (recommended) or Relaxed. See Recording privacy.

MDE (minimum detectable effect): in Varly called Smallest effect the test can detect. The smallest relative change a test must be able to prove, for example +10%. A smaller effect needs more visitors. Varly plans tests with 95% confidence and an 80% chance to spot an effect that is really there. See Launch a test.

Multivariate test: a test where you pick two or three parts of a page and give each one or two alternatives. Varly tests every combination and shows the Effect per part. See Multivariate tests.

N

New and returning visitors: a visitor is returning when Varly has seen their browser before. You can filter heatmaps, recordings and surveys on this, and choose it under Who takes part when you launch a test.

O

Organization settings: the settings page for your whole organization, with its name, the prioritization framework, the AI key and the option to delete the organization. Open it with Settings at the top of your list of clients or sites. Not the same as Site settings. See Add your own AI key.

P

Page group: similar pages that Varly counts as one, so all product pages share one heatmap. Shown with a plain name, such as Product pages (/products/…), where … is the part that changes. Set it up under Settings → Group similar pages. See Group similar pages together.

Placed an order / No order: visitors who did or did not place an order. You can filter heatmaps, recordings and survey answers on this once your orders are connected. With and without order shows both. See Orders and revenue.

Planned duration: how many weeks a test runs before Varly gives a verdict. Always at least two full weeks.

Preview: a link that shows a test on your real site, only to you, without measuring anything. See Preview and check a test.

Priority score: a score from 0 to 100 that Varly calculates with your scoring framework, from an idea's impact, ease and research findings. See Prioritize ideas.

p-value: in frequentist statistics, how surprising the difference would be if the versions were really the same. Below 0.05 counts as significant with one variant. A smaller p-value is stronger proof.

R

Rage click: three or more clicks on the same spot within a second. A sign of frustration. See Heatmaps.

Recording: a replay of a real visit, with every click, scroll and page. Also called a session recording. See Watch recordings.

Research findings: the research linked to an idea, such as click impact, rage clicks or survey answers. Shown on the idea page under Research behind this idea. See Ideas.

Research round: when Varly reads the latest research and turns clear findings into ideas. Start one with Find new ideas, or let the Daily research round run every day.

S

Sample ratio mismatch: when visitors aren't split over the versions as planned. Varly checks this under Split check. Uneven split means the results can't be trusted. See Read your test results.

Scroll depth: how far down a page visitors scroll, as a share of the page. The Scroll heatmap and the Scroll depth card show it.

Session: one visit to your site. A new session starts after 30 minutes without a new pageview.

Site settings: the settings page of one site, opened with Settings under Site in the site menu: tracking, consent, grouping similar pages, recordings, experiments and data storage. See Site settings and the Install page.

Smallest effect the test can detect: see MDE (minimum detectable effect).

Snippet: the one line of code you add to your site's <head>. It measures visits, shows surveys and runs tests. See Install the snippet.

Survey sample: the share of visitors who can see a survey, set with Show to … of visitors. A visitor stays in or out of the sample on every visit.

U

Uplift: how much better a variant converts than the original, in percent. Varly shows it as Expected change.

V

Variant: a changed version of a page in a test, built by the AI from your idea. The unchanged page is the Original.

Visitor: one browser, recognized by a random ID that Varly stores after consent. Varly never uses names or IP addresses for this. The same person on a phone and a laptop counts as two visitors.

W

Who takes part: the visitors who are in a test, chosen when you launch it: devices, new or returning visitors, traffic sources and a share of visitors. Everyone else sees the original and is not counted. A smaller group makes a test take longer. See Launch a test.