All tests and the test overview

Many teams track their tests in a spreadsheet next to their testing tool. In Varly the ideas, tests, screenshots and results already live together, so All tests is that spreadsheet, kept up to date for you.

What you see

Each row is one test, or one idea that has no test yet ("Idea, no test yet"). An idea with a test shows as that test. The Status column shows where it stands:

• Idea and Shortlist: ideas you haven't started.
• In preparation: the test is being made, for example Marking changes or Building.
• Ready to launch: built and waiting for your launch.
• Running: live, or Paused.
• Done: the test has ended.
• Rejected ideas only show when you pick that status in the filter.

Outcome shows Won, Lost, Inconclusive or Stopped early, with the decision under it. "Suggested: won" means Varly suggests this from the results and nobody has confirmed it yet. Change is the expected change with its range, for example "+5.4%" and "+1.8% to +9.1%". Archived tests and ideas are not listed.

Table, Board and Gallery

• Table: all rows with status, outcome, change, owner, labels and date.
• Board: the same rows in columns per status, from Idea to Done. Click a card to open it.
• Gallery: finished tests only, with the original and the variant side by side, the outcome, the change and what you learned. A good place to look before you start a new test: was this tried before?

Filter, search and export

1. Type in Search to look in titles, hypotheses and learnings.
2. Pick a Status, Outcome, Label or Owner, and how to Sort.
3. Click Page and dates for more: a page, and From and To. Dates use when a test ended, or else started, or else was created.
4. Click Apply. Clear removes all filters.

The filters stay in the address, so you can bookmark or share a view with your team. Export CSV downloads the rows you see, with the same filters, to open in Excel or Google Sheets.

Owner and labels

Open a test (or an idea) to set:

• Owner: who works on it. You can pick anyone on your team who can see this client.
• Labels: click + Add label. Type to find a label, or type a new name, pick a color and click Create. Under Suggested you find ready-made labels for the page type (Home, Category, Product page, Cart, Checkout, Landing page, Search, Sitewide) and the lever (Trust, Usability, Comprehension, Motivation, Cost). Click the cross on a label to remove it.

Labels belong to your whole organization, so every client uses the same set. Team members who can only see some clients see the labels used on those clients and labels that aren't used anywhere yet, so a label made for one client doesn't show up for people who can't see that client. A test made from an idea gets the idea's owner and labels. Viewers can see owners and labels but can't change them.

The test overview

Click a test in All tests to open its overview. It tells the story in five steps:

1. Idea: the hypothesis and the research behind it.
2. Plan: primary goal, who takes part, duration and the smallest effect to detect, plus who approved the test plan.
3. Original and variant: screenshots on desktop or mobile.
4. Result: the change with its range and chance, the visitors, the split check and, when your orders are connected, the revenue impact. While a test runs, the live numbers are on the experiment page.
5. Decision and learning: the outcome, the decision and what you learned. Click Edit result to change them.

At the top you find Test plan, Results report (once the test has ended) and a link to the experiment itself.

Frozen results

When you click End test, Varly freezes the numbers of the test. Varly keeps visit data for 13 months and orders for 25 months; frozen results stay readable after that, and they don't change when you change your settings. An older test that ended without frozen numbers gets them the first time you open it, or open All tests, as long as the data is still there. If it isn't, the overview says the numbers are no longer available.

All tests across clients (agencies)

At the top of your organization, click All tests. You see the tests of every client you have access to, with the client and website in each row, and a Website filter under Website and dates. Each client's data stays separate: this view only lists them side by side.

Common problems

• A finished test shows no outcome. It was ended without recording a result (for example from an AI chat). Open it and fill in Decision and learning.
• No screenshots in the gallery. Screenshots are made when Varly builds a test. Tests without a built version have none.
• An idea is missing. Archived ideas are not listed, and rejected ideas only show with the status filter Rejected.