Prioritize ideas with PXL, PIE, ICE or your own framework

Choose a framework

A framework is the list of questions every idea is scored on. Open Optimize → Prioritization and click the Scoring framework: … button at the top right. It shows the name of the framework in use, for example Scoring framework: PXL.

Framework	How it scores	Best for
PXL (recommended)	Mostly yes/no questions about the change and the evidence, such as "Noticeable within 5 seconds?" and "Supported by heatmaps or recordings?". Some questions count double.	Teams that want scores that are objective and hard to game.
PIE	Potential, Importance and Ease, each from 1 to 10.	A quick first ranking.
ICE	Impact, Confidence and Ease, each from 1 to 10.	Fast-moving teams that score many ideas.
Custom	Your own questions, weights and points.	Agencies with their own method.

The score is the share of the maximum points an idea gets, so 100 means every question got the best answer. That keeps ideas comparable when you change the framework.

Make your own framework

Pick a framework as a starting point in the Framework list, then change anything. Each question is one compact row showing its type, weight (for example ×2), whether Varly fills it in ("by Varly: traffic") and its maximum points. Click a row to open it, or use Expand all:

• Question and Explanation (optional): the explanation shows next to the question when you score an idea.
• Type: Yes / no, Scale (1 to 3, 5 or 10) or Choice (your own answers, each with its own points, for example "Under 4 hours" = 3 points).
• Weight sets how heavily a question counts, from ×0 to ×5. With ×2, a "yes" is worth 2 points. With ×0 (off) the question doesn't count toward the score.
• Fill in automatically lets Varly answer the question from your data (see below).
• Add a question offers ready-made questions, such as Revenue impact, Affects the primary KPI? and Validated by earlier tests?, or + Your own question.
• Use the arrows to change the order, and × to remove a question.

The bar at the bottom stays in view while you scroll. It shows the maximum points and Unsaved changes when you changed something. Click Save framework to keep your changes, or Discard to undo them. If you try to leave the page with unsaved changes, Varly asks first.

Picking PXL, PIE or ICE replaces your questions, but nothing is lost until you save: a message offers Undo, and Custom in the list brings your own questions back. Answers are stored per question, so switching frameworks after saving loses no answers either: when you switch back, the old answers are still there.

One framework for all clients, or per client

• For every client at once: an owner sets the default under Organization settings → Prioritization framework → Set the default framework. Every client without its own framework uses it right away.
• For one client: on the client's framework page, choose Own framework for [client]. That client keeps its own framework, even when the default changes. To go back, choose the option that starts with Use the default (in an agency: Use the agency default) and click Switch to the default.

Without any setting, Varly uses PXL.

What Varly fills in for you

Questions with Fill in automatically get an answer from your data. In the table you see it as a lighter answer with a dashed border; on the idea page it is marked by Varly, and a list shows what Varly picks, for example Let Varly decide (now: Under 2 days). In the menu Fill in automatically you pick where the answer comes from:

• Research behind the idea: heatmaps, rage or dead clicks and scroll depth; survey answers; click impact, page speed and JavaScript errors linked to the idea. In the menu these are Heatmap, rage, dead or scroll evidence, Survey evidence, Click impact evidence, Numbers: click impact, speed or errors, Any evidence in Varly and Strength of the evidence.
• Traffic of the page: "yes" for the busiest fifth of your pages, or a higher number on a scale for busier pages.
• Expected extra revenue per month: the page's sessions × its conversion rate × your average order value × the expected effect. This needs connected orders.
• Impact estimate of the idea and Ease estimate of the idea.

Click an answer to set it yourself. Click the same answer again, or pick Let Varly decide, to hand it back to Varly. Amounts in Revenue impact are in your shop's own currency.

Score ideas quickly

The table on Optimize → Prioritization has a column for every question. It shows Not tested yet by default: ideas that are New or on the Shortlist. Click All ideas to see ideas in every status (archived ideas are left out). Click Yes or No, or pick a value, right in the table. The score updates as you click. The Idea and Score columns stay in view when you scroll sideways, so you always see the effect of your answers. The order stays the same until you reload the page, so rows don't jump while you work.

On a phone, the page shows a card per idea with its score and test duration. Tap Score this idea → to answer the questions on the idea's page.

On an idea's own page, the Priority card shows the same questions with their explanation and how many points each answer adds.

How the test duration is calculated

Varly uses the standard calculation for comparing two conversion rates:

1. Conversion rate. The share of the page's sessions that placed an order in the last four weeks. If Varly has no orders for your site yet, it assumes 3%.
2. Smallest effect the test can detect (also called the minimum detectable effect). This is the smallest relative improvement the test must be able to prove. Varly takes it from the idea's impact: 1 = +2%, 2 = +4%, 3 = +7%, 4 = +12%, 5 = +20%. A +10% effect on a 3% conversion rate means going to 3.3%.
3. Sample size. This is how many visitors each version needs to detect that effect with 95% confidence and an 80% chance to spot it. In plain words: if the change does nothing, there is at most a 5% chance of a false winner. If the change really works this well, the test has an 8 in 10 chance of showing it.
4. Duration. The visitors needed for both versions, divided by the sessions per week, rounded up to whole weeks. A test always runs at least 2 full weeks, so every weekday counts twice. If a test would need more than 8 weeks, Varly shows Not enough traffic, because results that take that long get muddled by seasons, cleared cookies and other changes to your site. You can still launch such a test; the launch checklist then shows ways to get a clearer answer. See Launch a test.

Under Can we test this? on the idea page you see this in full, for example "12,400 visitors per version, about 3 weeks". Over 4 weeks it adds "(long, but possible)". Below that, Varly shows the smallest effect the page could detect in 4 weeks.

How to read the result

• A few weeks: a good candidate. Start with the highest score among these.
• Not enough traffic: the effect is too small for the traffic. Test a bolder change (a bigger effect needs fewer visitors), choose a page with more traffic (similar pages, such as all product pages, count as one), or set the page to Whole site. When you launch, a goal that happens more often than an order, such as Added to cart, also needs fewer visitors.
• Estimated conversion: without order data, every duration is based on the 3% assumption. Connect your orders for real numbers.

Common problems

• No traffic yet, and Sessions / week shows "–". Varly saw no visits to this page in the last four weeks. Open the idea and pick its Page from the list again, for example Product pages (/products/…). The list shows pages that had visits; for a new page choose Another page (type a path)…, but Varly can only plan a test once that page gets visits.
• A good idea has a low score. Open questions count as 0 points. Answer the open questions in the table, and link research to the idea so Varly can fill in the answers about research.
• "by Varly" gives the wrong answer. Varly only knows what is in your data. Click the right answer to overrule it.
• Revenue impact stays empty. It needs connected orders and visits to the idea's page. See Orders and revenue.
• The duration here differs from the launch checklist. The checklist lets you choose the Smallest effect the test can detect yourself and accounts for extra variants, so its number can be different.