WooCommerce A/B Testing and Heatmaps Without Breaking Cache
Plugins vs SaaS tools for WooCommerce A/B testing, why caching and script optimizers break tests, how to track orders correctly, and a step-by-step setup.
By Joost BakkerPublished 11 min read
On this page
Your options: plugins vs SaaS
WordPress plugins
They install like any plugin and store data in or near your WordPress site.
| Plugin | What it tests | Pricing (checked 5 Nov 2026) | Notes |
|---|---|---|---|
| Nelio A/B Testing | Pages, headlines, WooCommerce products, themes, templates, CSS, JavaScript; includes heatmaps and scroll maps | Basic: 5,000 tested page views/month, 1 site. Professional: 50,000, 5 sites. Enterprise: 200,000+. Yearly billing saves 20%. Amounts weren't shown in the page text we checked (Nelio pricing) | Session recordings are an add-on |
| FunnelKit | A/B tests of funnel steps (checkout, offers, thank-you pages) | Funnel A/B testing from the Plus plan, $149.50/year for 2 sites (FunnelKit pricing) | Only for pages built in FunnelKit |
| abtestkit | Full pages, posts and WooCommerce products | Free on WordPress.org (abtestkit) | Newer plugin; check its active installs and last update on WordPress.org. Results stored in your WordPress database |
Good for: small shops that want everything inside WordPress, and funnel builders already using FunnelKit. Watch out for: test traffic quotas (Nelio counts tested page views), extra load on your server, and compatibility with your cache and optimization plugins.
SaaS tools
They add a script to the <head> of every page and run the test in the visitor's browser.
- Testing tools: Convert (from $299/month billed annually), Kameleoon (from $495/month), and quote-only suites such as Wingify (VWO and AB Tasty) and Optimizely. Vendr's buyer data puts a typical VWO contract at about $1,400 a month (Vendr) and Optimizely from about $3,300 a month for one product (Vendr), as estimates. Prices and differences in VWO alternatives.
- Research tools: Microsoft Clarity (free), Contentsquare (formerly Hotjar), Mouseflow, Lucky Orange. Compared in Hotjar alternatives.
- All-in-one: Crazy Egg (heatmaps plus A/B tests from its Plus plan) and Varly (research and testing).
Good for: shops that want stronger stats, research tools and testing in one place, or that run more than WordPress. Watch out for: flicker (the original page flashing before the variant) and script optimizers. See the next section.
Caching: the main reason tests break
WooCommerce shops almost always run a page cache (WP Rocket, LiteSpeed Cache, W3 Total Cache, a host cache or a CDN). A page cache stores the finished HTML of a page and serves the same copy to every visitor. That has two effects on testing:
1. Tests that pick the variant on the server can break. If a plugin decides "this visitor gets version B" in PHP, the first visitor's version gets cached and everyone after them sees the same one. Plugins handle this by assigning variants in the browser or by excluding tested pages from the cache. Nelio's documentation advises testing your setup "especially if you cache full pages or combine and delay JavaScript," and purging the cache after you start or stop a test (see Nelio's plugin readme).
2. Script optimizers can delay the testing script. This is the more common problem, and it affects SaaS tools too. The testing script needs to run as early as possible so it can change the page before the visitor sees it. Optimizers do the opposite:
| Optimizer | What it does | How to exclude your testing or analytics script |
|---|---|---|
| WP Rocket: Delay JavaScript execution | Scripts only run after the visitor interacts (mouse move, scroll, tap, key press) (WP Rocket docs) | Add the script's URL or a keyword to "Excluded JavaScript Files," or add a nowprocket attribute to the script tag |
| LiteSpeed Cache: Load JS Deferred → Delayed | "Delayed runs the JS only after detecting user activity" (LiteSpeed docs) | Add the URL or a unique part of it to "JS Deferred/Delayed Excludes"; for combining, use "JS Excludes" |
| Cloudflare Rocket Loader | Changes how scripts load across the page | Add data-cfasync="false" to the script tag, before the src attribute (Cloudflare docs) |
What goes wrong if you don't exclude it: the visitor sees the original page, then the variant appears after their first scroll. Some visitors leave before interacting, so they're never counted in the test at all. That can cause a sample ratio mismatch: one version gets noticeably more visitors than planned, and the results can't be trusted. See sample ratio mismatch.
After every change: clear all caches (plugin, host and CDN), open the page in a private window, and check in the page source that the testing script is in the <head>, only once, and not rewritten by the optimizer. Then watch for flicker: our flicker article shows how to measure it.
Tracking orders and revenue correctly
An A/B test on a shop should usually decide on orders. That means each order has to be linked to the visitor and the version they saw. There are two ways:
Browser-side (thank-you page script). The tool counts an order when the visitor lands on the thank-you page (/checkout/order-received/ in WooCommerce). Simple, but it misses orders when the visitor closes the tab early, uses an ad blocker, or the payment provider doesn't return them to your site.
Server-side (from WooCommerce itself). A plugin sends each paid order from your server to the tool, together with an ID that links it to the visit. This is more reliable, and it can also handle refunds and cancellations.
Things to check whichever tool you use:
- Which order statuses count? Usually Processing and Completed. Pending payment orders shouldn't count.
- Are refunds and cancellations subtracted? Otherwise revenue in your testing tool will drift above WooCommerce's.
- Are test orders excluded?
- What happens without consent? In the EU, visitors who decline cookies usually can't be linked to their order. Their orders count in totals but not in tests.
Expect some difference between your tool's numbers and WooCommerce reports. Our help article why revenue numbers differ explains the usual causes.
Heatmaps and recordings on WooCommerce
Any heatmap tool works on WooCommerce through a script in the <head>. Tips specific to WordPress:
- Exclude the cart, checkout and account pages from recordings. They contain names, addresses and order details. Check your tool's defaults.
- Group product pages. All products share a template; analyze them together.
- Watch the mini cart and filter drawers. Many WooCommerce themes open them as overlays, which screenshot-based heatmaps don't capture well.
- Exclude yourself and your team so admin visits don't pollute the data.
Consent plugins and tracking
In the EU, heatmaps, recordings and most A/B testing tools should only run after the visitor agrees. Popular WordPress consent plugins include Complianz, CookieYes, Cookiebot and Usercentrics. Two things to check:
- Does your tool understand your banner's signal? Many tools read Google Consent Mode. If your consent plugin supports Consent Mode, switch it on in the plugin's settings; without it, tools that rely on it get no signal and measure nothing.
- Which category does the tool belong to? Usually "statistics" or "analytics." Put all CRO tools in the same category, so a visitor's one choice applies to all of them.
For the legal background, see our GDPR guide for heatmaps, recordings and A/B testing. This is not legal advice.
Setup step by step (with Varly as the example)
This is how the setup works in Varly, the tool we build, following its Install page. Varly has its own WooCommerce plugin that adds the snippet and sends orders (Varly help: Install on WooCommerce).
1. Pick your platform
In Varly, open your site, click Install and pick WooCommerce under 1.
2. Install the Varly plugin
- Click Download the Varly plugin.
- In WordPress, go to Plugins → Add New → Upload Plugin, upload the file, click Install Now and then Activate.
- Open Settings → Varly and paste your Site key (it starts with
pk_) from the Install page. - For orders: an Owner in Varly clicks Create API key under Orders and copies it. Paste it in WordPress as Order API key. Click Save.
The plugin adds the snippet to the top of every page. If you added the Varly snippet another way before (in your theme or Google Tag Manager), remove that copy so visits aren't counted twice.
Caching: if you use WP Rocket, LiteSpeed Cache or Rocket Loader, exclude the Varly script from delaying and combining (see the table above), and clear the cache.
3. Turn tracking on
Switch Tracking on on the Install page or under Settings → Tracking & privacy. Varly only measures after a visitor accepts cookies, and never in the checkout.
Then pick your banner under Settings → Tracking & privacy → Consent banner. Varly works with CookieScript, Cookiebot and OneTrust, and with banners that send Google Consent Mode signals (such as Complianz, CookieYes and Usercentrics) when Consent Mode is on in the banner (Varly help: consent).
4. Check that data comes in
Open your shop in a private window, accept cookies and click around. The Install page shows the page views and clicks of the last 24 hours. If nothing shows up, check Check that Varly is working, and look for a speed plugin delaying the script.
5. Connect your orders
With the order API key saved in the plugin, orders flow in automatically:
- Every paid order (status Processing or Completed) is sent in the background, so checkout doesn't get slower.
- After a refund or cancellation, the order is sent again with the new total (0 if canceled).
- Each order is linked to the visit through a cookie set after the visitor accepted cookies.
- Varly receives only the order number, total, currency, number of items, time and visit. No names, addresses or email addresses.
- If sending fails, the plugin adds a note to the order and retries at the next status change.
By default, Varly never tracks pages starting with /checkout or /cart (or the Dutch /winkelwagen and /afrekenen). If your cart is at /basket or /en/checkout, add it under Never track these pages. To ask customers a question right after ordering, add /checkout/order-received as your thank-you page under Settings → Tracking & privacy; Varly can show a survey there without recording the page (Varly help: site settings).
Add-to-cart as a goal: on WooCommerce, your developer sends an add_to_cart event with Varly.track (Varly help: custom events). That's useful for small shops, because add-to-carts happen more often than orders.
6. Choose where to read results
Pick Only in Varly, or In Varly and Google Analytics 4 so GA4 also records which version each visitor saw (Varly help: GA4).
Pricing: €99 per website per month, with heatmaps, recordings, surveys, funnels, click impact and A/B tests included.
Comparison table
| Nelio A/B Testing | FunnelKit | abtestkit | SaaS testing tool (e.g. Convert) | Varly | |
|---|---|---|---|---|---|
| Where it runs | WordPress plugin | WordPress plugin | WordPress plugin | Script on every page | Plugin adds script; orders sent from server |
| Tests | Pages, products, themes, CSS/JS | Funnel steps | Pages, posts, products | Any page | Any page (visual editor, split URL) |
| Heatmaps/recordings | Heatmaps included; recordings add-on | No | No | No | Yes |
| Surveys | No | No | No | No | Yes |
| Pricing basis | Tested page views | Per year, by sites | Free | Tested users | Per website: €99/month |
| Works outside WordPress | No | No | No | Yes | Yes |
When Varly is not the right choice
- You only test FunnelKit funnels. FunnelKit's built-in tests fit that workflow.
- You want everything inside WordPress, with no external service. Use a plugin.
- You want to test prices or shipping costs without a developer. Varly's visual tests change what's shown on the page, not what WooCommerce charges. Price tests need a server-side test in your own PHP code, built by a developer.
- You want revenue per visitor to decide the winner. Varly's verdict compares conversion rates.
FAQ
Is there an official WooCommerce A/B testing extension? There used to be. Its WooCommerce.com page now says "We retired this product."
Does A/B testing work with WP Rocket? Yes, if you exclude the testing script from "Delay JavaScript execution" and from combining, and clear the cache after starting or stopping tests.
Do I need a plugin or a SaaS tool? If you only run a few page tests and want everything inside WordPress, a plugin is fine. If you want research tools, stronger stats or the same tool across several shops, choose SaaS.
Can I A/B test the WooCommerce checkout? Yes, it's part of your own site, unlike Shopify's. Take care with payment fields and privacy, and measure completed orders. Many tools (Varly included) skip checkout pages by default.
Why doesn't my test get as many visitors as expected? Common causes: consent (only visitors who accept take part), a delayed script, or caching serving one version. Check the split first: sample ratio mismatch.
Varly's WooCommerce plugin adds heatmaps, recordings, surveys and A/B tests, and sends orders from your server. Request access.
