CRO Script Weight Study: Do Tools Slow Your Shop? (2026)

Why script weight matters for shops

Every tool you add with a <script> tag downloads JavaScript, which the browser then has to unpack, parse and run. On a fast laptop that's barely noticeable. On a mid-range phone on a busy mobile connection, it adds up.

Two of Google's Core Web Vitals are affected:

• LCP (Largest Contentful Paint): how fast the main content appears. Scripts that block rendering, or tags that hide the page until they're ready (anti-flicker), delay it directly.
• INP (Interaction to Next Paint): how fast the page responds to taps and clicks. Long JavaScript tasks on the main thread make taps feel slow.

The main thread is the part of the browser that runs JavaScript and responds to the visitor. While a script runs, a tap waits.

Three things make CRO tools special:

1. They run on every page, including your fastest landing pages.
2. Testing tools often load synchronously (before the page renders) to avoid flicker, the flash of the original page before the test version appears. See the A/B test flicker effect.
3. Shops stack them: one for heatmaps, one for surveys, one for tests, plus analytics and marketing tags.

None of this means you shouldn't use CRO tools. It means you should know what each one costs, remove the ones you don't use, and pick lighter options when they do the same job. As one CRO agency put it: "A testing win that introduces a 400ms LCP regression will cost more in conversion rate than it gains from the variant" (Linear Design).

Method

Date: November 5, 2026. Browser: Chromium 156 (headless) driven by Playwright 1.64, desktop window 1366×768.

Test page. One blank product-style page (a heading, text, a button, a long scroll area), with nothing on it except one tool's install snippet, copied from the vendor's documentation or from a public site using it. One tool per run.

Which account. Most tags download settings for the account they belong to. Where we could, we used the tag ID from the vendor's own website (for example Contentsquare, Mouseflow, Lucky Orange, PostHog, Convert, Optimizely, Kameleoon and Wingify use their own tools on their own sites). For Microsoft Clarity, VWO's classic SmartCode, AB Tasty and Matomo Cloud we used tag IDs from public customer sites. We don't name those sites. Clarity's main script is the same file for every site; the testing tools' size depends on the account.

Domain matching. Some tags only fully load on the domain they're registered for. So we served our blank test page locally at an address on that domain (for example https://www.convert.com/__script-weight-test) using the browser's request interception. Nothing was requested from or sent to those websites.

Like a real visitor. Fresh browser profile and empty cache for every run, a normal Chrome user-agent (some tags skip automated browsers), a few mouse moves, a click and a scroll, then 10 seconds of waiting so lazy-loaded modules arrive.

No data sent. We blocked every data-collection request (POST requests, beacons, tracking pixels). We only counted downloads of code and configuration (scripts and their settings files).

What we recorded:

• Transferred: bytes over the network for the tool's own domains, including headers, with whatever compression the vendor's CDN chose (mostly brotli, sometimes gzip).
• Uncompressed: the size of the code after unpacking. This is roughly what the browser has to parse.
• Requests: number of downloads from the tool's domains.
• CPU time: Chrome's "script duration" metric for the page, with the CPU slowed down 4× to imitate a mid-range phone. On a blank page, almost all of it is the tool. Treat it as indicative: it varies between runs and machines.

Repeats: 5 runs per tool; we report the median. Units: 1 kB = 1,024 bytes.

Varly. Varly's main script needs a site key, and Varly is in private beta, so there's no public site to load it from. We served the exact deployed build (the same bytes the CDN serves) with a minimal site configuration and consent already given, and measured it the same way. Its recorder and survey scripts were downloaded from the live CDN. Varly's transferred size is the gzip size, which is what its CDN serves.

Results

Heatmap and session recording tools

* Mouseflow's run on its own site's configuration also triggered two small lookups to third-party IP services; we didn't count those bytes. Varly's row shows the size on November 5, 2026. Since then, custom JavaScript for test variants, rules for which visits to always record, support for more cookie banners, flicker-free variants, test targeting and page speed measurement were added, and the main script grew to 22.7 kB gzip. After that, the code that sends tests to GA4 and other tools and the survey timing moved into files that load only when needed, so it went down to 20.9 kB gzip. Tests across several pages then added 0.4 kB: on November 7, 2026 the main script is 21.3 kB gzip (56.1 kB uncompressed). Targeting rules such as country or UTM add 1.4 kB, only on sites where a test uses them. Form analytics (about 6 kB) loads only on pages with a form.

** The Crazy Egg account we could test had no active tracking, so only the 4 kB loader ran. When tracking is active, the loader adds a common tracking script of about 64 kB (gzip), which we downloaded directly. We couldn't measure its CPU time.

On-site surveys

A/B testing tools

*** Kameleoon's own account also loads a Google tag (199 kB) through an integration, which inflated the CPU time. We left that tag out of the bytes.

For scale: a Google Analytics 4 tag (gtag.js) for one property on a public site transferred about 193 kB (595 kB uncompressed) in the same setup. Analytics tags are heavy too.

Like for like: gzip only

CDNs choose the compression. Most vendors serve brotli; Varly's CDN serves gzip. To compare like with like, we also downloaded the main script files with gzip only:

This is why the same three-tool stack is about 145 kB here and about 130 kB as transferred: Hotjar serves brotli, which makes its files 65 kB instead of 80 kB. Varly's main script is about 21 kB gzip in the current build (measured 13.7 kB on November 5, 2026, before custom JavaScript for test variants, targeting and more cookie banner support were added).

Testing tools grow with every experiment

Look again at the testing table: the same type of tool ranges from 61 kB to 278 kB. That's not only code quality. Testing tools bundle the account's configuration into their download: every active experiment, audience, goal and variation code. Convert says so in its docs: the base script is "~50 kB (compressed, over network)" and "Script size increases with the number of active experiences, locations, audiences, and goals in the project" (Convert).

That's why vendor sites (with many demo experiments) and big customers produce large tags, and why your own number may be lower or higher. Three practical consequences:

1. Archive finished experiments. Paused and finished tests often stay in the script until you archive them.
2. Measure your own tag, not a vendor's claim. A size on a marketing page and what your account actually downloads can differ a lot, because most tags also load a configuration file that grows with your tests and settings.
3. Watch anti-flicker settings. The official VWO SmartCode 2.1 we tested contains settings_tolerance=2000 and hide_element='body': it hides the whole page for up to 2 seconds while it loads its settings. On a slow connection, that's 2 seconds of blank screen.

Main-thread time across millions of sites (HTTP Archive)

Our CPU numbers come from a blank page. For a view across the real web, the open-source third-party-web project analyzes Lighthouse runs from the HTTP Archive (mobile, throttled CPU). Its March 2026 dataset reports average script execution time per page for each tool's domains (third-party-web data):

These numbers are much larger than ours because real pages are busy, Lighthouse's mobile simulation is slow, and the tag's work grows with the page (more elements to watch, more events). Some domains are incomplete: many VWO installs load from visualwebsiteoptimizer.com, which isn't counted under vwo.com. Use the table for rough comparisons, not as precise figures.

What this means in practice

A typical three-tool stack. In our test, Hotjar (65 kB) + Convert (61 kB) + Zigpoll (4 kB) transferred about 130 kB across 5 requests before any survey or test variant appeared (about 145 kB with gzip for all three, like for like). With VWO's SmartCode instead of Convert, it was about 298 kB. The same jobs in Varly: about 21 kB gzip for the main script in the current build, plus about 29 kB only for visits that are recorded and 12 kB only when a survey shows.

Bytes aren't everything. A small synchronous script that blocks rendering can hurt more than a large one that loads after the page. Check how a tool loads (async or blocking, anti-flicker or not) as well as how much.

Compare with your page. If your product page already loads 3 MB of images and 1 MB of JavaScript, a 26 kB tag is noise. If you worked hard on a lean theme, 300 kB of testing tag isn't.

Mobile is where it hurts. Uncompressed size is a decent proxy for parse and run time. 1 MB of uncompressed JavaScript is a lot for a mid-range phone.

How to measure your own site in 5 minutes

With Chrome DevTools:

1. Open your product page in Chrome, right-click → Inspect → Network tab.
2. Tick Disable cache and reload.
3. Type the tool's domain in the filter box (for example hotjar, clarity, convert).
4. Read the bottom bar: requests and "transferred" (compressed) vs "resources" (uncompressed).
5. Repeat after accepting cookies: many tools only load after consent.

With Lighthouse (in DevTools): run a mobile report and look at Reduce the impact of third-party code. It lists each third party's transfer size and main-thread time on your page.

Before/after comparison: use DevTools' Request blocking (or WebPageTest's block option) to block one tool's domain, and compare LCP and total blocking time with and without it. That shows what the tool costs on your page.

How to reduce the cost

1. Remove tools you don't use. The old heatmap tool from two agencies ago, the survey nobody reads, the testing tag with no running tests. Removing is the biggest win.
2. Archive finished experiments so they leave the testing tag.
3. Load research tools after consent. In the EU you need consent anyway (GDPR and CRO); visitors who decline then don't download the tool at all.
4. Record a share of visits on busy sites. Tools that load their recorder only for recorded visits then save bytes for everyone else.
5. Check anti-flicker settings. Shorten the timeout, or hide only the element being tested instead of the whole page. Shopify recommends removing A/B testing scripts when no tests are running (Shopify).
6. Load testing scripts early but small. A testing script has to run early to avoid flicker, which is exactly why its size matters most.
7. Consider one tool instead of three if one does what you need. Each vendor brings its own runtime, its own settings download and its own event handlers.

When Varly is not the right choice

We built Varly to keep script weight low, and we measured it the same way as everyone else. But be fair to the numbers: our test used a minimal site configuration, while several competitors were measured with their own, heavily used accounts. A Varly site with many surveys and tests will download a bit more configuration. And if you need a mobile-app SDK, US data residency or a tool you can buy today, Varly isn't for you yet: it's in private beta.

Raw data and change log

Per-file data (November 5, 2026, median of 5 runs; transferred includes headers)

Tool	File	Encoding	Transferred	Uncompressed
Clarity	clarity.ms/tag/{id}	—	1.2 kB	0.8 kB
Clarity	scripts.clarity.ms/0.8.75/clarity.js	br	24.9 kB	71.5 kB
Hotjar	static.hotjar.com/c/hotjar-{id}.js	br	7.4 kB	18.4 kB
Hotjar	script.hotjar.com/modules.{hash}.js	br	57.5 kB	234.7 kB
Contentsquare	t.contentsquare.net/uxa/{id}.js	br	6.9 kB	17.8 kB
Contentsquare	t.contentsquare.net/releases/15.281.1/…js	br	208.7 kB	732.7 kB
Mouseflow	cdn.mouseflow.com/projects/{id}.js	br	59.1 kB	194.5 kB
Lucky Orange	lo.js, settings, botd.js, web-vitals.js, core.js, 2 integrations	gzip	135.9 kB	435.7 kB
Lucky Orange	messenger (chat) files incl. chunk-vendors.js (302 kB)	gzip	367.8 kB	1,276.7 kB
Crazy Egg	pages/scripts/{id}.js + site data	gzip	4.0 kB	9.5 kB
Crazy Egg	common-scripts/{hash}.js (direct download)	gzip	64.0 kB	200.6 kB
FullStory	edge.fullstory.com/s/fs.js	br	82.4 kB	308.0 kB
PostHog	static/array.js + array/{token}/config.js	br/gzip	162.0 kB	565.6 kB
PostHog	recorder, surveys (+ API), conversations, logs, dead clicks, web vitals, exceptions	br/gzip	208.6 kB	700.8 kB
Matomo Cloud	cdn.matomo.cloud/{site}/matomo.js	br	44.1 kB	152.9 kB
Zigpoll	cdn.zigpoll.com/static/js/main.js	gzip	4.2 kB	10.5 kB
VWO 2.1	j.php, settings.js, va_gq, nc, worker	gzip/br	228.5 kB	951.0 kB
Wingify 3.0	tag/{id}.js (150 kB) + 9 more	br/gzip	277.9 kB	1,172.4 kB
Convert	v1/js/{acct}-{proj}.js + optional settings	gzip	61.4 kB	201.8 kB
Optimizely	cdn.optimizely.com/js/{id}.js + ODP + storage iframe	gzip	127.6 kB	450.9 kB
AB Tasty	loader, main.{hash}.js (90 kB) + 6 more	br/gzip	111.7 kB	609.6 kB
Kameleoon	engine.js + map, geolocation, live-ff config	br/gzip	74.4 kB	333.3 kB
Varly	main script (gzip, deployed build)	gzip	13.7 kB	38.7 kB
Varly	cdn.varly.ai/rec.js	gzip	24.2 kB	77.5 kB
Varly	cdn.varly.ai/survey.js	gzip	7.7 kB	19.9 kB

Change log

• December 3, 2026: first edition (measured November 5, 2026). We plan to rerun the study every six months with the same method.

FAQ

Does Hotjar slow down my site? In our test, Hotjar's classic tag transferred 65 kB (253 kB uncompressed) and loads asynchronously, so it doesn't block the page from appearing. Its effect on interaction speed depends on how busy your page is.

Is Microsoft Clarity lightweight? Relatively: 26 kB transferred in our test. Across real sites, the HTTP Archive data shows an average of 405 ms of script execution per page in Lighthouse's mobile simulation.

Why are A/B testing scripts so heavy? They contain your experiments, audiences and goals, and they must run early to avoid flicker. Archive old experiments to keep them small.

Should I remove anti-flicker snippets? Not blindly: without them you may get flicker. But keep the timeout short, hide only what you test, and remove the testing tag when no test runs.

Can I reproduce your results? Yes. The method above uses standard tools (Chrome, Playwright, DevTools). Your numbers will differ by account, configuration, location and date.

---

Varly puts heatmaps, recordings, surveys and A/B tests in one script of about 21 kB. It's in private beta: join the waitlist.

Related: The A/B test flicker effect · Hotjar alternatives for ecommerce · VWO alternatives · What a CRO tool stack really costs · How to analyze heatmaps