AI Credits vs Bring Your Own API Key: Real CRO Costs

Two ways tools charge for AI

Most heatmap, survey and A/B testing tools now have AI features: summaries of recordings, themes from survey answers, test ideas, even generated test variants. Somebody pays the AI provider for every one of those requests. There are two common models.

1. AI included (credits or limits). The vendor pays the AI provider and builds that cost into your plan. You usually get a monthly allowance: a number of "credits", AI summaries or tool calls. For example, Contentsquare's free plan includes 300 AI tool calls a month for connecting an AI assistant (Contentsquare). When you run out, you wait for next month, buy more or upgrade.

2. Bring your own key (BYOK). You create an account with an AI provider, such as Anthropic (the company behind Claude), and paste an API key into the tool. An API key is a long password that lets the tool use the AI on your account. The provider bills you directly for what you use; the tool doesn't charge for AI on top.

Neither model is a scam, and neither is automatically cheaper. With credits, the vendor carries the risk of heavy users and usually adds a margin, but you get one invoice and no setup. With BYOK, you see the real cost and pay exactly that.

How AI usage is priced: tokens in plain English

AI providers charge per token. A token is a piece of a word. You pay for:

• Input tokens: everything the tool sends to the AI: your instructions plus the data (survey answers, a recording's event log, heatmap numbers).
• Output tokens: what the AI writes back. Output is more expensive than input.

Prices are quoted per million tokens ("MTok"). Anthropic's list prices on November 5, 2026 (Anthropic pricing):

Anthropic also offers a Batch API at a 50% discount for work that doesn't need an answer right away.

How many tokens is a word? It depends on the model and language. Anthropic says its newer models use a tokenizer that produces about 30% more tokens for the same text than older ones (Anthropic pricing FAQ). For the examples below we assume 1 English word ≈ 1.75 tokens. German and Dutch text usually needs more tokens per word, so add a margin if your customers write in those languages.

Example calculations

These are worked examples, not quotes. Real costs depend on how much data the tool sends and how long the answers are. All prices are Anthropic's list prices of November 5, 2026, without batch or caching discounts.

1. Summarize 1,000 survey answers

Assumptions: 1,000 open answers of 25 words on average, about 10 words of context per answer (page, device, date), plus 500 words of instructions. That's about 35,500 words, or roughly 62,000 input tokens. The AI writes a summary with themes and quotes of about 800 words (1,400 output tokens).

2. Summarize 50 session recordings

Assumptions: each recording is sent as a text log of what happened (pages, clicks, scrolls, errors), about 3,000 tokens, plus 1,000 tokens of instructions. The AI writes a 300-token summary per recording. Long visits produce longer logs.

3. Write 10 test hypotheses from your research

Assumptions: the tool sends a summary of heatmaps, click data, survey themes and analytics: about 20,000 input tokens. The AI writes 10 hypotheses of about 120 words each (2,100 output tokens).

4. Build one A/B test variant

Assumptions: building a variant usually takes a few rounds. We assume 3 requests, each sending about 60,000 tokens (the page's HTML and instructions) and getting 5,000 tokens of code back. This varies more than any other task.

5. A daily "research round" for a month

Assumptions: once a day the tool sends new research (30,000 tokens) and gets 2,000 tokens of ideas back. 30 days.

Putting it together: one shop, one month

Hypothetical month: 4 survey summaries, 50 recording summaries, 2 sets of hypotheses, 2 test variants and a daily research round.

Two lessons from these numbers:

1. The model choice matters more than the task. Haiku is about 40 times cheaper than Opus here. For summaries of short survey answers, a smaller model is often good enough. For test ideas and variant code, the bigger model is usually worth it.
2. Background jobs add up. In this example the daily research round is half the Opus bill. If you don't read the ideas every day, run it weekly or on demand.

The same month on five AI providers (list prices, November 2026)

At list prices checked on November 7, 2026, the same shop-month costs between $1.12 (DeepSeek, off-peak) and $9.29 (Claude Opus 5.5) with the models Varly recommends for each provider. We computed it for the five providers Varly accepts keys from, using each provider's own price page.

Method: the same hypothetical month as above, which comes to about 1.75 million input tokens and 115,000 output tokens. Summaries (surveys and recordings) are 448,000 input and 20,600 output tokens. Hypotheses, test variants and the daily research round are 1.3 million input and 94,200 output tokens. We priced it two ways: everything on the main model, and main + fast: summaries on the provider's cheaper "fast" model, everything else on the main model. That is how Varly splits the work when you pick a fast model (Add your own AI key). Standard prices, no batch or caching discounts. We keep the token counts the same for every provider, so this compares prices, not tokenizers.

DeepSeek prices are for input without a cache hit. OpenAI's most capable model, GPT-6 Astra ($10 / $50), would cost $23.22 for the same month.

What the numbers show:

• A fast model saves less than you might expect. On Anthropic, main + fast is $7.14 instead of $9.29, about 23% less. Summaries are only about a quarter of the input tokens in this month; the daily research round and test variants run on the main model either way. Running the daily round weekly saves more.
• Prices change during the year. Google's price page lists Gemini 4 Argon at double the price from December 1, 2026, and Gemini 3.8 Flash at double from January 1, 2027. The same month on Gemini then costs as much as on Claude Opus 5.5.
• The time of day matters on DeepSeek. DeepSeek charges double from 01:00 to 04:00 and 06:00 to 10:00 UTC, Monday to Friday. In winter that is 07:00 to 11:00 in Amsterdam or Berlin: a European morning of work falls partly in peak hours.
• Price is not quality. We didn't compare how good each model's ideas or variant code are. A cheaper model that writes weaker variants can cost more in your time than it saves in tokens. Try two models on the same task with your own data before you settle.

Advantages of bringing your own key

• You see the real cost. Every request shows up in your provider's console. No guessing what a "credit" is worth.
• No markup on AI. You pay the provider's list price. The tool earns its money on the software, not on reselling tokens.
• You choose the model. Switch to a cheaper model for simple work, or a better one when quality matters.
• You set the limits. A spending cap at the provider stops runaway costs, whatever the tool does.
• Your contract with the AI provider. The data terms are between you and the provider, not hidden in your CRO vendor's sub-processor list.
• No "out of credits" surprises in the middle of a client project.

Disadvantages

• Setup. Someone has to create a provider account, add a payment method and create a key. That's 10 minutes for a marketer who has done it before, longer for someone who hasn't.
• A second invoice. Finance gets a bill from the tool and one from the AI provider.
• Costs are variable. A busy month costs more. With a credit plan, the price is fixed (until you hit the limit).
• The key is a secret. Treat it like a password: don't paste it into chats or shared documents, and replace it if it leaks.
• Limited provider choice. Many BYOK tools support only one or two providers, so check that yours is on the list. Varly, for example, supports Anthropic (Claude), OpenAI (ChatGPT models), Google Gemini, DeepSeek and Mistral.
• New accounts start with low limits. AI providers raise limits as your account builds history (Anthropic rate limits).

Data and privacy terms with AI providers

Whichever model you choose, your visitors' data (survey answers, recording logs) is sent to an AI provider when you use AI features. Ask three questions:

1. Is the data used to train AI models? For Anthropic's commercial products, including the API: "By default, we will not use your inputs or outputs from our commercial products" to train models, with exceptions such as feedback you choose to send (Anthropic privacy center).
2. How long does the provider keep it? Anthropic says it automatically deletes API inputs and outputs "within 30 days of receipt or generation", with exceptions (for example a longer agreement, legal requirements, or usage-policy enforcement) (Anthropic privacy center).
3. What exactly does the CRO tool send? Good tools strip what isn't needed. Ask whether email addresses, names or full URLs with query strings are removed before anything goes to the AI.

With BYOK, the AI provider processes data under your account, so add it to your records of processing and, where needed, your privacy policy. With credits, it's one of your CRO vendor's sub-processors: check their list.

How to set a spending limit

Do this before you paste the key into any tool.

In the Anthropic Console (Anthropic docs):

1. Go to Billing (platform.claude.com/settings/billing).
2. Under Spend limits, click Set limit (or Adjust limit).
3. Enter a monthly amount below your tier's cap. Each usage tier has its own monthly cap: $500 on the Start tier, for example.

When you reach your own limit, requests stop until you raise it or the month resets. The AI features in your tool stop working; nothing else breaks.

Tip for agencies: create a separate workspace in the Anthropic Console for your CRO tool and give it its own key and spend limit. You can't set a spend limit on the default workspace, so a dedicated one keeps the CRO tool's spending separate from anything else your team uses the API for.

In your CRO tool: look for a usage overview. In Varly, the AI usage, last 30 days card shows research rounds, tests built, summaries and an estimated cost, and you can turn the daily research round off. Varly also has its own daily AI limit, $5 per day by default: a task that would go over it doesn't run until the next day (UTC) or until an owner raises the limit (Add your own AI key). That limit only covers use through Varly, so keep the limit at your provider too.

Questions to ask vendors

1. Do you charge for AI separately, include it in the plan, or use my own key?
2. If included: what's the monthly allowance, and what happens when I run out?
3. If credits: how many credits does each task use (one summary, one variant)?
4. Which AI provider and model do you use, and can I choose?
5. Is any AI feature running in the background without someone clicking a button?
6. What data do you send to the AI provider? Do you remove emails and names first?
7. Is the AI provider allowed to train on our data?
8. Can I see the cost per task, or at least per month?

How Varly handles this

Varly, currently in private beta, uses the BYOK model: an owner pastes an API key from Anthropic (Claude), OpenAI (ChatGPT models), Google Gemini, DeepSeek or Mistral once for the organization, picks a model (for example Claude Opus 5.5), and pays that provider directly at its own prices. Varly adds no AI markup. The planned pricing at launch is €99 per website per month (it may still change before launch), with AI usage billed separately by your AI provider. Without a key, Varly's fixed rules still turn research into ideas; summaries, ideas and test variants from Oran, the AI inside Varly, need the key. In an agency account, Oran only sees the research of the site it's working on, never other clients' data.

If you'd rather have AI fully included and never think about tokens, a tool with included credits may suit you better.

FAQ

Is bringing your own key cheaper? Usually, if the vendor would otherwise add a margin. For light use, both can be pennies. The bigger difference is transparency and control.

Do I need to be technical to create an API key? No. You create an account at the provider, add a payment method and click "Create key". Then copy it into the tool once.

Which model should I pick? Start with the mid-sized model (Sonnet) for everyday work. Use the largest (Opus) for test ideas and variants. Try the smallest (Haiku) for bulk summaries and check the quality.

What happens if my key stops working? The AI features pause. Your data and non-AI features keep working.

---

Want AI that runs on your own key from Anthropic, OpenAI, Google Gemini, DeepSeek or Mistral, with no markup? Varly is in private beta. Join the waitlist.

Related: AI for conversion optimization: what actually works · Connecting Claude or ChatGPT to your CRO data with MCP · What a CRO tool stack really costs