RAG Chunk & Token Calculator
Estimate tokens, chunk count, overlap overhead and embedding cost for a RAG pipeline.
e.g. the total size of your document corpus
Fills in the ratio and price. Both stay editable.
~4 for English prose
Check current pricing
10–20% of chunk size is a common starting point
- Estimated tokens
- 250,000
- at 4 chars/token
- Chunks
- 542
- last chunk 58 tokens
- Tokens per chunk
- 512
- 462 new + 50 overlap
- Embedding cost
- $0.0055
- 277,050 tokens embedded (+10.8% overlap)
Estimates only. Real token counts depend on the model's tokenizer and your content. Everything runs in your browser.
How RAG chunking and token estimates work
Retrieval-augmented generation (RAG) splits your documents into chunks, turns each chunk into an embedding vector, and stores the vectors for similarity search. How you chunk affects retrieval quality, how many vectors you store, and how much embedding costs.
Estimating tokens without a tokenizer
Modern BPE tokenizers average roughly 4 characters per token for English prose, or about ¾ of a word per token. Code, JSON and non-Latin scripts produce more tokens per character, so use a lower ratio for them (around 3 for code, 1–2 for Chinese or Japanese). The estimate is good for budgeting. For hard limits, count with the model's own tokenizer.
Chunk size and overlap
A sliding window of chunk size tokens moves forward by chunk size − overlap
each step. Overlap keeps a sentence that straddles a boundary intact in at least one chunk, but every
overlapping token is embedded twice:
chunks = 1 + ceil((tokens − size) / (size − overlap))
embedded ≈ tokens × size / (size − overlap)- 256–512 tokens: precise retrieval for FAQs, support docs and Q&A.
- 800–1,500 tokens: more context per hit for long-form prose and reports.
- Overlap of 10–20% is a common default. Beyond that you mostly pay for duplicates.
What embedding costs
Embedding APIs charge per input token, and the total is embedded tokens (overlap included), not your raw corpus size. Prices change often, so the price field is editable. Check your provider's pricing page. Running embeddings locally is free per token, and you pay in hardware and time instead. The chunk count is also the number of vectors your database stores.