I Reverse-Engineered Exa.ai Infrastructure Cost with Napkin Math

October 2026 note. I wrote this in late 2025 using 20B indexed pages and 300M monthly requests as round assumptions. Exa now reports a 100B-page served index and 2B requests per month. That's 5x the page count and about 6.7x the average request rate used in the math below. The search and cost fundamentals still apply at this larger scale.

Looking for the short version? Check key takeaways

Exa is a cool web search engine company. They recently passed $100M in annualized revenue and raised $250M in a Series C. There are multiple parts of Exa AI architecture that they have described in their blogs. I've compiled them together and added some of my own notes and lots of assumptions for infra cost napkin math as a fun exercise for myself. Please take my estimates with a grain of salt since it's not possible to be 100% right and my goal is to just get in the right ballpark. Also feel free to DM me on X or Linkedin if you see any wrong or suboptimal assumptions :)

📦 Content Storage

Let’s estimate the storage requirements for large-scale text datasets:

• Fineweb
  • The current default dataset has 25.9B documents in 53.4 TB of Parquet files, or about 2.06KB per document. The files also include metadata, so this is only a rough proxy for page storage.
  • The dataset card reports over 18.5T GPT-2 tokens. At 1.3 tokens per word, that is roughly 14.2T words.
    • 1 doc ~= 712 tokens ~= 547 words.
    • English words on avg have 5 chars, so 547 * 5 bytes ~= 2.7KB / doc before spaces and punctuation.
    • 1B docs ~= 2.7TB (estimated uncompressed)
• English Wikipedia
  • 7M docs occupy 24 GB (compressed), 1B docs ~= 3.42 TB (compressed)
  • It has 708 words / doc on avg. Assuming the standard rule of 1 word = 1.3 tokens, this means 920 tokens / doc on avg.
    • 1 doc ~= 920 tokens ~= 708 words
    • 708 * 6 bytes (reason) = 4248 bytes
    • 1B docs => 4.2TB (estimated uncompressed)

Given these, let’s assume Exa’s crawlers curate pages and extract their text efficiently enough such that:

• They are similar to Fineweb but a little more dense.
• 1B docs require at least 3TB of letters alone (uncompressed) and around 2.1TB compressed.
• On avg 1 page ~= 600 words ~= 780 tokens. At 5 bytes per word, the letters alone take 3KB before spaces and punctuation. I'm using 2.1KB per compressed page as a rough assumption.

The SSD rate is my allocation assumption. On AWS, instance-store SSD comes with the instance, rather than having a separate per-GB bill.

🔍 Lexical/Keyword search (BM25):

• They use the standard BM25 algorithm for lexical search
• Initially, this required about 1.8 TB RAM for 1B docs
• After optimizations in this post it dropped to 900GB (-50%) RAM for 1B docs

Napkin math

• Postings lists take a lot of BM25 memory, but the data structures around them also add overhead.
• As per their blog, each posting list entry takes:
  • doc_id → 4 bytes (uint32)
  • term_freq → 1 byte (fp8)
  • 5 bytes total
• Dividing 1.8 TB by 5 bytes gives 360 entries per doc, but only if postings were the entire index. Exa also counts padding, allocations and metadata in that RAM figure, so it doesn't tell us the actual number of unique terms per doc.
• If 10M vocabulary terms averaged 5 bytes each, their strings alone would take 50MB. The lookup structures and posting lists add more.
• Exa used uint32 document IDs in that design. IDs can be local to each shard, so this doesn't tell us the total number of indexed documents. Delta encoding later reduced the average ID size.
• I'm assuming that 100% of index is kept in RAM for faster iteration on posting lists (~2-5x faster compared to sequential reads in SSD - source)

🧭 Vector search:

• As per this, they use binary quantization (BQ) and some clever CPU tricks for vector distance calculations (SIMD + precomputing possible outputs and keeping them in CPU registers for skipping calculation).
• They train Matryoshka embeddings with 4096 dims but only keep first 256 dims BQ vectors for index
• They use IVF Index (also used in FAISS & pgvector) which an Approximate Nearest Neighbours Algorithm. It basically relies on k-means clustering, and during search time, finds the relevant clusters by comparing distance with respective cluster centroids, and only queries few of those clusters.
• For the storage estimate, I'll assume a first re-ranking stage with 1024d BQ vectors and a second stage with 2048d fp16 vectors. Exa describes re-ranking with less compressed vectors, but these exact sizes are my assumptions.

Napkin math

• I'll use Qwen3-Embedding-8B as a compute cost proxy. Exa describes 4,096-dimensional embeddings, but the dimension alone doesn't identify its model.
• I'll assume one vector per page. Longer pages could need chunking, so the real vector count might be higher.
• Let's estimate the storage requirements of vectors
  • Each original vector would take 4096 dims * 2 bytes (assuming fp16) = 8KB by default. This means 1B vectors would take 8 * 10^3 bytes * 10^9 = 8 * 10^12 = 8 TB for 1B docs.
  • By truncating matryoshka to 256 dims it becomes, 8 / (4096 / 256) TB = 8 / 16 TB = 512 GB for 1B vectors.
  • With binary quantization, 16 bits (since fp16) becomes 1 bit. So 512 GB / 16 = 32 GB per 1B vectors or 32 bytes (256 bits) per vector
  • Note how MRL truncation + BQ led to 16*16=256x reduction
• For IVF index with MRL truncated 256d BQ vectors for their "neural search"
  • IVF Index needs centroids generated from k-means clustering algos, If they have 20B vectors, they should have 200k clusters, each having 100K vectors since it becomes very hard to brute-force compare vectors in segments larger than 100k within their latency budget.
  • Each cluster will have a centroid and I'm assuming they compare with uncompressed centroid vectors for maximizing recall.
  • Centroids will require 200k * 4096 * 2 bytes (f16) ~= 1.64 GB
  • If IDs take 8 bytes each, 20B * 8 bytes = 160 GB. This is an assumption for the estimate, since shard-local IDs could be smaller.
  • Each 256d truncated + BQ compressed vector will take 32bytes (256 bits), so 20B * 32 bytes = 640 GB
  • So total requirements are like 1.64 + 160 + 640 GB ~= 802 GB for 20B vectors
    • This is 40GB for 1B vectors
    • Index size is just 40/8000*100 = 0.5% of the full vectors (nice!)
• For 1st stage re-ranking with 1024 dim MRL truncated BQ vectors:
  • Storage for 1B vectors takes 128GB (1024 / 8 * 1B / 1e9)
• For 2nd stage re-ranking with 2048 f16 vectors:
  • Storage for 1B docs takes 4000GB (8TB/2). This is okay for disk but storing all of them in RAM would be too costly.
  • I'll assume the frequently accessed 25% (1000 GB) is cached in RAM. The actual hot set could be different.
• Some of these components must be cached in RAM for fast queries and all the components must be kept on local (ephemeral) disk for paging in/out of memory quickly and handle a node crash. The ultimate source of truth can be S3 which is regularly updated to keep the index, vectors, and content fresh.

Note how this is 2.8k / 1.9k = ~1.5x of lexical search. The 2048d vector RAM cache contributes $2,000, or about 73% of the vector total under these assumptions.

🗂️ Metadata storage:

• For each web page, they also need to store metadata like title, url, publishedDate, author, id, image, favicon, text, summary, etc.
• Their search API example returns page attributes that can be stored in < 300 bytes. It's a tiny overhead so let's assume 512bytes (0.5KB)
• So total metadata storage on disk + S3 is 1B docs * 0.5KB = 0.5TB for 1B doc metadata
• Assuming that metadata for the same 25% points are cached, we need 125GB of RAM

🌐📤 Network egress costs:

The APIs respond back to the user and that also costs money. Napkin math says it's $0.1 per GB for "internet egress"

I did a basic search API call with Exa and response size was around 3KB. If they serve 300M requests / month, that's about 116 requests/s on average. I'll assume a provisioned peak of 300 requests/s.

3 KB * 300M => 900 GB of egress => 900 * 0.1 $ = 90 $ / month for 300M requests which is very cheap!

🔢 Embedding generation:

• As we previously derived/assumed, let's say each Exa doc has 780 tokens.
• If they were to embed 1B docs with OpenAI text-embedding-3-large embedding in batches (see pricing)
  • 780 * 1B * 0.065 $ / M token = 50.7k $ for 1B docs (insane!)
  • At this price, OpenAI's batch API costs about 3.9x the rented-H100 proxy below. Exa uses its own model.
• As per this blog, assuming 100% utilization with H100 GPUs rented at $ 2 / hr, the cost is $ 0.0166 / 1 M tokens.
  • 780 * 1000 * 0.0166 $ / M token ~= 13k $ for 1B docs.
  • This is much better! However, notice that this still surpasses all the costs.
  • Exa built its Exacluster for model training and embedding work:
    • One of the cheapest H100 is offered at 1.87$ / hr. But general pricing is 2-4$ / hr.
    • I'll assume 1.50 per productive GPU-hour for this scenario. Scaling the rented-GPU proxy gives 0.0166 / 2 \* 1.5 = 0.0125 \ / 1M tokens. Exa's actual cost also depends on power, hosting and utilization.
    • Exacluster has H200s & A100 instead of H100s. But let's assume the same rates.
    • Note that the Exacluster is also used for training embedding model and re-ranker so there's more value to be derived from the Exacluster.

At these assumed GPU rates, Exacluster saves $3.25k per 1B docs embedded**, or **$65k for one complete pass over 20B docs. The saving depends on how much of the cluster's time and cost goes to embedding.

Re-ranker:

• They might be using something like the Qwen Rerankers
• Public providers charge $ 0.010 / 1M tokens for the 0.6B model. Let's assume the same rates.
• It compares 1 query (same 10 tokens) against 100 documents (each with 780 tokens avg). Total tokens: 100 * (780 + 10) = 79k
• Note: This is purely a search time cost unlike document embeddings which have to be stored/maintained beforehand.

At an assumed $5 per 1k queries, the$0.79 small-reranker example would take about 16% of the price. This doesn't tell us which model Exa uses or how many candidates it re-ranks.

🕷️ Crawling updates

• Web pages change at different rates, and many changes don't require a new embedding.
• There might be a single digit monthly growth of the dataset - say 2% (assuming aggressive crawling)
• Content disappearance: ~0.5% monthly
• Overall let's assume 30% of documents need new embeddings each month as a scenario for freshness and growth.

💰📊 Overall costs:

For this estimate, I'm assuming 20B docs based on Exa saying it crawled and parsed tens of billions of pages around the Exa 2.0 launch.

Storage and RAM costs recur on the full 20B-doc index each month. Only the embedding row uses the assumed 30% refresh rate. The egress row uses the separate assumption of 300M requests per month.

Notes:

• The first complete embedding pass costs $195k at the assumed rate. That is a one-time compute job. The storage rows above recur every month.
• These assumptions give about $162k/month, or$1.94M/year, before re-ranking, crawling, clustering, replicas and query compute.
• The total also leaves out Exacluster costs beyond my assumed GPU-hour rate, query CPU and GPU work, LLMs for websets, and observability.
• At a much larger index, the storage and refresh costs in this model would rise with the number of indexed pages. The rates here are assumptions, so I wouldn't treat the result as a minimum bill.

⏱️ Search latency analysis:

Network latency:

I queried Exa AI DNS records and looks like they are only present in AWS us-west-2 region. This has downsides because it increases latency a lot. The round trip time from India is 600-800ms. You can find ping times from all over the world here. There are two solutions to this:

• Expand into new regions. But it's very costly to replicate whole infra in different region.
• Use something like Cloudflare Edge network. It basically proxies the requests through Cloudflare edge network which is faster and more stable than traditional internet route. It's not zero overhead, but is definitely a big improvement.

Update: Looks like Exa has switched to Cloudflare Edge like Perplexity with Exa 2.0. This means the latency across the globe has decreased. I see that now the round trip latency for empty request from India is at 300-400ms. Good job, Exa team!

You can also use these commands to get rough idea of latency:

time curl --head https://api.exa.ai/search # from your machine
globalping http api.exa.ai from World --host api.exa.ai --path '/search' --limit 20 --latency # globally

Lexical search latency

The Exa blog says they retrieve thousands of top results with lexical search in under 500ms. It doesn't label that number as p50, and it doesn't tell us which retrievers run in Fast mode.

BM25 latency can vary more than vector search in some setups, especially with:

• Longer queries (More posting lists to scan. This downside can be saturated using wAND but you still see more variance in latency)
• More data (less pronounced. But this happens especially if you get common but non-stopword terms)

For example effect of longer queries part can be seen in this benchmark:

You can see how average latency is similar here, while lexical search has 7.3x higher p999 latency. This is one benchmark with its own corpus and hardware, so the ratio doesn't predict Exa's latency.

Also, if Exa operates with 20B+ docs (with Exa 2.0), I can imagine that they split 20B docs into multiple "posting list shards" and query them in parallel and combine results afterwards.

Neural search latency (Exa fast)

Previously Exa Fast search had p50 = 423ms and p95 = 604ms in its published chart.

With Exa 2.0, the chart shows p50 = 346ms and p90 = 462ms. The p50 values differ by 77ms, although the benchmark regions also changed. We can't compare the old p95 with the new p90 directly.

Here are the main steps I would expect in a Fast search. This is a pipeline sketch, not a measured breakdown of the 423ms:

• Network round trip and request handling.
• Query embedding and retrieval from the vector index.
• Fetching candidates and re-ranking them.

The public p50 covers all of these steps. Without timings for each one, I can't assign the 77ms difference to a particular optimization.

What if Exa used HNSW instead of IVF?

• In this billion-scale KNN benchmark, HNSW reaches about 4ms average search latency at 90% recall for top 10 results on one setup. The same article shows that maintaining the graph reduces indexing throughput.
• IVF searches selected clusters. HNSW follows graph links. Their latency and recall at 20B pages depend on the shard layout, hardware, filters, update rate and recall target. A small-scale benchmark alone can't give us Exa's latency at that size.
• Similar flaws of scaling IVF were also demonstrated in my previous benchmark blog with Nirant that compared Qdrant's HNSW and PGVector's IVF Index and our blog accelerated the implementation of HNSW in pgvector.
• If Exa switched to HNSW, graph memory and indexing work would be added to the bill. Whether search got faster at the same recall would need a like-for-like benchmark.

Key takeaways:

• Exa applies heavy compression through matryoshka embeddings and binary quantization, then offsets the loss with uncompressed vectors and re-ranking.
• In the query pipeline, compressed vector search keeps the index small. Re-ranking more candidates can add substantial work per request.
• Documents change over time, so embedding work keeps recurring. At the assumed 30% monthly refresh, the 20B-page model lands at about $162k/month before crawling, replicas, training and query compute.

Closing remarks:

Web search started in 1994 with Yahoo directories, then search engines like Alta Vista introduced Keyword search using TF-IDF in late 1990s, and then eventually Google came in 1998 and dominated the game with Pagerank. Over the next two decades, Google added lots of heuristics and statistics based approaches. Then around 2015 they started integrating ML models such as RankBrain and BERT, enabling semantic understanding and context-aware ranking.

And very recently, we see that even those are being surpassed by more generalizable LLMs. Infact, now you can ask more complex queries to the web which were impossible with previous techniques. It's literally Bitter lesson 101 which says approaches that scale with compute beat human designed heuristics.

We’re entering an era where startups are on their way to recreate Google scale search and number of searches will rise exponentially because of AI agents driving majority of search request. That's why products like Exa, Parallel, and Perplexity are effectively "caching" the web and serving "reads" in an optimized way. While others like Perplexity Comet, OpenAI Operator, etc want to "write" to the web (i.e. take actions). Web and search are changing fast and I'm very excited about their futures!

Acknowledgements:

Thanks to my friend Nirant for reviewing the blog.

• https://github.com/sirupsen/napkin-math
• https://www.tensoreconomics.com/p/why-are-embeddings-so-cheap
• https://www.youtube.com/watch?v=y72AhP4oLZE
• https://exa.ai/blog/building-web-scale-vector-db
• https://exa.ai/blog/bm25-optimization
• https://exa.ai/blog/fastest-search-api
• https://exa.ai/blog/evals-at-exa
• https://exa.ai/blog/websets-evals