Data Intelligence · RAG · Semantic Search · Production

Weights, prompt or index: choosing where a fact lives when the corpus grows daily

Published · Updated

A daily publisher adds facts faster than any model can be retrained. Routing each class of fact to the store that can actually be updated is a design decision, not a retrieval-tuning problem.

Tatano Energy runs a multilingual e-commerce platform across four country domains with a daily SEO autoblog: 8 SEO articles published every day, 7 languages served, 0 manual intervention required. A system like that produces new facts continuously. Before asking how retrieval is tuned, ask a plainer question: when a new fact appears, which store is supposed to hold it, and what does writing to that store cost?

Three stores, not two

The RAG paper frames this as two memories. Lewis et al. introduce models where the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever. The split matters because the two behave differently under change. One is fixed at training time. The other is a database you can write to.

In practice, there is a third store that gets used without being named: the prompt. Anything you paste into the context window is knowledge the model has for exactly one call. It is the cheapest store to write and the most expensive to read repeatedly, and, unlike an index, it has a hard ceiling.

The prompt has a ceiling you can measure

Anthropic states the threshold directly: if your knowledge base is smaller than 200,000 tokens – about 500 pages of material – you can just include the entire knowledge base in the prompt, with no need for RAG or similar methods. That is a real and underused answer. Many internal corpora never outgrow it, and skipping an index removes an entire class of retrieval failures.

A daily publisher is the case where this answer expires. A corpus that gains articles every day, in several languages, moves in one direction only. The useful move is to know which side of that threshold you are on and to check it as the corpus grows, rather than building retrieval on day one out of habit.

Updating is the part that decides the design

Lewis et al. are explicit that providing provenance for their decisions and updating their world knowledge remain open research problems. Read that as a routing rule. Any fact that changes on a schedule you control – product details, prices, this week's articles – should not live in model weights, because you have no write path to it between training runs.

The same paper reports that RAG models generate more specific, diverse and factual language than a parametric-only baseline, and set the state of the art on three open-domain QA tasks. The gain comes from reading a store that can be updated. If your non-parametric store is stale, you have paid for the architecture without getting the property it exists to provide.

What each write costs

On the Postgres side, the write cost is visible. pgvector performs exact nearest neighbour search by default, which provides perfect recall; you add an index to use approximate search, which trades some recall for speed. HNSW has better query performance than IVFFlat on the speed–recall trade-off, but slower build times and more memory. When rows arrive every day, build time is a recurring bill, not a one-off.

Text search has its own write-time step. Postgres provides tsvector for storing preprocessed documents and tsquery for processed queries, and full-text indexing lets documents be preprocessed and an index saved for later rapid searching. The documentation notes that LIKE and ILIKE lack many essential properties required by modern information systems. Preprocessing at write time is what buys the properties.

Routing a fact, not a question

Two attributes decide the store: how often the fact changes, and whether it has to be matched exactly. Embedding models excel at capturing semantic relationships but can miss crucial exact matches, which is why a part number or a model name belongs in a lexical index as well. A fact that never changes and is never quoted is the only one a frozen store can safely hold.

Then measure the store you chose. Anthropic uses 1 minus recall@20, the percentage of relevant documents that fail to be retrieved within the top 20 chunks, and reports that contextual retrieval reduces failed retrievals by 49%, and by 67% when combined with reranking. Note what moved there: what gets written into the store, not only how it is queried.

Write the routing table down before tuning anything. For each class of fact, name the store, the write path and who triggers the write. Facts with no write path are the ones that will go stale silently, and a retrieval metric will not tell you that, because the chunk it returns is still the best match for a corpus nobody updated.

Sources

pgvector (GitHub) — Open-source vector similarity search for Postgres — https://github.com/pgvector/pgvector

arXiv — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., arXiv:2005.11401) — https://arxiv.org/abs/2005.11401

Anthropic — Introducing Contextual Retrieval — https://www.anthropic.com/engineering/contextual-retrieval

PostgreSQL 18 Documentation — Full Text Search: 12.1. Introduction — https://www.postgresql.org/docs/current/textsearch-intro.html

Neurolinks case study — Four markets, one codebase — https://neurolinks.be/work/tatano-energy

Working on a project where these methods apply?