Vector Stores, Hybrid Search, and Citations
Search by meaning, combine it with keyword search, refuse when nothing matches, and make every answer traceable to a source.
Objective
By the end of this lesson, you can explain how a vector store finds passages, set a similarity floor so the system can say "I don't know," and design citations a reader can check.
Vector stores
A vector store holds embeddings and finds the nearest ones to a question's vector. Options range from an extension in a database you already run (such as pgvector for PostgreSQL) to dedicated vector databases and managed cloud services.
For most community-scale projects, a few thousand to a few hundred thousand chunks, start with the database you already have. Fewer systems mean fewer things to secure and back up. Move to a dedicated store when you measure a real need.
Things to check in any store:
- Metadata filtering: search only chunks the user may see, in their language, from current documents.
- Deletion: when a document is withdrawn, its chunks must disappear too.
- Backups and access control: the index holds copies of your documents. Protect it like the originals.
Retrieval with a floor
The first run downloads Python into your browser (several MB).
The similarity floor is a minimum score. Below it, the system does not answer from documents. Set it by testing on real questions, not by guessing.
Hybrid search and reranking
Keyword search (often BM25, a standard ranking method) is strong on exact terms: names, program codes, form numbers. Vector search is strong on meaning. Hybrid search runs both and merges the results. It often beats either alone on real documents.
A reranker is a second model that re-scores the top results more carefully. It adds a little delay and often improves which passage lands first.
Add these only when your retrieval evaluation (next lesson) shows a need.
Citations people can check
A citation is only useful if a reader can follow it. Design for that:
- Cite the specific source: document title and section, with a link when possible.
- Show the date of the source, so readers know if it may be out of date.
- Check the citation in code: the cited chunk ID must be one that was actually retrieved.
- Spot-check that the cited passage supports the claim. Models can attach real citations to wrong statements.
Grounded answer prompt
Write the generation prompt for a RAG system that answers from retrieved passages, cites them, and refuses when they are silent.
Your prompt
Act as an assistant that answers only from the numbered passages provided.
Quiz: Retrieval and citation check
Go deeper: Business Intelligence for Real Operators: Turning CRM & ERP Data into Decisions
Discussion
Ask. Answer. Build together.
Loading…