lesson
60 min

Vector Stores, Hybrid Search, and Citations

Search by meaning, combine it with keyword search, refuse when nothing matches, and make every answer traceable to a source.

Objective

By the end of this lesson, you can explain how a vector store finds passages, set a similarity floor so the system can say "I don't know," and design citations a reader can check.

Vector stores

A vector store holds embeddings and finds the nearest ones to a question's vector. Options range from an extension in a database you already run (such as pgvector for PostgreSQL) to dedicated vector databases and managed cloud services.

For most community-scale projects, a few thousand to a few hundred thousand chunks, start with the database you already have. Fewer systems mean fewer things to secure and back up. Move to a dedicated store when you measure a real need.

Things to check in any store:

  • Metadata filtering: search only chunks the user may see, in their language, from current documents.
  • Deletion: when a document is withdrawn, its chunks must disappear too.
  • Backups and access control: the index holds copies of your documents. Protect it like the originals.

Retrieval with a floor

Run it. The parking question finds nothing above the floor, so the system says it does not know. Lower floor to 0.0 and run again. What does the parking question return now? Why is that dangerous?
Python 3
Ctrl+Enter to run | Tab to indent | Python runs in your browser via Pyodide

The first run downloads Python into your browser (several MB).

The similarity floor is a minimum score. Below it, the system does not answer from documents. Set it by testing on real questions, not by guessing.

Hybrid search and reranking

Keyword search (often BM25, a standard ranking method) is strong on exact terms: names, program codes, form numbers. Vector search is strong on meaning. Hybrid search runs both and merges the results. It often beats either alone on real documents.

A reranker is a second model that re-scores the top results more carefully. It adds a little delay and often improves which passage lands first.

Add these only when your retrieval evaluation (next lesson) shows a need.

Citations people can check

A citation is only useful if a reader can follow it. Design for that:

  • Cite the specific source: document title and section, with a link when possible.
  • Show the date of the source, so readers know if it may be out of date.
  • Check the citation in code: the cited chunk ID must be one that was actually retrieved.
  • Spot-check that the cited passage supports the claim. Models can attach real citations to wrong statements.

Grounded answer prompt

Write the generation prompt for a RAG system that answers from retrieved passages, cites them, and refuses when they are silent.

Who should the AI act as?

What exactly should it do?

What does it need to know?

What shape should the answer take?

How should it sound?

Your prompt

Act as an assistant that answers only from the numbered passages provided.

Do not paste private details (full name, address, ID numbers) into any AI tool.

Quiz: Retrieval and citation check

  1. 01A user asks about a program by its form number, "Form 1040-SR." Which search is most likely to find it?

    Choose one.

  2. 02The model cites passage [7], but only passages 1 to 5 were retrieved. What should your code do?

    Choose one.

Go deeper: Business Intelligence for Real Operators: Turning CRM & ERP Data into Decisions

Ask. Answer. Build together.