When the retriever is a search API

Most explanations of RAG assume the retriever is a vector index you built: you ingested documents, chunked them, embedded them, and search means nearest-neighbour lookup. That’s one implementation of the R, not the definition of it.

The R stage is anything that turns a question into passages. A live query to a search engine qualifies. The generate stage downstream barely notices the difference — but the plumbing in between changes quite a lot, and so do the failure modes.

The retrieve stage becomes an HTTP request

In the index architecture, retrieval is: embed the query, search the vector store, get back k chunks with similarity scores. You own every piece of that. You decided what was in the corpus, how it was split, and what “relevant” means.

In the search-backend architecture, retrieval is: send a query string to a search API, get back a ranked list of results. You own the query and nothing else. The corpus is the open web (or whatever slice of it the operator you attached to the query allows), the ranking is somebody else’s, and the unit of retrieval is whatever the search engine decided to show.

Everything after that — assembling a prompt, instructing the model to stick to sources, citing what it used — is unchanged. This is why the swap is possible at all: the interface between retrieval and generation is just “here are some passages and where they came from,” and both retrievers can produce that.

What comes back is not a chunk

The important difference is the payload. A chunk from your index is a contiguous span of a document you control, at a length you chose. A search result is a small, standardised record about a page.

Serply’s documentation for its Google search endpoint is a good reference for the shape, because it spells the fields out: each organic result carries a title, a link, a description, a cite field holding the domain and breadcrumb, and an additional_links array of sub-links, while the response as a whole carries a total estimated result count and a related-questions block. That is representative of what this class of API returns — the documented response schema for a hosted Google search endpoint is the concrete version of the abstract “results come back as ranked records” claim.

Read that list again with the generate stage in mind. The only piece of actual source text in it is description — a snippet of a couple of dozen words, chosen by the search engine to justify the ranking, often with an ellipsis in the middle and sometimes with the query terms bolded into the markup. It was written to persuade a human to click, not to answer a question in isolation.

The adapter in the middle

So there’s a translation layer, and it is where most of the design work lives.

Snippet-only. Take the descriptions as your passages. Cheapest and fastest — one HTTP call, no page fetches. Works when the answer is a short fact that the snippet contains outright, and fails whenever the answer needs a paragraph of context around it.

Fetch and read. Take the top few links, fetch those pages, extract the readable text, and pass that to the model. Now you have real source text, but you’ve added a fetch per result, a boilerplate-stripping step, and a page that might be paywalled, rendered by JavaScript, or 900 KB of navigation.

Snippet first, fetch on demand. Answer from snippets when they clearly suffice; fetch only when they don’t. Two different latencies for two different question types, which is honest but makes your performance profile lumpy.

Whichever you pick, the adapter must also produce metadata the generate stage can use: the URL, the site, and — if you can get it — a date. This is the same information a chunk carries in its metadata, just assembled from a different source.

What the generate stage inherits

The prompt looks almost identical to the index version, and everything in what happens after retrieval still applies. Three things do change.

The passages are shorter and more numerous. Ten snippets is a lot of separate voices and very little text. Instructions that worked on five substantial chunks can produce a shallow, list-like answer here.

Attribution is easier. Every passage arrives with a real, public URL attached — which is exactly what citations that point somewhere real asks for and often has to work to reconstruct in an index system.

Disagreement is the norm, not the exception. Ten pages from ten organisations will contradict each other routinely. The handling described in when sources disagree stops being an edge case and becomes the main path.

Where the query comes from

An index retriever takes the user’s question more or less as written, because embeddings tolerate paraphrase. A search retriever does not: it wants keywords, and it responds to operators.

So there’s usually a query-writing step — the model turns “what did they say about the delay” into a search query with the product name, the year, and maybe a site: restriction. That step is a new place for the pipeline to go wrong, and it is worth logging separately from the results, because a bad answer here is far more often a bad query than a bad ranking.

Failure modes an index doesn’t have

The corpus changes underneath you. The same query on Tuesday returns different pages than on Monday. Reproducing a complaint means storing the results you actually got, not just the query.

A network dependency on the critical path. Retrieval can now time out, rate-limit, or return an error. An index can too, but it’s yours to fix.

Nothing was curated. Every page in the result set is one you never vetted, which is a trust problem serious enough to deserve its own treatment — and it is the same door that prompt injection in retrieved documents walks through.

None of these makes the architecture wrong. They make it a different architecture, with its own operational surface, wearing the same three letters.