Rerankers

A reranker reorders a candidate list using the query and each passage together, instead of comparing the query to a vector that was computed on its own. That’s why it can tell whether a passage answers this question rather than just sitting nearby in embedding space — and why it costs more per candidate than the retriever did.

Set it with reranker in Config or grid.reranker in a config file. Every reranker is a spec string resolved through the registry:

from contextgrid.rerank import RERANKERS, get_reranker

RERANKERS.names()

['lexical', 'litellm-rerank', 'mmr', 'none', 'tei-rerank']

Note

Only RERANKERS, Reranker and get_reranker are exported from top-level contextgrid. The five reranker classes themselves — LexicalOverlapReranker, MMRReranker, NoReranker, TEIReranker, LiteLLMReranker — are not on contextgrid (there is no contextgrid.LexicalOverlapReranker). Import them from contextgrid.rerank instead:

from contextgrid.rerank import LexicalOverlapReranker, MMRReranker

candidates: the parameter every blog post skips

Almost all “use a reranker” advice stops at naming a model. It rarely says how many candidates to hand it, and that second number is where most of the effect actually lives:

• Over the top 10, a reranker can only reorder what the retriever already found.
• Over the top 100, it can rescue a passage the retriever ranked 47th.

Cost scales with depth; benefit doesn’t scale the same way, and the right point on that curve is specific to your corpus. That’s why candidates is its own axis on the grid:

grid:
  reranker: [null, lexical, "tei-rerank:bge-reranker-base"]
  candidates: [10, 50, 100]

With a reranker set, contextgrid.pipeline.BuiltPipeline.search asks the retriever for candidates results and the reranker cuts that back to k. With no reranker, the retriever is asked for k directly, so the no-reranker arm never pays for depth it would only throw away.

candidates is meaningless without a reranker, so contextgrid.grid.matrix.canonicalise resets it back to its default (50) whenever reranker is None — a sweep doesn’t waste runs on configurations that differ only in an unused number:

from contextgrid.grid.matrix import canonicalise
from contextgrid.pipeline import Config

canonicalise(Config(reranker=None, candidates=100)).candidates

50

canonicalise(Config(reranker="lexical", candidates=100)).candidates

100

Which means a sweep of candidates: [5, 20, 50] with no reranker anywhere on the grid comes back as one row, not three — the three depths are one configuration. The sweep says so rather than leaving you to work it out from a leaderboard shorter than the shape you were quoted:

CAUTION [rerank]: the candidates axis was swept over 5, 20, 50 with no reranker on the matrix,
so every depth ran the identical search and they were folded into one row. ...
Nothing here measures `candidates`.

Pair the two axes and both become real:

grid:
  reranker: [null, lexical, mmr]
  candidates: [10, 50, 100]

The five rerankers

name	spec	extra needed	when to use it
none	none	—	the arm every reranker has to beat
lexical	lexical or lexical:0.4	—	free floor: query-term coverage
mmr	mmr or mmr:0.6	—	fixes a top-k that’s near-duplicate passages
tei-rerank	tei-rerank:bge-reranker-base	none (plain urllib)	a real cross-encoder, self-hosted
litellm-rerank	litellm-rerank:cohere/rerank-english-v3.0	pip install "context-grid[llm]"	a hosted cross-encoder: Cohere, Jina, Voyage, AWS

none — NoReranker

from contextgrid.rerank import NoReranker

Keeps the retriever’s order — no fields, nothing to configure. Not a placeholder: an honest comparison needs this baseline sitting on the same leaderboard, with the same cost and latency columns, as every reranker you’re actually testing.

lexical — LexicalOverlapReranker

Scores a passage by how much of the query it actually contains, divided by the passage’s length raised to length_penalty — a cross-encoder without the encoder. Weak, free, and a real floor: a neural reranker that costs money and beats this by 0.01 has told you something about whether it’s worth deploying at all.

from contextgrid import Chunk, Span
from contextgrid.rerank import get_reranker

texts = [
    "Shipping takes five to seven business days.",
    "Refunds are issued within thirty days of purchase.",
    "The office is closed on public holidays.",
    "Digital goods are not refundable once downloaded.",
]
chunks = [
    Chunk(id=f"doc:{i}", span=Span("doc", i * 100, i * 100 + len(t)), text=t)
    for i, t in enumerate(texts)
]
top = get_reranker("lexical").rerank("do I get a refund on digital goods?", chunks, k=2)
[scored.chunk_id for scored in top]

['doc:3', 'doc:2']

length_penaltyfloatdefault 0.25

How much longer passages are punished for their length. lexical:0.4 sets length_penalty=0.4.

mmr — MMRReranker

Maximal marginal relevance: relevance minus similarity to what’s already been picked. It’s the fix for a top-k that’s five near-copies of the same paragraph — overlapping chunks make a leaderboard look fine (the evidence really was retrieved, five times), while a generator reading that context sees one fact repeated instead of five distinct ones.

diversity isn’t validated anywhere — there’s no bounds check. diversity=0 keeps the retriever’s order; diversity=1 ignores relevance entirely and greedily picks whatever remaining passage is most different. The relevance signal MMR reorders against is synthesized from the incoming rank (1 - position / len(candidates)), not from any real score — MMR here always reorders, it never rescales an actual number.

mmr_texts = [
    "Refunds are issued within thirty days of purchase.",
    "You can get a refund within thirty days of buying the product.",
    "Purchases are refundable for up to thirty days.",
    "Shipping takes five to seven business days.",
    "The office is closed on public holidays.",
]
mmr_chunks = [
    Chunk(id=f"doc:{i}", span=Span("doc", i * 100, i * 100 + len(t)), text=t)
    for i, t in enumerate(mmr_texts)
]
none_order = [s.chunk_id for s in get_reranker(None).rerank("refund policy", mmr_chunks, k=3)]
mmr_order = [s.chunk_id for s in get_reranker("mmr:0.6").rerank("refund policy", mmr_chunks, k=3)]
none_order, mmr_order

(['doc:0', 'doc:1', 'doc:2'], ['doc:0', 'doc:1', 'doc:3'])

Three candidates in a row say almost the same thing about refunds. none keeps all three; mmr:0.6 swaps the third for the shipping passage instead, once two refund answers are already in the top 3.

diversityfloatdefault 0.3

0 keeps the retriever’s order. 1 always picks the most different remaining passage, regardless of relevance. mmr:0.6 sets diversity=0.6.

tei-rerank — TEIReranker

from contextgrid.rerank.remote import TEIReranker

A cross-encoder served by text-embeddings-inference (TEI). No API key, no extra Python dependency — reached over plain urllib.

docker run -p 8081:80 ghcr.io/huggingface/text-embeddings-inference:cpu-latest \
    --model-id BAAI/bge-reranker-base

grid:
  reranker: ["tei-rerank:bge-reranker-base,api_base=http://localhost:8081"]

Caution

The default port is 8081, distinct from the embedder’s TEI default of 8080. One TEI process serves one model, so embedding and reranking need two separate servers on two separate ports. Assuming one server does both fails with a confusing HTTP error, not one that names the real problem.

modelstrdefault ""

Required, e.g. bge-reranker-base.

api_basestr | Nonedefault http://localhost:8081

The TEI server’s /rerank endpoint. Confirmed constructor default — matches the note above about port 8081 being the value used in every working example.

batch_sizeintdefault 64

Candidates sent per request.

max_charsint | Nonedefault 8000

Passage text is trimmed to this many characters before sending — a character-count guard, not a tokenizer-aware one.

timeoutfloatdefault 60.0

Seconds.

retriesintdefault 2

Retried only for transient errors — timeouts, rate limits, 5xx.

transportCallable | Nonedefault None

Replaces the network call entirely. Used below to run every example with no server and no key.

You can drive TEIReranker with no server at all by supplying transport, a function that takes (query, passages) and returns [(position, score), ...]:

def by_keyword(word):
    def transport(query, passages):
        return [(i, 1.0 if word in p.lower() else 0.0) for i, p in enumerate(passages)]
    return transport

# reuses `chunks` from the lexical example above:
# doc:0 shipping, doc:1 refunds, doc:2 office hours, doc:3 digital goods refund
tei = TEIReranker(model="bge-reranker-base", transport=by_keyword("refund"))
top = tei.rerank("refund", chunks, k=2)
[(scored.chunk_id, scored.score) for scored in top]

[('doc:1', 1.0), ('doc:3', 1.0)]

litellm-rerank — LiteLLMReranker

from contextgrid.rerank import LiteLLMReranker

A hosted reranker through litellm: Cohere, Jina, Voyage, AWS, one name each. Needs pip install "context-grid[llm]".

grid:
  reranker: ["litellm-rerank:cohere/rerank-english-v3.0"]

from contextgrid.rerank import get_reranker

get_reranker("litellm-rerank:cohere/rerank-english-v3.0")

LiteLLMReranker(model='cohere/rerank-english-v3.0', api_base=None, api_key_env=None, timeout=60.0, retries=2, batch_size=64, max_chars=8000, transport=None)

The key comes from the environment (COHERE_API_KEY, JINA_API_KEY, VOYAGE_API_KEY, …), never from the config file. Set api_key_env to the name of the variable, not the key itself.

modelstrdefault ""

Required, provider/model, e.g. cohere/rerank-english-v3.0.

api_basestr | Nonedefault None

Override the provider’s default endpoint.

api_key_envstr | Nonedefault None

Name of the environment variable holding the key.

batch_sizeintdefault 64

Same meaning as tei-rerank.

max_charsint | Nonedefault 8000

Same meaning as tei-rerank.

timeoutfloatdefault 60.0

Same meaning as tei-rerank.

retriesintdefault 2

Same meaning as tei-rerank.

LiteLLMReranker(model="cohere/rerank-english-v3.0").rerank("refund policy", chunks, k=2)

Every candidate must come back, or the run fails

Both remote rerankers insist on one thing: the backend has to return a score for every candidate it was sent, or contextgrid.rerank.RerankerError is raised. A backend that quietly returns fewer results than it was given — a passage too long, a batch silently capped — has dropped documents from the ranking. On a leaderboard that looks exactly like the reranker judging those documents irrelevant, which is a completely different claim:

from contextgrid.rerank import RerankerError

def dropping_transport(query, passages):
    return [(i, 1.0) for i, _ in enumerate(passages)][:-1]  # drops the last one

bad = TEIReranker(model="bge-reranker-base", transport=dropping_transport)
bad.rerank("refund", chunks, k=2)

Traceback (most recent call last):
    ...
contextgrid.rerank.remote.RerankerError: tei-rerank scored 3 of 4 candidates in the batch starting at 0. Every candidate must come back, or the ones that did not look like the model rejected them.

Ties are broken by the incoming rank ((-score, position)), so two passages scored identically keep the retriever’s order rather than an arbitrary one.

See also

• Retrieval Strategies — what builds the candidate list a reranker reorders.
• Query Transforms — what rewrites the question before any of this runs.
• The Ten Axes — where reranker and candidates sit among the other eight.