Hev meets Jev

Building your own SOTA reranker is now highly achievable.

I was given early access to Jev, TypeSafe’s new structured-output model, and my first thought was a reranker. If you don’t live in search, here is the two-minute version.

What a reranker does

Say you have a couple of ways to search a set of documents. Keyword search produces one list of candidates, vector search produces another, and you need to sort those N lists into one final top 10 (top-K) for the person on the other end. Those last two steps are ranking and reranking. Ranking is mostly algorithmic: reciprocal rank fusion merges the lists. Reranking is mostly inference: a model looks at the query and each candidate and scores how well they match. It runs at query time, so it has to be fast, and there is a boutique industry of models that do exactly this and nothing else.

Why Jev

Jev does not generate text. You send it a state and a set of typed questions, and it answers every question in parallel with a calibrated probability. It is designed for structured output at high throughput and low latency, which sounded to me like exactly the shape a reranker needs.

The question type I used is a Noul, Jev’s true-or-false type: a statement plus criteria for true and for false, answered as a probability from 0 to 1. The whole prompt is one question per candidate document:

question: "Document `documents.{id}` is relevant to `query`: it contains information that answers or directly addresses it."
criteria:
  "true": "The document contains information that answers the query or directly addresses what it asks about."
  "false": "The document is only loosely related, on a similar topic, or does not address what the query asks."

The state is the query plus the thirty candidates, keyed D00 through D29:

{"query": "...", "documents": {"D00": {...}, ..., "D29": {...}}}

Thirty Nouls, one per key, and the rank is the sort by probability. That is the batch shape: one call per query. The single shape asks the same question one (query, document) pair per call. The phrasing is generic on purpose. I tuned it on SciFact’s train split only, and a version written for that corpus did no better, which is what you want if the reranker is going to sit inside a search engine that does not know the corpus.

The experiment

Three BEIR datasets with labeled queries: SciFact (300 scientific claims), NFCorpus (323 medical queries), and FiQA (300 financial questions). BM25 pulls a top-30 shortlist per query as the baseline, and every reranker permutes that same list. I scored nDCG@10 against the labels, averaged it over the three datasets, and compared Jev with Voyage, Cohere, and Mixedbread on quality, price, and latency. Differences are paired bootstrap over queries. Language models got their own run, further down.

The results

Amazingly, Jev lands alongside the purpose-built models on all three axes. That is with no tuning, and none of the query-intent data you could capture in production.

Costs are each run’s actual usage at list price, normalized to 1,000 queries and averaged over the three datasets. Voyage’s number comes from its own billed usage.total_tokens, Cohere charges per search, and Mixedbread’s rerank API bills per token — see the correction above.

The honest read: Voyage rerank-3 is a hair better and a hair cheaper, Mixedbread’s v3.1-listwise is better than either at two and a half times Jev’s price, and the tail on Jev’s batch calls (p95 near 1.4 s) is worse than the specialized rerankers’. The single shape fixes the tail at the cost of thirty times the requests. Paired per query, the batch shape is at or above Cohere on all three datasets, above mxbai-rerank-large-v2 on all three, and trades with Voyage: an edge on SciFact, a tie on NFCorpus, a loss on FiQA. Against v3.1-listwise it ties on SciFact and loses on NFCorpus and FiQA.

What Jev returns that none of them do is a calibrated probability per document. On SciFact, documents Jev scored above 0.9 were judged relevant 76% of the time, and documents it scored below 0.1, half a percent of the time. A cross-encoder gives you a logit you have to threshold per corpus. A probability lets you prune an overfetched pool, compare scores across shards and retrieval legs, and gate a downstream step, with one number you did not have to calibrate yourself. One call over the pool is a rerank and a prune in the same round trip.

Do you even need a reranker?

With agentic search, you can legitimately ask whether you need a reranker at all, or whether a small LLM could do the job. It can. I was surprised to see gpt-5.6-luna land at nearly the same quality and cost as the purpose-built models, and Opus 5 tie Voyage outright. The catch is latency, and at the top end, price: seconds per query instead of a fifth of a second, and Opus costs nearly two hundred times what Voyage does. On SciFact, the one dataset all of them ran on, with the same Jev run for comparison:

Opus also refused 12 of the 300 scientific claims outright.

What I checked

• Determinism. Re-issuing 30 of the batch calls moved scores by 0.004 on average, 0.14 at worst, and changed zero top-1 results.
• Position bias. Reversing the candidate order kept per-document scores at Spearman 0.83 and nDCG@10 within 0.005.
• A floor. On SciFact, a random permutation of the same shortlist scores 0.12; BM25’s own order scores 0.667.
• Question shape. A single Choice over the documents (“which one answers this?”) is slightly better when one document is relevant and much worse when many are. One Noul per document is the shape to ship.

Caveats

These are public benchmarks, so training-set contamination is possible for every model in the table. Rerank depth was 30. The hosted rerankers accept much deeper lists per call, while Jev’s 32k-token request budget holds about 50 passages. Latency was measured from a laptop with network included. The two open-weight rows are the exception and are not comparable on speed: they ran on an Apple M-series GPU with no batching and no network, which is why they read in seconds rather than milliseconds. That is a statement about my hardware, not about the models — served properly on a GPU with batching they would be far faster, and anyone choosing to self-host should benchmark their own deployment rather than read those two latency cells as a verdict. And Jev is hosted only, so the documents leave your environment.

Conclusion

If you are on a team that tests ranking models already, you owe it to yourself to at least look at this. I am opening up my extremely simple ranking approach at github.com/hev/reranker: the prompt, the request schema, a 90-line wrapper with chunking and a prune threshold, and the results with confidence intervals. It is on PyPI as hev-rerank. I am excited to see what others build that is similar.

pip install hev-rerank

from hev_rerank import rerank

hits = rerank(query, shortlist, top_n=10, threshold=0.1)