Recommenders, Ranking & Search

01The funnel

Look at the diagram first. Everything else on this page is a detail hanging off one of these three boxes.

Three stages, each with a different job:

• Retrieval (also called candidate generation). It reduces a billion items to a thousand, and it must be sublinear in corpus size, so it cannot look at user and item together.
• Ranking. It scores those thousand precisely, and it can afford rich features because a thousand is a small number.
• Reranking. It fixes what pointwise scoring cannot see: three near-identical items in the top five, nothing published this week, a policy violation.

The two-stage version of this was set out in the 2016 YouTube paper, and the shape has barely changed since. What changed is what goes inside each box.

02Collaborative filtering: the idea everything else refines

Before any neural network, one observation: you do not need to know what an item is to recommend it.

Collaborative filtering uses only the interaction matrix — who touched what. No genres, no descriptions, no features. Users who agreed in the past will agree again.

Write it as a matrix $R$ with users as rows and items as columns. It is enormously sparse; a typical user has touched a vanishing fraction of the catalogue. Matrix factorization fills the gaps by assuming the matrix is secretly low-rank:

Every user and every item becomes a $k$-dimensional vector. A prediction is one dot product.

The idea is easier to believe once you see it work on numbers.

import numpy as np

# 6 users x 6 items. 0 = never interacted, so the model never sees it.
# Users 0-2 like items 0-2. Users 3-5 like items 3-5. Nobody says so.
R = np.array([[5., 0., 4., 1., 1., 0.],
              [4., 5., 5., 1., 1., 1.],
              [5., 4., 5., 1., 0., 1.],
              [1., 1., 1., 5., 0., 5.],
              [1., 1., 1., 4., 5., 5.],
              [0., 1., 1., 5., 5., 4.]])
mask = R > 0                       # learn from observed entries only

k = 2                              # two latent factors -- the bottleneck
rng = np.random.default_rng(0)
U = rng.normal(0, .1, (6, k))      # user vectors  (6, 2)
V = rng.normal(0, .1, (6, k))      # item vectors  (6, 2)

for _ in range(6000):              # plain SGD, with weight decay
    err = mask * (R - U @ V.T)
    U += .02 * (err @ V - .05 * U)
    V += .02 * (err.T @ U - .05 * V)

P = U @ V.T
print("shapes:", U.shape, "@", V.T.shape, "->", P.shape, "\n")
for (u, i) in [(0, 1), (3, 4), (2, 4), (5, 0)]:
    print(f"user {u}, item {i}: unseen -> predicted {P[u, i]:.1f}")

03Two-tower retrieval, and the constraint that shapes it

The obvious architecture is banned. Understanding why is the single most useful thing on this page.

You want a model that reads a user and an item together and outputs a score. Cross-attention between them, rich interaction features, the works. That model would be far more accurate than a dot product.

You cannot use it for retrieval. At a billion items you would run a billion forward passes per request.

So retrieval accepts a hard constraint: the user and the item must never meet until the final dot product. Two separate encoders, one score:

Two towers, no interaction until the end. That separation is worth a great deal.

Because $g_{\phi}(i)$ does not depend on the user, you can compute every item vector offline, once, and load them into an index. At request time you embed the user, then do approximate nearest neighbour search. The billion never gets touched.

import torch, torch.nn as nn, torch.nn.functional as F
torch.manual_seed(0)

D = 32                                 # embedding width
user_tower = nn.Sequential(
    nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, D))
item_tower = nn.Sequential(
    nn.Linear(48, 128), nn.ReLU(), nn.Linear(128, D))

B = 8
user_feats = torch.randn(B, 64)        # (B, 64)
item_feats = torch.randn(B, 48)        # (B, 48) item i is user i's +ve

u = F.normalize(user_tower(user_feats), dim=-1)   # (B, D)
v = F.normalize(item_tower(item_feats), dim=-1)   # (B, D)

logits = u @ v.T / 0.05                # (B, B) every user vs every item
labels = torch.arange(B)               # the diagonal is the positive
loss = F.cross_entropy(logits, labels)

print("u", tuple(u.shape), "v", tuple(v.shape))
print("logits", tuple(logits.shape), "loss", round(loss.item(), 3))

04Ranking: where the expensive model earns its keep

A thousand candidates is small enough to score extravagantly, and this stage decides what you see.

Retrieval could not let user and item interact. Ranking can, because it only handles a thousand of them. So the ranker gets everything retrieval was denied: cross features, long user histories, the item's recent click-through rate, the time of day.

The recurring difficulty is that useful signals are usually combinations. "This user likes cooking videos" is weak. "This user likes cooking videos, on a phone, in the evening" is strong. A plain MLP can in principle learn that. In practice, on sparse categorical features, it does so slowly and badly.

Two families of architecture attack this directly:

• Wide & Deep (Google, 2016) runs two paths side by side. A wide linear model over hand-crafted cross features memorises specific combinations that are known to matter. A deep network generalises to combinations never seen. They are summed and trained jointly.
• DeepFM (2017) removes the hand-crafting: a factorization machine component learns all pairwise feature interactions automatically, sharing embeddings with a deep component, so no feature engineer is required.

Sequence matters, and modelling it changes the answer

A user's history is ordered, and the order carries information. Somebody who watched three episodes of one series in a row wants the fourth. A bag of their all-time favourites does not capture that.

SASRec (2018) applied a causal self-attention stack to the interaction sequence, predicting the next item from the ones before it. This is a language model over items in place of tokens, with the same architecture from page 08, with a different vocabulary.

05Learning to rank: three ways to phrase the loss

You are producing an ordered list, but the obvious loss functions score one item at a time. That mismatch has three standard answers.

Pointwise treats ranking as regression or classification: predict a click probability, sort by it. It is simple and it is what most production rankers do, because sorting by a well-calibrated probability is a strong baseline.

Pairwise asks a narrower and better-posed question. Given a clicked item and a skipped one, is the clicked one scored higher? BPR is the canonical version for implicit feedback.

import torch, torch.nn.functional as F

# For one user: an item they clicked, and one they saw but skipped.
s_pos = torch.tensor([2.10, 1.80, 1.50])   # scores for clicked items
s_neg = torch.tensor([1.30, 1.10, 1.90])   # scores for skipped items

# BPR maximises P(positive above negative) = sigmoid(s_pos - s_neg).
margin = s_pos - s_neg
per_pair = -F.logsigmoid(margin)

print("margins:  ", [round(m, 2) for m in margin.tolist()])
print("per-pair: ", [round(x, 3) for x in per_pair.tolist()])
print("total:    ", round(per_pair.mean().item(), 4))

Listwise goes after the metric itself. The trouble is that ranking metrics are step functions of the ordering — swap two items and NDCG jumps discontinuously — so they have no usable gradient. LambdaRank sidesteps this by defining the gradient directly, weighting each pair by how much swapping it would move NDCG.

The metrics, and what each one refuses to tell you

• Recall@k. Of the items the user wanted, how many made the top $k$? The right metric for the retrieval stage, where you only care about not losing anything.
• MRR. One over the rank of the first relevant hit. Right when there is one correct answer, such as a navigational search.
• NDCG@k. Discounts gains logarithmically by position, then normalises by the best possible ordering. The standard ranking metric because it is the one that knows position 1 beats position 10.

06The data lies, and it lies in a specific direction

If you take one thing from this page beyond the funnel, take this. It is the thing that separates people who have run a recommender from people who have read about one.

A click is not a statement that the user wanted an item. It is a statement that the user wanted it out of what they were shown. And what they were shown was chosen by the previous version of your model.

Three distinct problems fall out of that, and they compound:

• Position bias. Items at the top get clicked because they are at the top. Train naively and you learn that whatever you ranked first is good, which is circular.
• Exposure bias. An item never shown has no clicks, so it looks bad, so it stays unshown, and the set of items you can learn about shrinks over time.
• Popularity feedback. Popular items get recommended, which makes them more popular, which makes the model more certain. The catalogue quietly collapses toward a small head.

The practical mitigations are unglamorous and they work:

• Model position explicitly. Feed the displayed position in as a feature during training, then fix it to a constant at serving time. The model is forced to attribute some of the click to the slot and not only to the item.
• Inverse propensity weighting. Weight each logged example by one over its probability of being shown, so rarely-shown items count for more.
• Deliberate exploration. Spend a small slice of traffic on items the model is uncertain about, specifically to generate data you would not otherwise get.

That last one is a genuine cost. You are knowingly showing something worse to some users now, to have a better system later. Section 07 is about how much to spend.

07Cold start and exploration

Collaborative filtering needs interaction history. New users and new items have none, which is a problem every real system faces daily.

Three cases, and only two have good answers:

• New item. This one is tractable. Use content features — text, images, category — so the item tower can produce a vector on day zero without any interactions.
• New user. Fall back to popularity and context (locale, device, referrer), then adapt fast. The first session matters disproportionately.
• New system. There is no data at all, so the system is content-based or rule-based until enough interactions accumulate, which is why the first version of a recommender is usually not a recommender.

Exploration is the systematic version of the same problem. You can exploit what you know, or explore to learn more. Epsilon-greedy shows a random item some fraction of the time. Thompson sampling is better behaved: sample from the posterior over each item's value and show the argmax, so uncertainty drives exploration and not a coin flip. Page 15 covers the bandit machinery properly.

08Making retrieval fast: ANN indexes

The two-tower model gives you a vector. Finding its neighbours among a billion others is a separate engineering problem with its own literature.

Exact nearest-neighbour search is linear in corpus size, which defeats the point. Approximate nearest neighbour search trades a small amount of recall for orders of magnitude of speed.

Two families dominate:

• Graph-based (HNSW). It builds a navigable small-world graph over the vectors and greedily walks it toward the query, which gives an excellent recall-latency tradeoff at a high memory cost.
• Partition and quantise (IVF-PQ, as in FAISS). It clusters vectors, searches only the nearest clusters, and compresses each vector into a compact code, which gives a far smaller memory footprint for some accuracy lost to quantisation.

09Build this

The feedback loop in section 06 is the one thing here you should not take on trust. You can watch it happen in an afternoon, with no data and no GPU.

10What breaks

Recommender failures are rarely a bad model. They are a metric that stopped meaning what you thought.

• Offline metrics improve, the A/B test does not. The single most common outcome. Your offline set was generated by the old ranker, so it rewards agreeing with it.
• The filter bubble closes. Coverage drops quietly while engagement looks stable, until the catalogue is effectively a few hundred items.
• Popularity masquerades as personalisation. A model that just predicts global popularity can score respectably. Always compare against that baseline explicitly — it is embarrassing how often it wins.
• Training and serving features drift apart. The ranker was trained on a feature computed one way in the warehouse and served one computed another way. Page 24 calls this training-serving skew; it is endemic here because there are so many features.
• In-batch negatives punished popular items. No logQ correction, so the model systematically suppresses exactly the items most people want.
• The index degraded. Recall@k of the ANN index drifts as the corpus grows and nobody re-tunes it. Measured end to end, this looks like the model getting worse even though the model has not changed.
• Delayed feedback. A click arrives in seconds, a purchase or a cancellation in days. Train on the fast signal alone and you optimise for clickbait.

11Where you meet this in the wild

Same funnel, very different weightings.

12Interview questions

13Go deeper

14Related pages

Other techniques for this problem

A scoped slice of the full Technique Map — every technique this page covers, grouped by what it solves.