Collaborative Filtering (Neighborhood) · Recommender Systems

01 · Let behaviour speak

The collaborative idea

No attributes. No descriptions. Just the matrix of who rated what.

Content-based filtering needed someone to build the ICM. Collaborative filtering needs none of that — its premise is purely behavioural: users who agreed in the past tend to agree in the future. If Alice and Bob rated the same films almost identically, a film Bob loved but Alice hasn’t seen is a strong recommendation for Alice — regardless of what that film is about. The single input is the URM. There are two ways to read it: user-based (compare rows) and item-based (compare columns).

Content-Based (Ch. 4)

Similarity from the ICM. Handles new items. Needs attributes. Low diversity (filter bubble).

Collaborative (Ch. 5)

Similarity from the URM. No attributes needed. Finds taste patterns. Blind to brand-new items (cold start).

!

The cold-start flip side

Because CF only knows an item through its ratings, a brand-new item with zero ratings is invisible — the exact case content-based handled effortlessly. This is why real systems hybridise the two (Ch. 9).

02 · Compare the rows

User-based collaborative filtering

Find your nearest neighbours among users; borrow their ratings.

Two users are similar if their rating rows point the same way. For implicit data (1/0) the dot product counts items both interacted with; the cosine similarity normalises it. For explicit ratings we use the same cosine on the values, then add the familiar shrinkage term so a pair with one or two films in common isn’t trusted like a pair with twenty.

Hands-on

User neighbours · who agrees with whom

Similarity between the selected user's rating row and every other. The shrink term h pulls low-overlap pairs toward zero; switch to implicit to compare 0/1 interaction patterns instead.

Shrink h0.0

Bob0.89

Carla0.83

Finn0.21

Eva0.17

Dan0.06

TakeawaySame cosine as content-based filtering, now on URM rows. For explicit data, subtracting each user's mean first (Pearson) measures agreement about above/below-average — not who presses higher numbers.

Once we have user $u$ ‘s K nearest neighbours, the predicted rating for an unseen item is a similarity-weighted average of the neighbours’ ratings for that item.

!

Different users use the scale differently

One user’s “3” is another’s “5”. Plain cosine ignores this. The pearson correlation fixes it by subtracting each user’s average rating $b_u$ before comparing — so it measures whether two users agree about what is above/below their own average, not whether they press the same buttons.

Formal Definition 5.1 — user similarity & prediction

Cosine similarity between users $u,v$ (with shrink $h$ ), and the bias-removing Pearson correlation:

User similarity

s_{uv} = \frac{\sum_i r_{ui}\,r_{vi}}{\lVert r_u\rVert\,\lVert r_v\rVert + h}, \qquad s_{uv}^{\text{Pearson}} = \frac{\sum_i (r_{ui}-b_u)(r_{vi}-b_v)}{\sqrt{\sum_i (r_{ui}-b_u)^2}\,\sqrt{\sum_i (r_{vi}-b_v)^2} + h}.

Prediction over the K most similar users: $\hat r_{ui} = \dfrac{\sum_{v\in\text{KNN}(u)} s_{uv}\,r_{vi}}{\sum_{v\in\text{KNN}(u)} s_{uv}}$ .

03 · Compare the columns

Item-based collaborative filtering

Two items are similar if the same people rated them the same way.

Turn the matrix on its side. Two items are similar when the same users rated them similarly — a similarity read straight from the URM columns. The slide’s one-liner captures it: item-based CF is “content-based filtering, but without knowing the attributes.” It produces an item–item similarity matrix just like Chapter 4 — except learned from behaviour, not tags.

Hands-on

Item similarity from behaviour · slide K to sparsify

Item-item cosine from the URM columns. Darker = more similar. This is collaborative filtering discovering structure attributes can't see — similarity from co-ratings, not tags.

Neighbours K7non-zero: 56 / 56

i1i2i3i4i5i6i7i8TopGun·0.960.740.860.130.140.960.20MI0.96·0.810.700.140.161.000.20Inter0.740.81·0.440.270.310.810.26Martian0.860.700.44·0.240.200.700.32Notting0.130.140.270.24·0.960.140.99LoveAct0.140.160.310.200.96·0.160.94Aveng0.961.000.810.700.140.16·0.20LaLa0.200.200.260.320.990.940.20·

TakeawayThere are usually far fewer items than users, and item columns are more stable than user rows — so this matrix can be precomputed once and reused, which makes item-based CF a *model*.

key

Why item-based is usually preferred

There are typically far fewer items than users, and an item’s rating column is more stable over time than a user’s row (tastes drift; a movie’s audience pattern doesn’t). So the item–item matrix $S$ is smaller, cheaper, and can be pre-computed once and reused — which makes it a model (§5.5).

Formal Definition 5.2 — item similarity

Cosine over item columns, and adjusted cosine which removes the user bias $b_u$ (subtracted per-user, since the same users span both columns):

Item similarity

s_{ij} = \frac{\sum_u r_{ui}\,r_{uj}}{\lVert r_{\cdot i}\rVert\,\lVert r_{\cdot j}\rVert + h}, \qquad s_{ij}^{\text{adj}} = \frac{\sum_u (r_{ui}-b_u)(r_{uj}-b_u)}{\sqrt{\sum_u (r_{ui}-b_u)^2}\,\sqrt{\sum_u (r_{uj}-b_u)^2} + h}.

The item–item matrix is symmetric with a zero diagonal, then sparsified with KNN — exactly as in Ch. 4.

04 · From similarity to a ranking

Making recommendations

One weighted sum over the K nearest neighbours — and one matrix product.

Both flavours score an unseen item the same way: a similarity-weighted sum over the K nearest neighbours, normalised by the total similarity. In matrix notation the entire prediction is a single product — $\hat R = S\,R$ for user-based (multiply on the left by user similarity) and $\hat R = R\,S$ for item-based (multiply on the right by item similarity).

Hands-on

CF recommender · user-based vs item-based

Only unseen items are scored, by a similarity-weighted average over the K nearest neighbours. Switch method and user — the rankings stay inside each taste cluster.

Neighbours K3

1Martian4.62because you liked MI

2Notting1.66because you liked Inter

3LaLa1.00because you liked LoveAct

Takeawayr̂ = S·R (user-based) and r̂ = R·S (item-based) are the whole model. Item-based wins in practice: S is smaller, more stable, user-independent — precompute once, serve new users without retraining.

Formal Definition 5.3 — prediction & matrix form

Item-based prediction, and the bias-aware user-based form (predict the delta from the user’s mean, then add it back):

Prediction

\hat r_{ui} = \frac{\sum_{j\in\text{KNN}(i)} s_{ij}\,r_{uj}}{\sum_{j\in\text{KNN}(i)} s_{ij}}, \qquad \hat r_{ui} = b_u + \frac{\sum_{v\in\text{KNN}(u)} s_{uv}\,(r_{vi}-b_v)}{\sum_{v\in\text{KNN}(u)} s_{uv}}.

Matrix form: $\hat R = S\,R$ (user-based) and $\hat R = R\,S$ (item-based).

05 · A unifying view

Memory-based vs model-based

The same two-step skeleton that organises every recommender in the course.

Every recommender is two steps: build a model from the data, then score with that model and a user profile — $\text{model}=f(\text{URM})$ or $f(\text{ICM})$ , then $\hat r = g(\text{model}, \text{profile})$ . The distinction is whether the user’s profile is baked into the model:

Memory-based — the user profile is part of the URM the model was built from. User-based CF is the archetype: to score Alice you re-scan the stored rows. You can only recommend to users already in the model.
Model-based — the profile is not needed to build the model. Item-based CF learns $S_{II}=f(\text{URM})$ once; then $\hat r = g(S_{II}, \text{profile})$ works for any profile — even a brand-new user who wasn’t in the training URM.

key

The killer feature of item-based CF

Because the item–item similarity $S_{II}$ doesn’t depend on any particular user, you can serve a user who just signed up the moment they rate a couple of items — no retraining. User-based CF cannot: the new user wasn’t one of the rows. Everything from SLIM (Ch. 6) to Matrix Factorization (Ch. 8) is a smarter way to learn that model.

06 · Exam intel

What the exam tests

Distinguish user-based vs item-based CF; compute a user–user (or item–item) cosine from the URM; explain why Pearson / adjusted cosine remove bias; and classify a method as memory- vs model-based.

Q

Worked question — cosine between two users (implicit and explicit)

From the URM, compute the cosine similarity between Alice = [5,4,5,4,0,0,4,0] and Bob = [4,5,4,0,0,1,5,0].

Implicit — binarise: Alice = [1,1,1,1,0,0,1,0], Bob = [1,1,1,0,0,1,1,0]. Co-rated dot = 4 (TopGun, MI, Inter, Avengers); each has 5 ones. $s = 4/(\sqrt5\,\sqrt5) = 4/5 = 0.80$ .
Explicit — dot $= 5{\cdot}4 + 4{\cdot}5 + 5{\cdot}4 + 4{\cdot}5 = 80$ ; $\lVert\text{Alice}\rVert=\sqrt{98}$ , $\lVert\text{Bob}\rVert=\sqrt{83}$ ; $s = 80/(\sqrt{98}\,\sqrt{83}) \approx 0.887$ .

Both confirm Alice and Bob sit in the same action/sci-fi cluster. Traps: saying CF “needs item attributes” (that’s content-based); calling user-based CF model-based (it is memory-based); forgetting the zero diagonal of $S$ .

07 · Exam · past papers

Past-paper questions

Past paper Exam 2020 · 7 pts — user-based CF end to end

Q. Idea + predictions with/without user bias; cosine; Pearson (when/why); implicit Top-N simplification; the shrink term; KNN.

Model answer. Predict $r_{ui}$ from users similar to $u$ . Without bias: $\hat r_{ui}=\frac{\sum_{v} s_{uv}\,r_{vi}}{\sum_{v} s_{uv}}$ ; with bias: $\hat r_{ui}=b_u+\frac{\sum_{v} s_{uv}(r_{vi}-b_v)}{\sum_{v} s_{uv}}$ ; matrix form $\hat R = S\,R$ . Cosine $s_{uv}=\frac{\sum_i r_{ui} r_{vi}}{\lVert r_u\rVert\lVert r_v\rVert + h}$ . Pearson subtracts each mean (use on explicit data when users use the scale differently). Implicit + Top-N: ranking only needs the order, so drop the denominator and bias → $\hat r_{ui}=\sum_v s_{uv} r_{vi}$ ( $\hat R = S\,R$ ). Shrink $h$ down-weights similarities from few co-ratings. KNN keeps each user’s top-K neighbours (inverted-U quality curve).

Past paper RS Exam · 8 pts — similarity measures compared

Q. Basic idea; different approaches to measure user similarity (advantages, disadvantages, peculiarities); how to estimate ratings; user-based in matrix notation.

Model answer. Cosine — angle between rating rows; simple, but conflates harsh/generous raters and rewards overlap. Pearson — cosine after subtracting each user’s mean $b_u$ ; corrects rating-scale bias, best for explicit data, but unstable with very few co-ratings. Jaccard / dot-product — for implicit data, count shared interactions; cheap, natural for 0/1, ignores magnitude. All add a shrink term $h$ . Estimate: $\hat r_{ui}=b_u+\frac{\sum_v s_{uv}(r_{vi}-b_v)}{\sum_v s_{uv}}$ . Matrix: $\hat R = S\,R$ — each predicted row is a similarity-weighted combination of the other rows, so $S$ multiplies $R$ on the left.

Past paper Practice Exam 1 · user-based vs item-based

Q. Difference between user-based and item-based CF; how cosine is calculated and its role; the similarity matrix in item-based prediction.

Model answer. User-based compares URM rows and recommends what similar users liked ( $\hat R = S\,R$ ); item-based compares URM columns ( $\hat R = R\,S$ ). Item-based is usually preferred — fewer, more stable items, and $S_{II}$ is user-independent so it can be precomputed (a model). Cosine $s=\frac{\langle x,y\rangle}{\lVert x\rVert\lVert y\rVert + h}$ provides the neighbour weights. Item-based: build item-item $S$ (cosine over columns, zero diagonal, KNN), then $\hat r_{ui}=\frac{\sum_{j\in\text{KNN}(i)} s_{ij}\,r_{uj}}{\sum_{j\in\text{KNN}(i)} s_{ij}}$ , i.e. $\hat R = R\,S$ .

08 · Self-check

Three questions before you move on

Collaborative filtering computes similarity from:

Item-based CF is called "content-based filtering, but without knowing the attributes." Why?

Item-based CF precomputes S_II = f(URM) once and can then score a brand-new user not in the training data. This makes it:

09 · Recap

One-screen summary

Chapter 05 — load-bearing ideas

CF = similarity from the URM: rows → user-based, columns → item-based. Same cosine + shrink + KNN machinery as content-based, pointed at behaviour.
Remove bias for explicit ratings: Pearson (user) and adjusted cosine (item) subtract the user mean so “agreement” means above/below-average, not pressing the same number.
Item-based is model-based: $S_{II}=f(\text{URM})$ is precomputed and user-independent, so it serves new users — user-based CF (memory-based) cannot.
Matrix form: $\hat R = S\,R$ (user) and $\hat R = R\,S$ (item). CF is blind to zero-rating items (cold start).