Content-Based Filtering · Recommender Systems

01 · Similar in, similar out

The content-based idea

Compare items by their attributes — and recommend the neighbours of what you liked.

The principle is disarmingly simple: a user who expressed a preference for an item is likely to like similar items. It needs only one ingredient — a way to measure how similar two items are by their content. That content is the ICM from Chapter 1: rows are items, columns are attributes, $a_{if}=1$ if item $i$ has attribute $f$ . Content-based filtering never looks at other users — it reads only the target user’s own ratings plus the ICM.

why

New items are not a problem

A brand-new movie has no ratings, so collaborative methods (Ch. 5) are blind to it — but it does have attributes the moment it is added. Content-based filtering can recommend it on day one. The flip side: it keeps recommending more of the same — the low-diversity filter bubble from Chapter 3.

02 · Counting shared attributes

Measuring item-item similarity

Start with the dot product; normalise it into a cosine.

Take two item rows of the ICM. Their dot product $\langle a_i, a_j\rangle$ counts the attributes they share — large means very similar. But the dot product is unbounded: an item with many attributes scores high against everything. The fix is the cosine similarity, which divides by the vector lengths to land in $[0,1]$ .

Hands-on

Similarity explorer · two movies, one cosine

The dot product is the count of shared attributes; the cosine divides it by the product of the two norms. Raising the shrink term t demotes pairs with few attributes.

Item iItem jShrink t0.0

Top Gun: Action, Adventure, Cruise

Mission: Impossible: Action, Thriller, Cruise

shared (2): Action, Cruise

⟨i, j⟩

2

‖i‖ · ‖j‖

3.00

cosine

0.67

shrunk (t=0.0)

0.67

TakeawayCosine = shared attributes ÷ (‖i‖‖j‖), bounded in [0,1] unlike the raw dot product. The shrink term in the denominator pulls down pairs we have little evidence about.

Formal Definition 4.1 — cosine & shrink

For item attribute rows $a_i, a_j$ :

Cosine

\langle a_i, a_j\rangle = \sum_f a_{if}\,a_{jf} = \#\{\text{attributes shared by } i \text{ and } j\}, \qquad s_{ij} = \frac{\langle a_i, a_j\rangle}{\lVert a_i\rVert\,\lVert a_j\rVert}, \qquad s_{ij}^{\text{shrunk}} = \frac{\langle a_i, a_j\rangle}{\lVert a_i\rVert\,\lVert a_j\rVert + t}.

03 · From pairs to a matrix

The similarity matrix & K-nearest-neighbours

Compute every pair once, then throw most of it away.

Doing this for every pair gives the similarity matrix $S$ (items × items). With cosine it is symmetric, and its diagonal is removed — an item is trivially identical to itself, which would swamp everything. Two problems remain: $S$ is dense (expensive), and its many small values are mostly noise. The cure is K-nearest-neighbours: in each row, keep only the $K$ largest similarities and zero the rest. The matrix becomes sparse, fast, and cleaner — though no longer symmetric, since $j$ can be in $i$ ‘s top-K without the reverse.

Hands-on

Similarity matrix · slide K to sparsify

The cosine similarity of every movie pair. Slide K to keep only each row's top-K neighbours — too small starves the recommender, too large lets noisy weak similarities back in.

Neighbours K7non-zero: 22 / 56

i1i2i3i4i5i6i7i8TopGun·0.670.410.41MI0.67·0.41Inter·0.500.500.41Martian0.410.50·0.50Notting·1.000.41LoveAct1.00·0.41Aveng0.410.410.500.50·LaLa0.410.410.41·

TakeawayWith cosine, the full matrix is symmetric with a zero diagonal; KNN sparsification breaks symmetry (j can be in i's top-K without the reverse) but removes the noisy long tail of weak ties.

!

K is a hyper-parameter you must tune

Too small ⇒ not enough data to estimate a score (unstable quality). Too big ⇒ weak, noisy similarities leak back in. The relationship between $K$ and quality is an inverted-U, so you find it by evaluating (Chapter 3), not by guessing.

04 · Turning similarity into a ranking

From similarity to recommendations

One matrix multiply: your ratings, weighted by item similarity.

To score item $i$ for user $u$ , take a similarity-weighted sum of the user’s own ratings: items the user liked that are similar to $i$ pull its score up. In matrix form this is the whole model: $\hat R = R\,S$ . For the Top-N task we rank by that score; for rating prediction we normalise by the total similarity so the result stays on the rating scale.

Hands-on

Content-based recommender · scores & explanations

Only unseen items are scored, by a similarity-weighted sum of the user's own ratings. Each row names the rated item that contributed most — the recommendation explains itself.

Neighbours K3

Alice rated: TopGun (5), MI (4), Inter (5), Martian (4), Aveng (4)

1LaLa2.04because you liked Inter (shared: Drama)

2Notting0.00no attribute overlap with your profile

3LoveAct0.00no attribute overlap with your profile

Takeawayr̂ = R·S in one line, and every score is explainable. The cost: recommendations stay inside the user's taste cluster — content-based filtering has low diversity by construction.

Formal Definition 4.2 — scoring

Score

\hat r_{ui} = \sum_{j \,:\, r_{uj}\neq 0} s_{ij}\, r_{uj} \quad\text{(Top-N)}, \qquad\qquad \hat r_{ui} = \frac{\sum_{j} s_{ij}\, r_{uj}}{\sum_{j} s_{ij}} \quad\text{(rating prediction)}.

With $\tilde A$ the row-normalised ICM, $S = \tilde A\,\tilde A^\top$ (zero diagonal, optionally KNN-sparsified) and $\hat R = R\,S$ computes every user’s scores at once.

05 · Not all attributes are equal

Improving the ICM: weights & TF-IDF

A binary ICM treats “Romance” and “released in 2012” as equally meaningful. They’re not.

So far the ICM is binary, so every attribute counts the same. But rare, specific attributes are more informative than common ones. You could assign weights by hand, or let TF-IDF do it automatically: $w_{a,i} = \text{tf}_{a,i}\cdot \text{idf}_a$ .

Term Frequency $\text{tf}_{a,i} = n_{a,i}/(\#\text{attributes of } i)$ — if an item has many attributes, each weighs less.
Inverse Document Frequency $\text{idf}_a = \log\frac{\lvert\text{ITEMS}\rvert}{n_a}$ — an attribute shared by every item has $\text{idf}=\log 1 = 0$ ; a rare one scores high.

On our 8-movie catalog, the most discriminative attributes are the ones only one film has — Thriller and Musical — with $\text{idf}=\log_{10}(8/1)\approx 0.90$ ; the most common (Action, SciFi, Romance, each in three films) score lowest at $\log_{10}(8/3)\approx 0.43$ .

key

Why it helps

Re-weighting the ICM by TF-IDF before computing cosine means two films sharing a rare attribute (both Musicals) are judged far more similar than two sharing a common one (both Action). The similarity matrix becomes much more discriminating — for free, with no manual tuning.

06 · Exam intel

What the exam tests

Compute a cosine (with and without a shrink term) from two attribute vectors; explain KNN and the role of $K$ ; compute an IDF; and state what CB reads (ICM) vs what CF reads (URM).

Q

Worked question — cosine and shrunk similarity

Item $i$ has attributes {Action, Adventure, Cruise}; item $j$ has {Action, Thriller, Cruise}. Compute the cosine, then the shrunk similarity with $t = 3$ .

Dot = 2 (shared: Action, Cruise). Norms $\lVert i\rVert=\lVert j\rVert=\sqrt 3$ .
Cosine $= 2/(\sqrt3\cdot\sqrt3) = 2/3 \approx 0.67$ .
Shrunk $= 2/(3 + 3) = 0.33$ — the shrink term halves a 3-attribute pair’s similarity.

Traps: confusing CB (similarity from the ICM) with CF (similarity from the URM); forgetting to remove the diagonal of $S$ , or assuming a KNN matrix stays symmetric; believing larger $K$ is always better.

07 · Exam · past papers

Past-paper questions

Past paper AT-Sample · 5 pts — a learned content-based model (matrix + loss)

Q. Describe a purely content-based model with machine learning: write it in matrix and summation notation with a loss function; the underlying idea; the constraints and why; and the predicted-ratings equation.

Model answer. Idea. Recommend items whose content is similar to what the user liked, but learn the item-item weight matrix $S$ from data (its support restricted to content-similar pairs, from the ICM $A$ — that is what keeps it content-based). Model. $\hat R = R\,S$ ; $\hat r_{ui}=\sum_j s_{ij}\,r_{uj}$ . Loss. $\min_S\;\tfrac12\lVert R - R\,S\rVert_F^2 + \tfrac{\beta}{2}\lVert S\rVert_F^2 + \lambda\lVert S\rVert_1$ (reconstruct the URM from itself through learned similarities, with L2+L1 regularisation). Constraints. $\operatorname{diag}(S)=0$ — otherwise the trivial $S=I$ reconstructs $R$ perfectly but generalises nothing (an item predicting itself = leakage); $s_{ij}\ge 0$ for interpretability, sparsity for efficiency. This is exactly the SLIM idea of Chapter 6, restricted to content-similar pairs.

Past paper Practice Exam 3 · CBF idea, TF-IDF, model + loss

Q. The main idea behind CBF and how it recommends; how TF-IDF weights attributes; and the CBF model in matrix notation with its loss.

Model answer. CBF builds an item-item similarity from item attributes (the ICM $A$ ), then recommends items similar to those the user liked — a similarity-weighted sum of the user’s own ratings. It needs no other users, so it handles new items and is explainable. TF-IDF: weight each attribute by $w_{a,i}=\text{tf}_{a,i}\cdot\text{idf}_a$ with $\text{idf}_a=\log\frac{\lvert\mathcal I\rvert}{n_a}$ ; an attribute carried by every item has $\text{idf}=0$ , rare ones score high. Model & loss. With $\tilde A$ the TF-IDF-weighted, row-normalised ICM, $S=\tilde A\,\tilde A^\top$ (zero diagonal, KNN-sparsified) and $\hat R = R\,S$ ; the learned variant minimises $\tfrac12\lVert R - R\,S\rVert_F^2 + \tfrac{\beta}{2}\lVert S\rVert_F^2 + \lambda\lVert S\rVert_1$ with $\operatorname{diag}(S)=0$ .

08 · Self-check

Three questions before you move on

Content-based item similarity is computed from:

Why keep only the top-K similarities per row (KNN) instead of the full similarity matrix?

An attribute that appears in every item in the catalogue has a TF-IDF inverse-document-frequency of:

09 · Recap

One-screen summary

Chapter 04 — load-bearing ideas

CB reads the ICM: item-to-item similarity from shared attributes, via cosine (with an optional shrink term).
Build $S$ , then sparsify with KNN: keep each item’s top-K neighbours; $K$ is a tuned hyper-parameter (inverted-U quality).
Score = $\hat R = R\,S$ : a similarity-weighted sum of the user’s own ratings. Explainable, handles new items — but low diversity.
TF-IDF sharpens attribute weights: $\text{idf}_a = \log(\lvert\text{ITEMS}\rvert/n_a)$ kills ubiquitous attributes.