Polimi · 2024/25

Natural Language Processing

From raw text and tokenizers to bag-of-words, language models, embeddings, transformers and large language models — the full Politecnico di Milano NLP course, rebuilt as an interactive, exam-focused study guide.

11 chapters~9 h reading 142 past-exam questions

Practice past exams 142 questions from 2023–2025 papers — filter by chapter, year and difficulty, with instant grading and worked solutions.

After this course you can

  • Preprocess raw text and compute TF-IDF weights for retrieval and classification
  • Build and evaluate text classifiers with precision, recall, F1 and the confusion matrix
  • Compute n-gram probabilities with smoothing and read perplexity as language-model quality
  • Derive scaled dot-product attention and explain why attention alone lacks positional awareness
  • Fine-tune a pretrained transformer and explain masked language modelling, instruction tuning and RLHF
  • Assemble a retrieval-augmented generation pipeline and reason about its inference cost

Syllabus

  1. Foundations of NLP & Text Preprocessing

    What natural language is, why it is hard to process, and how to turn raw bytes into the tokens a model can read — tokenisation, normalisation, stemming vs lemmatisation, and the regular expressions that still do the surgical work.

     high exam weight. 45 minutes reading. 20 past-exam questions.

  2. Classifying Text

    Turn documents into labels. Bag-of-Words as the canonical sparse representation, then three linear classifiers — Naïve Bayes, logistic regression, and the linear SVM — plus how to grade them with precision, recall and F1, and spelling correction as Naïve Bayes in disguise.

     high exam weight. 50 minutes reading. 10 past-exam questions.

  3. Searching & Clustering

    Vector-space information retrieval. TF-IDF reweights counts by how discriminative a word is; cosine similarity ranks by angle not magnitude; the inverted index makes search fast; then unsupervised structure — k-means clustering and LDA topic models.

     high exam weight. 45 minutes reading. 16 past-exam questions.

  4. Language Models & Embeddings

    Assign probabilities to word sequences. The chain rule and the n-gram Markov assumption, smoothing for unseen events, perplexity, and why sparse n-grams fail — motivating distributional semantics, word2vec, and the dense embeddings whose geometry encodes analogy.

     high exam weight. 50 minutes reading. 22 past-exam questions.

  5. Sequence Models & Labelling

    Order matters. Recurrent networks and the vanishing-gradient problem, the LSTM gates that fix it, and bidirectional context — applied to the two canonical token-labelling tasks, POS tagging and named-entity recognition, with the Viterbi algorithm for globally consistent tag sequences.

     high exam weight. 45 minutes reading. 10 past-exam questions.

  6. Seq2Seq, Attention & Transformers

    The architecture that changed everything. The seq2seq bottleneck, attention as its fix, self-attention via query/key/value, the Transformer block, positional encoding, and the BERT-vs-GPT split between encoders and decoders — plus the pretrain-then-fine-tune recipe.

     high exam weight. 55 minutes reading. 15 past-exam questions.

  7. Using Pretrained Models

    Putting transformers to work. Probing and fine-tuning BERT for understanding, sentence embeddings (SBERT) and vector databases for semantic search, CLIP for multimodal alignment, and the generation side — decoding strategies and in-context (zero/one/few-shot) learning.

     medium exam weight. 50 minutes reading. 6 past-exam questions.

  8. Large Language Models

    What changes when language models get big. Scaling laws and emergent abilities, alignment via instruction tuning and RLHF, chat templates, prompting and chain-of-thought, sampling, and the characteristic failure modes — hallucination, prompt injection, and overconfidence.

     high exam weight. 45 minutes reading. 17 past-exam questions.

  9. Efficient Inference & Deployment

    Making large models practical to run. Open vs closed weights and runtimes, quantization, the KV cache, grouped-query attention, FlashAttention, rotary embeddings, sliding-window attention, pre-norm, and mixture-of-experts — the toolbox that fits big models on small hardware and serves them fast.

     medium exam weight. 45 minutes reading. 3 past-exam questions.

  10. RAG, Agents & Adaptation

    Extending a frozen LLM. Retrieval-augmented generation grounds answers in real documents; ReAct-style agents reason and call tools; LoRA fine-tunes cheaply; and distillation compresses a big model into a small one. The toolkit for shipping LLM applications.

     high exam weight. 40 minutes reading. 10 past-exam questions.

  11. Speech & Dialogue

    Language as sound and conversation. Representing speech as spectrograms, automatic speech recognition (CTC and seq2seq), text-to-speech, dialogue state tracking and turn-taking, voice agents — and the ethical stakes of synthetic voices and always-listening systems.

     high exam weight. 40 minutes reading. 13 past-exam questions.