From raw text and tokenizers to bag-of-words, language models, embeddings, transformers and large language models — the full Politecnico di Milano NLP course, rebuilt as an interactive, exam-focused study guide.https://www.polistudy.me/enhttps://www.polistudy.me/nlp/foundations/https://www.polistudy.me/nlp/foundations/What natural language is, why it is hard to process, and how to turn raw bytes into the tokens a model can read — tokenisation, normalisation, stemming vs lemmatisation, and the regular expressions that still do the surgical work.tokenizationbyte-pair encodingstemminglemmatisationambiguityZipf's lawregular expressionshttps://www.polistudy.me/nlp/classifying-text/https://www.polistudy.me/nlp/classifying-text/Turn documents into labels. Bag-of-Words as the canonical sparse representation, then three linear classifiers — Naïve Bayes, logistic regression, and the linear SVM — plus how to grade them with precision, recall and F1, and spelling correction as Naïve Bayes in disguise.bag-of-wordsNaïve BayesLaplace smoothinglogistic regressionsupport vector machineprecision recall F1edit distancehttps://www.polistudy.me/nlp/searching-clustering/https://www.polistudy.me/nlp/searching-clustering/Vector-space information retrieval. TF-IDF reweights counts by how discriminative a word is; cosine similarity ranks by angle not magnitude; the inverted index makes search fast; then unsupervised structure — k-means clustering and LDA topic models.TF-IDFinverse document frequencycosine similarityinverted indexBM25k-meanslatent Dirichlet allocationhttps://www.polistudy.me/nlp/language-models/https://www.polistudy.me/nlp/language-models/Assign probabilities to word sequences. The chain rule and the n-gram Markov assumption, smoothing for unseen events, perplexity, and why sparse n-grams fail — motivating distributional semantics, word2vec, and the dense embeddings whose geometry encodes analogy.language modeln-gramMarkov assumptionsmoothingperplexitydistributional semanticsword2vecword embeddingshttps://www.polistudy.me/nlp/sequence-labelling/https://www.polistudy.me/nlp/sequence-labelling/Order matters. Recurrent networks and the vanishing-gradient problem, the LSTM gates that fix it, and bidirectional context — applied to the two canonical token-labelling tasks, POS tagging and named-entity recognition, with the Viterbi algorithm for globally consistent tag sequences.recurrent neural networkvanishing gradientLSTMBiLSTMPOS taggingnamed-entity recognitionViterbi algorithmhttps://www.polistudy.me/nlp/transformers/https://www.polistudy.me/nlp/transformers/The architecture that changed everything. The seq2seq bottleneck, attention as its fix, self-attention via query/key/value, the Transformer block, positional encoding, and the BERT-vs-GPT split between encoders and decoders — plus the pretrain-then-fine-tune recipe.sequence-to-sequenceattentionself-attentionquery key valuetransformerpositional encodingBERTGPTfine-tuninghttps://www.polistudy.me/nlp/using-pretrained-models/https://www.polistudy.me/nlp/using-pretrained-models/Putting transformers to work. Probing and fine-tuning BERT for understanding, sentence embeddings (SBERT) and vector databases for semantic search, CLIP for multimodal alignment, and the generation side — decoding strategies and in-context (zero/one/few-shot) learning.probingfine-tuningsentence embeddingsSBERTvector databasesemantic searchCLIPdecodingin-context learninghttps://www.polistudy.me/nlp/large-language-models/https://www.polistudy.me/nlp/large-language-models/What changes when language models get big. Scaling laws and emergent abilities, alignment via instruction tuning and RLHF, chat templates, prompting and chain-of-thought, sampling, and the characteristic failure modes — hallucination, prompt injection, and overconfidence.scaling lawsemergent abilitiesRLHFinstruction tuningchain-of-thoughtpromptinghallucinationhttps://www.polistudy.me/nlp/efficient-inference/https://www.polistudy.me/nlp/efficient-inference/Making large models practical to run. Open vs closed weights and runtimes, quantization, the KV cache, grouped-query attention, FlashAttention, rotary embeddings, sliding-window attention, pre-norm, and mixture-of-experts — the toolbox that fits big models on small hardware and serves them fast.quantizationKV cachegrouped-query attentionFlashAttentionrotary embeddingsmixture of expertshttps://www.polistudy.me/nlp/rag-and-agents/https://www.polistudy.me/nlp/rag-and-agents/Extending a frozen LLM. Retrieval-augmented generation grounds answers in real documents; ReAct-style agents reason and call tools; LoRA fine-tunes cheaply; and distillation compresses a big model into a small one. The toolkit for shipping LLM applications.retrieval-augmented generationvector searchReActtool useLoRAknowledge distillationhttps://www.polistudy.me/nlp/speech-and-dialogue/https://www.polistudy.me/nlp/speech-and-dialogue/Language as sound and conversation. Representing speech as spectrograms, automatic speech recognition (CTC and seq2seq), text-to-speech, dialogue state tracking and turn-taking, voice agents — and the ethical stakes of synthetic voices and always-listening systems.spectrogramSTFTmel scaleautomatic speech recognitionCTCtext-to-speechdialogue state tracking