Large Language Models
What changes when language models get big. Scaling laws and emergent abilities, alignment via instruction tuning and RLHF, chat templates, prompting and chain-of-thought, sampling, and the characteristic failure modes — hallucination, prompt injection, and overconfidence.
01 · Recap
From language model to LLM
A large language model is the Chapter 4 next-token objective and the Chapter 6 decoder architecture — scaled up by orders of magnitude in parameters, data, and compute. Nothing about the loss is new; what changes is what emerges from sheer scale.
02 · Scale
Scaling laws and emergent abilities
Scaling laws are the empirical finding that test loss falls as a smooth power law in model size, dataset size, and compute — so performance is predictable before you train. The Chinchilla result refined the recipe: for a fixed compute budget, models were over-sized and under-trained; parameters and training tokens should scale together (roughly 20 tokens per parameter).
Emergent abilities are the surprise: some capabilities (multi-step arithmetic, in-context learning, instruction following) are near-random until a scale threshold, then appear sharply. The loss curve is smooth; specific task accuracies can jump.
Scaling laws vs emergence
Scaling laws describe the smooth, predictable fall of pretraining loss with scale. Emergence describes discontinuous jumps in specific downstream abilities once a model is large enough. They are compatible: a smoothly improving model can cross a usefulness threshold on a task abruptly. Chinchilla’s lesson: balance parameters and tokens — don’t just grow parameters.
03 · Alignment
Instruction tuning and RLHF
A raw pretrained LLM only continues text; it does not follow instructions or behave helpfully. Two steps bridge the gap:
1 · Instruction tuning
Supervised fine-tuning on (instruction, good response) pairs, teaching the model the assistant format and to actually answer rather than continue.
2 · RLHF
Collect human preference comparisons, train a reward model to predict them, then optimise the LLM against that reward (PPO or DPO). This aligns tone, helpfulness, and safety with human preference.
Why pretraining alone is not enough
Pretraining optimises next-token likelihood on internet text — which makes a fluent continuer, not a helpful, harmless assistant. Instruction tuning teaches the task format; RLHF aligns behaviour with human preferences (refusing harmful requests, being concise, admitting uncertainty) that the likelihood objective never captured.
04 · Chat structure
Chat templates
The chat box hides a structured prompt: a system message (persona and rules), then alternating user and assistant turns, wrapped in special tokens the model was tuned on. Getting this template right matters — a mismatched format degrades a chat-tuned model badly, because it was trained to expect exactly that structure.
05 · Prompting
Prompting and chain-of-thought
Most LLM use is prompt engineering: shaping the input to steer a frozen model. In-context examples (Ch. 7) are one lever; another is chain-of-thought — asking the model to reason step by step before answering — which sharply improves multi-step arithmetic and logic by giving the model “scratch space” in its own output.
In-context learning — zero / one / few-shot
Pick a task and a shot count. The prompt below is exactly what you send; the model continues the pattern. No fine-tuning — the “learning” happens entirely in the context window.
Zero-shot (instruction only)
Classify the sentiment of the review as positive, negative, or neutral.
Review: This is the best book I have read all year.Sentiment: positiveHighlighted text = the continuation the model is expected to produce; everything above it is the prompt you actually send.
Chain-of-thought, in one line
Appending “Let’s think step by step” (zero-shot CoT) or showing worked reasoning (few-shot CoT) makes the model externalise intermediate steps. Accuracy on reasoning tasks rises because each step conditions the next — the model is no longer forced to leap to the answer in a single token.
06 · Decoding
Sampling at scale
The same decoding knobs from Chapter 7 govern an LLM’s output: greedy/beam for deterministic, factual-leaning tasks; temperature with top-k / top-p for open-ended generation. There is no universal best setting — match it to whether you want one reliable answer or creative variety.
Decoding strategies
Greedy takes the top word every step (deterministic, repetitive). Top-k / top-p sample from the trimmed head; temperature reshapes the distribution first. Press Generate to resample.
Generated: the cat quickly walked
the from the 0.50a 0.25my 0.15cat from cat 0.40dog 0.30robot 0.15quickly from quickly 0.35slowly 0.25happily 0.20walked from walked 0.40ran 0.30jumped 0.1807 · Failure modes
Where LLMs go wrong
Hallucination
The model states false information fluently and confidently. It optimises plausibility, not truth; it has no built-in fact-checker. Retrieval grounding (Ch. 10) is the main mitigation.
Prompt injection
Untrusted input (a web page, a document) contains instructions the model follows, overriding the developer’s. A genuine security problem for tool-using agents.
Overconfidence
Output fluency is uncorrelated with correctness; the model rarely signals uncertainty unless trained to. Confident tone is not evidence.
Sensitivity
Small prompt changes can swing the answer; the model can be sycophantic, agreeing with a leading question even when wrong.
Why hallucination is intrinsic, not a bug to patch
An LLM is trained to produce the most probable continuation, not the most true one. When it has not seen the fact, the most probable continuation is still a fluent, plausible-sounding statement — which may be false. This is why grounding (RAG), citations, and “I don’t know” calibration matter more than any single decoding tweak.
08 · Self-check
Questions before you move on
What do neural scaling laws describe?
Why is RLHF applied on top of a pretrained, instruction-tuned LLM?
Chain-of-thought prompting improves reasoning because it:
Hallucination in LLMs is best understood as:
09 · Recap
One-screen summary
Chapter 08 — load-bearing ideas
- An LLM is the next-token objective and decoder architecture scaled up massively — same loss, new behaviour.
- Scaling laws make pretraining loss predictable (power law); emergent abilities appear abruptly at scale. Chinchilla: balance parameters and tokens.
- Instruction tuning + RLHF turn a text continuer into an aligned assistant.
- Chat templates (system/user/assistant + special tokens) are the structure the chat box hides.
- Prompting steers a frozen model; chain-of-thought externalises reasoning steps and boosts multi-step tasks.
- Sampling knobs (greedy/beam vs temperature/top-k/top-p) trade reliability for diversity.
- Failure modes — hallucination (plausibility ≠ truth), prompt injection, overconfidence, prompt sensitivity — motivate grounding and calibration.
10 · Exam · past papers
Past-paper questions
Q-LLM1Neural scaling laws state that test loss decreases:
Q-LLM2The Chinchilla finding is that, for a fixed compute budget, you should:
Q-LLM3RLHF (Reinforcement Learning from Human Feedback) primarily:
Q-LLM4Chain-of-thought prompting most improves performance on:
Q-LLM5Hallucination occurs because an LLM optimises for:
Q-LLM6Prompt injection is a risk primarily for: