Prerequisites: Linear Algebra, Probability & the Z-Transform · Model Identification and Data Analysis

01 · Motivation

What MIDA1 assumes on day one

The MIDA1 exam is, on every structured paper, four problems: a process-analysis problem, a prediction problem, an identification problem, and open theory. Not one of them can be started without the mathematics on this page — and not one lecture stops to teach it. It is inherited from the Bachelor’s calculus, linear-algebra and probability courses, and from the signals-and-systems background. This chapter collects that inheritance and points each piece at the exact MIDA1 chapter that will spend it.

This material is used from the first lecture and taught in none of them. It carries no exam weight of its own — no past paper tests “take an expectation” directly, and every past paper depends on it.

Every row below is evidence, not a generic syllabus: each prerequisite is here because a later chapter spends it, and the audit cites the line where that first happens.

you need to be able to…	first spent in	graded by
read $\sum$ and $\prod$ over an indexed set	ch.01 `foundations`	AR/MA variance, least squares (ch.03, ch.06)
state a property $\forall\,\tau$ and chase an implication	ch.02 `stochastic-processes`	Stationarity & representations (ch.02)
use $\in$ , $\subseteq$ , $\cup$ and a cardinality $\lvert A\rvert$	ch.02 `stochastic-processes`	Power spectral density (ch.04)
take an inner product and a transpose $A^{\top}$	ch.01 `foundations`	Least squares estimation (ch.06)
form a linear combination of past values	ch.03 `ma-ar-arma-models`	AR(1) / MA(2) variance (ch.03)
take an expectation $E[\cdot]$ and a variance	ch.02 `stochastic-processes`	Mean, covariance & spectrum (ch.02)
read a Pearson correlation coefficient	ch.02 `stochastic-processes`	Mean, covariance & spectrum (ch.02)
recognise a random walk / integrated signal	ch.06 `identification`	Experiment design (ch.09)

Two further primers follow that the audit did not surface but the course quietly assumes: the Z-transform and the delay operator (section 05) — the language every model from chapter 03 on is written in — and a one-page least-squares refresher (section 06), the minimisation chapter 06 then applies to data. They are authored background, flagged again where they begin.

key

How to read this chapter

Skim the ledger. Anything you can already do, skip — each section is self-contained. Anything you cannot, read in full and do its exercise; the chapter that spends it will not slow down for you.

The running example: a room-temperature sensor

One example runs through every section, chosen because it is the object the course studies. A thermostat samples the deviation $y(t)$ of a room’s temperature from its setpoint, once a minute. The room has thermal inertia — it forgets slowly — and it is buffeted by random disturbances $e(t)$ : a door opening, sun through a window. The simplest honest model is first order:

The sensor model

y(t) = a\,y(t-1) + e(t), \qquad a = 0.5, \quad e(t)\ \text{white noise}.

Memory (the $y(t-1)$ term) is what makes it a dynamical system; the white $e(t)$ is the part no model can predict.

By the end of the chapter you will have written this signal’s mean and variance, its correlation across time, its transfer function and pole, and estimated its coefficient $a$ from a handful of readings by least squares — every prerequisite exercised on one object. It is deliberately the AR(1) process that chapter 02 analyses, chapter 03 solves with Yule–Walker, and chapter 06 identifies from data.

×

The prerequisite errors that cost the most marks

Treating the delay operator $z^{-1}$ as a number. It is the shift $z^{-1}y(t) = y(t-1)$ , not “one over $z$ ” to cancel at will. Mishandle it and every transfer function, spectral factorisation and predictor in chapters 04–07 comes out wrong.
Confusing the expectation $E[\cdot]$ with the sample average $\tfrac{1}{N}\sum$ . Expectation is the theoretical (ensemble) operator; the average is its estimator. Stationarity, Yule–Walker and the asymptotic analysis of chapter 07 are all stated in $E[\cdot]$ — conflate the two and the derivations dissolve.
Dropping a transpose in the regressor. Chapter 06 stacks data into a matrix $\Phi$ and a vector $y$ ; one misplaced transpose makes $\Phi^{\top}\Phi$ non-conformable and the normal equations meaningless.

02 · Notation

The notation you read in

Three pieces of notation appear on nearly every slide, and none is ever defined: the summation, the set, and the quantifier. Fluency here is not optional — a stationarity definition is one short sentence in this language and a paragraph without it.

Summation and product notation: $\sum_{t=1}^{N} x_t$ adds $x_1 + \dots + x_N$ ; $\prod_{t=1}^{N} x_t$ multiplies them. The index and its range are part of the meaning — $\sum_{t}$ over “all $t$ ” is an infinite sum and only makes sense when it converges.
Set notation: $x \in A$ (membership), $A \subseteq B$ (subset), $A \cup B$ and $A \cap B$ (union, intersection), and the cardinality $\lvert A\rvert$ (how many elements). Set-builder form $\{\, t : t \in \mathbb{Z} \,\}$ reads “the $t$ such that…”.
First-order logic: $\forall$ (“for all”), $\exists$ (“there exists”), and $\Rightarrow$ (“implies”). A property that must hold at every time is a $\forall t$ statement; a definition that trades one condition for another is an implication.

Watch these three combine. The sample mean of the sensor series over $N$ readings is a summation,

Sample mean

\hat m_N = \frac{1}{N}\sum_{t=1}^{N} y(t),

An estimator — a number computed from data. Section 04 contrasts it with the theoretical mean $E[y(t)]$ .

and weak stationarity, the property chapter 02 opens with, is a single quantified sentence: the mean is constant and the covariance depends only on the lag,

E[y(t)] = m \ \ \forall t \in \mathbb{Z}, \qquad \text{and} \qquad \mathrm{Cov}\big(y(t), y(t-\tau)\big) = \gamma(\tau) \ \ \forall t,\, \forall \tau.

If you cannot read that line — the $\forall$ , the index set $\mathbb{Z}$ , the fact that $\gamma$ is a function of $\tau$ alone — chapter 02 will feel like a wall. Read it as: no matter which instant $t$ you pick, the average is the same number $m$ , and the covariance between two samples depends on how far apart they are, not on where they sit.

Q

Where this is graded

Every “Stationarity & representations” question (ch.02) turns on parsing exactly this quantified statement, and every “Power spectral density” question (ch.04) sums a series $\sum_{\tau} \gamma(\tau) e^{-j\omega\tau}$ over the integers. The notation is half the answer.

A signal has E[y(t)] = 3 for every t, but its covariance Cov(y(t), y(t−τ)) depends on both t and τ. Is it weakly stationary?

03 · Core idea

Linear algebra — the machinery of every estimator

Identification turns data into a vector of numbers and a model into another vector, then measures the gap between them. That is linear algebra, and three operations carry all of it: the linear combination, the inner product, and the transpose.

Vector: An ordered list $\mathbf{v} = (v_1, \dots, v_n)^{\top} \in \mathbb{R}^n$ , written as a column. In this course a vector is usually a stack of past measurements or a stack of model parameters.
Linear combination: $c_1\mathbf{v}_1 + \dots + c_k\mathbf{v}_k$ — scale each vector and add. An AR model is a linear combination: $y(t)$ built from past outputs with coefficients $a_1, \dots, a_n$ .
Inner product: $\mathbf{v}^{\top}\mathbf{w} = \sum_{i} v_i w_i$ , a single number. It measures alignment; it is zero exactly when the two vectors are orthogonal.
Transpose: $A^{\top}$ flips rows and columns; $(AB)^{\top} = B^{\top}A^{\top}$ . A matrix transpose is what turns a stack of row-observations into the normal-equations product $\Phi^{\top}\Phi$ .
Euclidean norm: $\lVert \mathbf{v}\rVert = \sqrt{\mathbf{v}^{\top}\mathbf{v}} = \sqrt{\sum_i v_i^2}$ — the length of a vector, and therefore the size of an error when $\mathbf{v}$ is a residual.

The one identity to keep at your fingertips is that the squared length of a vector is an inner product with itself,

Norm as inner product

\lVert \mathbf{v}\rVert^2 = \mathbf{v}^{\top}\mathbf{v} = \sum_{i=1}^{n} v_i^2,

A sum of squares is a squared norm. Every least-squares cost in the course is one of these in disguise.

because the entire prediction-error framework minimises a sum of squared residuals, which is a squared norm, which is an inner product. When chapter 06 writes the least-squares cost as $\lVert y - \Phi\theta\rVert^2$ and chapter 05 defines the optimal predictor as the one with minimum mean-square error, they are both measuring a length with this formula.

tip

The sensor, as vectors

Take the readings $y = (1.0,\ 0.4,\ 0.5,\ 0.1,\ 0.2)$ . To ask “how well does the previous sample explain the next one?”, stack the inputs $\varphi = (1.0,\ 0.4,\ 0.5,\ 0.1)$ (each $y(t-1)$ ) against the targets $y = (0.4,\ 0.5,\ 0.1,\ 0.2)$ (each $y(t)$ ). Their inner product $\varphi^{\top}y = 1.0(0.4) + 0.4(0.5) + 0.5(0.1) + 0.1(0.2) = 0.67$ and the self-product $\varphi^{\top}\varphi = 1.0 + 0.16 + 0.25 + 0.01 = 1.42$ are the only two numbers the estimate of $a$ will need — section 06 finishes the job.

Q

Where this is graded

“Least squares estimation” and “ARX identification by least squares” (ch.06) are, mechanically, the assembly of $\Phi^{\top}\Phi$ and $\Phi^{\top}y$ from data. A dropped transpose or a mis-shaped stack is the most common way to lose the whole problem before any statistics begin.

Φ is an N×p matrix of stacked observations (N rows, p parameters). Which product is well-formed and gives the p×p matrix the normal equations need?

04 · Mechanics

Probability — the language of a random signal

A stochastic process is a signal whose value at each instant is a random variable. To describe one you need four numbers-from-randomness: the expectation, the variance, the autocovariance, and the correlation coefficient that normalises it.

Expectation: $E[X]$ , the probability-weighted average of a random variable — its theoretical mean, the number the sample average $\tfrac1N\sum$ estimates. Linear: $E[aX + bY] = aE[X] + bE[Y]$ .
Variance: $\mathrm{Var}(X) = E\big[(X - E[X])^2\big]$ , the expected squared spread about the mean. Its square root is the standard deviation.
Autocovariance: $\gamma(\tau) = \mathrm{Cov}\big(y(t), y(t-\tau)\big) = E\big[(y(t)-m)(y(t-\tau)-m)\big]$ — how the signal co-varies with a delayed copy of itself. $\gamma(0)$ is the variance.
Pearson correlation coefficient: $\rho(\tau) = \gamma(\tau)/\gamma(0) \in [-1, 1]$ — the autocovariance stripped of units, so $\rho = 1$ is perfect alignment and $\rho = 0$ is none.
Gaussian distribution: $\mathcal{N}(\mu, \sigma^2)$ , the bell curve fixed entirely by its mean and variance — which is why a stationary Gaussian process is fixed entirely by $m$ and $\gamma(\tau)$ .
White noise: A zero-mean process with $\gamma(\tau) = 0$ for every $\tau \neq 0$ : no structure across time, the part of a signal no predictor can improve on.

Now watch expectation do real work on the sensor. Assuming it has settled to a stationary regime with zero mean, take the variance of both sides of $y(t) = a\,y(t-1) + e(t)$ . Because $e(t)$ is white and independent of the past, the cross term vanishes and

AR(1) variance

\gamma(0) = a^2\,\gamma(0) + \sigma_e^2 \quad\Longrightarrow\quad \gamma(0) = \frac{\sigma_e^2}{1 - a^2}.

This is a one-line Yule–Walker calculation (ch.03) — and it only works because E[·] is linear and E[y(t−1)e(t)] = 0.

Push the same idea one lag further and $\gamma(\tau) = a^{\lvert\tau\rvert}\gamma(0)$ , so the correlation coefficient of the sensor is $\rho(\tau) = a^{\lvert\tau\rvert}$ . At lag one, $\rho(1) = a = 0.5$ : for an AR(1) process the lag-1 correlation coefficient is the model coefficient itself. That single fact links the Pearson correlation you are revising here to the autocorrelation of chapter 02 and the Yule–Walker equations of chapter 03.

key

Random walk — the edge case that breaks stationarity

Set $a = 1$ and the sensor becomes $y(t) = y(t-1) + e(t)$ , a random walk: each value is the running sum of every disturbance so far. The variance formula above divides by $1 - a^2 = 0$ and blows up — the process is not stationary, it wanders without bound. Chapter 09 removes the wandering by differencing ( $\nabla y(t) = y(t) - y(t-1) = e(t)$ , which is white and stationary), the “I” in ARIMA. Recognising an integrated signal is a prerequisite you meet again in experiment design.

Q

Where this is graded

“Mean, covariance & spectrum” (ch.02) and every “AR(1) / MA(2) variance” problem (ch.03) are the calculation above, run on the specific model in the paper. The marks are in the linearity of $E[\cdot]$ and the vanishing of the noise cross term — not in arithmetic.

For the sensor y(t) = 0.5·y(t−1) + e(t) in steady state, what is the correlation coefficient ρ(1) between consecutive samples?

05 · Worked example

Discrete-time signals & the Z-transform

key

Authored primer — not an audit finding

The prereq-audit could not surface this section: the delay operator is pure notation, invisible to a concept probe, and “transfer function” registers as taught later (chapter 03). But every model from chapter 03 onward is written in this language, so it is genuine assumed background. Treat it as an appendix the exam still leans on.

MIDA1 never manipulates a signal one sample at a time for long. It packs the whole history into an operator and does algebra instead. The tool is the unit delay operator $z^{-1}$ , defined by

The delay operator

z^{-1}\,y(t) = y(t-1), \qquad z\,y(t) = y(t+1),

$z^{-1}$ shifts back one step; $z$ shifts forward. Some texts write it $q^{-1}$ . It is an operator, never a number.

With it, the sensor’s difference equation collapses. Move the past output to the left:

y(t) - a\,y(t-1) = e(t) \quad\Longrightarrow\quad (1 - a\,z^{-1})\,y(t) = e(t).

Now divide — formally, as an operator — to write the output as the noise passed through a transfer function $W(z)$ :

Transfer function of the sensor

y(t) = \underbrace{\frac{1}{1 - a\,z^{-1}}}_{W(z)}\,e(t) = \frac{z}{z - a}\,e(t).

A transfer function is a ratio of polynomials in z. Its roots carry all the dynamics.

Pole: A root of the denominator of $W(z)$ — here $z = a = 0.5$ . Poles govern how the system responds and whether it settles.
Zero: A root of the numerator. The sensor’s only zero is at $z = 0$ .
Asymptotic stability: A discrete-time system is asymptotically stable exactly when every pole lies strictly inside the unit circle, $\lvert z\rvert < 1$ . The sensor’s pole at $0.5$ qualifies, so its response to a disturbance decays.

The unit circle is the whole stability story in discrete time, and it is not decoration: a pole at $\lvert z\rvert = 1$ is the random walk from the previous section — marginally stable, non-stationary — and a pole outside it is a signal that explodes. Chapter 04 chooses the canonical spectral factor by demanding poles and zeros inside the circle; chapter 05 builds the whitening filter $1/W(z)$ , which is only usable when $W(z)$ ‘s zeros are inside the circle so the inverse’s poles are too. Every one of those moves is a statement about where roots sit relative to $\lvert z\rvert = 1$ .

key

Where the course picks this up

Chapter 03 writes MA, AR and ARMA models as $y(t) = \frac{C(z)}{A(z)}e(t)$ — exactly this ratio, with higher-degree polynomials. Chapter 04 factors spectra into $W(z)W(z^{-1})$ . Chapter 05 predicts by long-division of these polynomials. If $z^{-1}$ is comfortable now, all three are bookkeeping; if it is not, they are fog.

A process has transfer function W(z) = 1 / (1 − 1.2·z⁻¹). Is it asymptotically stable, and what does that imply?

06 · Mechanics

Least squares in one page

key

Authored primer — the math, not chapter 06

Least squares is taught in chapter 06, so the audit correctly treats it as course content, not a prerequisite. What you need before chapter 06 is the underlying minimisation — set a gradient to zero, get a linear system. That refresher is here; the identification it powers (ARX, the meaning of the regressors) belongs to chapter 06 and is left there.

Before any algebra, the shape of the answer. Least squares is a projection: the residual sticks out perpendicular to everything the model can reach, and that perpendicularity is the normal equations. Every estimator in this course is a version of this picture.

Stack the data. Write each measurement as a target $y(t)$ and a row of knowns $\varphi(t)^{\top}$ (the regressor); collect $N$ of them into a vector $y$ and a matrix $\Phi$ . A parameter vector $\theta$ predicts $\hat y = \Phi\theta$ , and least squares picks the $\theta$ that makes the residual shortest — smallest squared norm, the identity from section 03:

The least-squares problem

\hat\theta = \arg\min_{\theta}\ \lVert y - \Phi\theta\rVert^2.

Minimise a sum of squared errors — a squared Euclidean norm — over the parameters.

The cost is a quadratic in $\theta$ , so it has a single minimum where its gradient vanishes. Differentiating $\lVert y - \Phi\theta\rVert^2$ and setting the result to zero gives the normal equations:

Normal equations

\Phi^{\top}\Phi\,\hat\theta = \Phi^{\top}y \quad\Longrightarrow\quad \hat\theta = (\Phi^{\top}\Phi)^{-1}\Phi^{\top}y,

Solvable exactly when ΦᵀΦ is invertible — chapter 06 calls that condition persistent excitation.

Geometrically, $\Phi\hat\theta$ is the projection of $y$ onto the space its columns span, and the residual $y - \Phi\hat\theta$ is orthogonal to every column — that orthogonality is the normal equations, $\Phi^{\top}(y - \Phi\hat\theta) = 0$ .

Worked example Estimating the sensor coefficient by least squares

1 · Set up the one-parameter regression

The sensor $y(t) = a\,y(t-1) + e(t)$ has a single unknown $a$ , so $\theta = a$ is a scalar and each regressor row is just the previous sample: $\varphi(t) = y(t-1)$ . The normal equations collapse to one scalar equation.

2 · Assemble the two inner products

Reuse the readings $y = (1.0,\ 0.4,\ 0.5,\ 0.1,\ 0.2)$ from section 03. The four regression pairs $\big(y(t-1),\, y(t)\big)$ are $(1.0, 0.4),\ (0.4, 0.5),\ (0.5, 0.1),\ (0.1, 0.2)$ , so

\Phi^{\top}y = \textstyle\sum y(t-1)\,y(t) = 0.67, \qquad \Phi^{\top}\Phi = \textstyle\sum y(t-1)^2 = 1.42.

3 · Solve

With scalars the inverse is a division:

\hat a = \frac{\Phi^{\top}y}{\Phi^{\top}\Phi} = \frac{0.67}{1.42} \approx 0.47.

Close to the true $a = 0.5$ from five noisy points — and it would tighten as $N$ grows. That convergence-with-data is exactly the consistency question chapter 07 makes precise.

Why can the least-squares solution θ̂ = (ΦᵀΦ)⁻¹Φᵀy fail to exist?

07 · Practice

Work these before chapter 1

Do each on paper before opening its solution. They are ordered to mirror the chapter, and every one is shaped like the MIDA1 problem that will spend the prerequisite.

Try it Exercise 1 — Read a stationarity statement

Task. A process satisfies $E[y(t)] = 2 + \sin(t)$ and $\mathrm{Cov}(y(t), y(t-\tau)) = e^{-\lvert\tau\rvert}$ . Using the quantified definition, decide whether it is weakly stationary and say which clause decides it.

Solution. Weak stationarity needs both a constant mean $\forall t$ and a lag-only covariance $\forall t, \forall\tau$ . The covariance clause holds — $e^{-\lvert\tau\rvert}$ depends on $\tau$ alone. But the mean $2 + \sin(t)$ varies with $t$ , so the first clause fails and the process is not weakly stationary. The mean clause is the decider; a lag-only covariance is not sufficient on its own.

Try it Exercise 2 — An inner product and a norm

Task. For $\mathbf{v} = (2, -1, 3)^{\top}$ and $\mathbf{w} = (1, 4, 0)^{\top}$ , compute $\mathbf{v}^{\top}\mathbf{w}$ , decide whether they are orthogonal, and give $\lVert\mathbf{v}\rVert$ .

Solution. $\mathbf{v}^{\top}\mathbf{w} = 2(1) + (-1)(4) + 3(0) = 2 - 4 + 0 = -2$ . It is non-zero, so the vectors are not orthogonal (orthogonality means inner product exactly zero). The norm is $\lVert\mathbf{v}\rVert = \sqrt{2^2 + (-1)^2 + 3^2} = \sqrt{4 + 1 + 9} = \sqrt{14} \approx 3.74$ .

Try it Exercise 3 — Variance of an AR(1)

Task. A stationary zero-mean process obeys $y(t) = 0.8\,y(t-1) + e(t)$ with $\mathrm{Var}(e) = \sigma_e^2 = 1$ . Find $\gamma(0)$ , $\gamma(1)$ and the correlation $\rho(2)$ .

Solution. From $\gamma(0) = \sigma_e^2/(1 - a^2)$ with $a = 0.8$ : $\gamma(0) = 1/(1 - 0.64) = 1/0.36 \approx 2.78$ . Then $\gamma(1) = a\,\gamma(0) = 0.8(2.78) \approx 2.22$ . The correlation coefficient is $\rho(\tau) = a^{\lvert\tau\rvert}$ , so $\rho(2) = 0.8^2 = 0.64$ . Note $\rho$ needs no knowledge of $\sigma_e^2$ — the variance cancels.

Try it Exercise 4 — Difference equation to transfer function

Task. Write $y(t) = 0.6\,y(t-1) + e(t) - 0.2\,e(t-1)$ in the delay operator, give its transfer function $W(z)$ , and state its pole and zero. Is it asymptotically stable?

Solution. Group each side with $z^{-1}$ : $(1 - 0.6\,z^{-1})\,y(t) = (1 - 0.2\,z^{-1})\,e(t)$ . Hence

W(z) = \frac{1 - 0.2\,z^{-1}}{1 - 0.6\,z^{-1}} = \frac{z - 0.2}{z - 0.6}.

The pole is $z = 0.6$ and the zero is $z = 0.2$ . The single pole satisfies $\lvert z\rvert = 0.6 < 1$ , so the system is asymptotically stable. (Both roots are inside the unit circle, so this is also a canonical form in the sense of chapter 04 — worth noticing now.)

Try it Exercise 5 — A scalar least-squares fit

Task. You believe $y = a\,x$ (no intercept) and observe the pairs $(x, y) = (1, 1),\ (2, 3),\ (3, 4)$ . Use the normal equations to find $\hat a$ .

Solution. With one parameter the normal equation is $\hat a = \big(\sum x\,y\big)/\big(\sum x^2\big)$ . Numerator: $1(1) + 2(3) + 3(4) = 1 + 6 + 12 = 19$ . Denominator: $1^2 + 2^2 + 3^2 = 1 + 4 + 9 = 14$ . So $\hat a = 19/14 \approx 1.36$ . The fitted line $y = 1.36\,x$ is the projection of the data onto the single regressor direction $x$ , and the residual is orthogonal to it.

Try it Exercise 6 — Spot the prerequisite in a MIDA1 problem

Task. A prediction problem says: “Given the ARMA process $y(t) = \frac{1 + 0.5z^{-1}}{1 - 0.7z^{-1}}e(t)$ , verify it is in canonical form and compute the optimal one-step predictor.” Which prerequisites from this chapter are you tested on before writing a single predictor term?

Solution. Three. (1) The delay operator (section 05) — reading $\frac{C(z)}{A(z)}$ as a signal-shaping filter rather than a fraction of numbers. (2) Pole/zero location versus the unit circle (section 05) — “canonical form” means the pole $z = 0.7$ and the zero $z = -0.5$ both lie inside $\lvert z\rvert = 1$ , which you check here as $0.7 < 1$ and $0.5 < 1$ . (3) Expectation and whiteness (section 04) — the predictor is optimal when its error is white, an $E[\cdot]$ statement about the residual. The long division that produces the predictor is the easy half; these three decide whether you set it up correctly at all.

Chapter 01 models the sensor as y(t) = 0.5·y(t−1) + e(t) and calls a predictor 'optimal' when its residual is white noise. Which two prerequisites from this chapter does that single sentence rest on?

08 · Recap

Chapter 00 — what you must carry forward

Weak stationarity is one quantified sentence: a constant mean $\forall t$ and a covariance that depends on the lag $\tau$ alone. Read it fluently and chapter 02 opens; stumble on the $\forall$ and it does not.
A sum of squares is a squared norm is an inner product. $\lVert\mathbf{v}\rVert^2 = \mathbf{v}^{\top}\mathbf{v}$ is the identity under every least-squares cost and every mean-square error in the course.
Expectation is linear and the noise cross term vanishes — that pair of facts turns $y(t) = a\,y(t-1) + e(t)$ into $\gamma(0) = \sigma_e^2/(1 - a^2)$ in one line, and gives $\rho(\tau) = a^{\lvert\tau\rvert}$ . For an AR(1), the lag-1 correlation is the coefficient.
The delay operator $z^{-1}$ is an operator, not a number. It packs a difference equation into a transfer function $W(z)$ , and the dynamics live in the roots.
Stability is a compass reading against the unit circle. Poles inside $\lvert z\rvert < 1$ decay; a pole on it is a random walk; a pole outside explodes. Canonical forms, whitening filters and spectral factors are all statements about roots versus that circle.
Least squares is a projection. Minimising $\lVert y - \Phi\theta\rVert^2$ gives the normal equations $\Phi^{\top}\Phi\,\hat\theta = \Phi^{\top}y$ ; they are solvable exactly when $\Phi^{\top}\Phi$ is invertible — the condition chapter 06 names persistent excitation.
Exam radar: this chapter is never asked directly and is assumed in every problem. When a process-analysis or prediction problem feels impossible, the missing piece is usually on this page — most often a mishandled $z^{-1}$ or a confusion between $E[\cdot]$ and the sample average.

Next: chapter 01 picks up exactly here — it writes the sensor’s difference equation, frames prediction as an optimisation, and calls a predictor optimal precisely when its residual is the white noise you just defined.