Prerequisites: Notation, Signals & the Algebra of the Delay Operator

01 · Motivation

What MIDA2 assumes on day one

A MIDA2 paper is five questions — three numerical exercises and two theory questions, some of them in MATLAB. None of the five can be started without the material on this page, and no lecture stops to teach it. Module 2 opens where Module 1 closed and simply carries on: of the seventeen prerequisites the audit found, twelve are taught by MIDA1 and the remaining five are pure notation that no course in the curriculum lists at all.

This chapter carries no exam weight of its own, and correctly shows no weight meter — no past paper asks you to “take a gradient” for marks. Every past paper depends on your being able to.

Every row below is evidence, not a generic syllabus: each prerequisite is here because a later chapter spends it, and the audit cites the line where that first happens.

The running example: an e-scooter speed loop

One object runs through every section, chosen because it is the kind of plant MIDA2 exists to identify and control. A small electric scooter is commanded by a normalised throttle $u(t)\in[0,1]$ and its speed $y(t)$, in km/h, is read from a wheel encoder every $\Delta T = 0.5$ s. The vehicle has inertia — it forgets a throttle change only gradually — so the simplest honest model is first order:

Two numbers fall straight out and will be reused all chapter. The steady-state gain is $\mu = 5/(1-0.8) = 25$ km/h at full throttle, and the time constant is $\tau = -\Delta T/\ln 0.8 \approx 2.2$ s. By the end you will have written this plant as a transfer function and divided it out into an impulse response, given its disturbance a stochastic description, estimated the two coefficients $0.8$ and $5$ from data by least squares, and derived the whole model from Newton’s second law.

02 · Notation

The notation you read in

Four pieces of notation appear on nearly every MIDA2 slide and none is ever defined: the summation, the set-and-modulus, the quantifier, and the conditioning bar. They are not decoration. A stability theorem is one short sentence in this language and a confusing paragraph without it, and the audit found all four used across four or more chapters while no course in the curriculum lists them as concepts.

Summation

$\sum_{k=0}^{+\infty} \omega(k)\,u(t-k)$ adds one term per index value. The index and its range are part of the meaning — an infinite upper limit is a claim that the sum converges, which for an impulse response is exactly the claim that the system is stable.

Set membership and modulus

$\lambda \in \mathbb{C}$ says $\lambda$ is a complex number; $\lvert\lambda\rvert$ is its modulus, the distance from the origin. In discrete time the unit circle $\lvert z\rvert = 1$ is the stability boundary, so a modulus is a stability verdict.

Quantifiers and implication

$\forall$ (“for every”), $\exists$ (“there exists”), $\Rightarrow$ (“implies”). Derivations in chapter 01 are chains of implications; the definitions of observability and reachability are single sentences built from $\forall$ and $\exists$, and swapping the two changes the meaning.

The conditioning bar

$\hat y(t+k\mid t)$ — “the estimate of $y$ at time $t+k$, given everything measured up to time $t$”. The bar is not division. It names the information set, and MIDA2 is largely the study of what happens as that set changes.

Summation: the convolution sum

The first place chapter 01 uses a summation is the third representation of a linear system. If you know how the plant responds to a single unit kick — its impulse response $\omega(0), \omega(1), \omega(2), \dots$ — then its response to any input at all is a weighted sum of delayed kicks:

Modulus and quantifier: the stability test

Chapter 01 states asymptotic stability as a condition on the eigenvalues of the state matrix $F$. In continuous time the test is on the sign of the real part; in discrete time — MIDA2’s world throughout — it is on the modulus:

So an eigenvalue of $-0.9$ is stable — it oscillates while decaying — and an eigenvalue of $2$ is not. The scooter’s single pole is $0.8$, comfortably inside.

The conditioning bar

Chapter 04 opens by separating three problems that look almost identical and are not. The only thing distinguishing them is what stands on each side of the bar:

The gap between the last two is a single sample of data, and that one sample is the entire difference between a predictor and a filter — a distinction chapter 04 grades directly.

03 · Core idea

The plant, its clock and its linear combinations

Before any identification can happen you need to be fluent with the object being identified: a dynamical system, sampled on a clock, whose next output is a linear combination of what came before. All three ideas are used in chapter 01 and chapter 02 without introduction, and MIDA1 is where they were taught.

Dynamical system

A system with memory: the output now depends not only on the input now but on the past. The minimum number of numbers you must carry to summarise that past is the state, and how many of them there are is the order.

Sampling time

$\Delta T$, the interval between two consecutive measurements, in seconds. Its reciprocal is the sampling frequency in Hz. Discrete time $t$ counts samples, so wall-clock time is $t\,\Delta T$.

Linear combination

$c_1 v_1 + \cdots + c_n v_n$ — scale each term and add. A difference equation is a linear combination of past outputs and past inputs, and the coefficients are exactly what identification estimates.

Memory is what makes it dynamic

A static system computes $y(t)$ from $u(t)$ and nothing else. The scooter cannot: open the throttle and the speed rises over a couple of seconds rather than jumping. That lag is memory, and the whole of MIDA2 exists because of it. Writing that memory down as a linear combination gives the general $n$-th order difference equation:

The scooter is the case $n=1$ with $a_1 = 0.8$ and $b_1 = 5$. Nothing about identification is conceptually different at $n=4$ — there are simply more coefficients to estimate.

The clock

Every discrete model is a model of a sampling interval. Change $\Delta T$ and the coefficients change with it, because $a_1$ is really $e^{-\Delta T/\tau}$ in disguise — a fact section 07 derives and chapter 08 leans on heavily. Two consequences are worth internalising now:

• Halving $\Delta T$ does not halve $a_1$. It moves the pole towards $1$, since $e^{-\Delta T/\tau} \to 1$ as $\Delta T \to 0$. Sample the scooter at 50 Hz instead of 2 Hz and its pole moves from $0.8$ to about $0.991$.
• Sampling too slowly loses the dynamics entirely. The Nyquist frequency $\pi/\Delta T$ is the fastest thing the model can represent; anything quicker folds back and is indistinguishable.

04 · Mechanics

Dividing polynomials in the delay operator

This is the one section with no MIDA1 equivalent, and the one that pays for the most marks. Chapter 01 divides a transfer function to recover an impulse response; chapter 08 divides with a remainder to build the $k$-step predictor that minimum-variance control is derived from. Both are the same school long division, run on polynomials in $z^{-1}$ instead of on digits.

The cut moves with $k$: predict further ahead and more of the bar becomes unpredictable. Nothing else changes.

That picture is what the mechanics below are for. Everything on the green side of the cut is arithmetic you can do; everything on the red side is noise that has not happened yet. Keep it in mind — chapters 01, 05 and 08 each read it differently.

Delay operator

$z^{-1}$, defined by $z^{-1}y(t)=y(t-1)$. It is an operator, not a number, but it obeys the ordinary rules of polynomial algebra, which is precisely why long division works on it.

Transfer function

$G(z)=B(z)/A(z)$, a ratio of two polynomials in $z^{-1}$. Multiplying a signal by $G$ means running it through the plant.

Division to a power series: the impulse response

Write the scooter’s model with every $y$ on the left and factor out the operator:

$y(t) - 0.8\,y(t-1) = 5\,u(t-1) \;\Longrightarrow\; (1 - 0.8z^{-1})\,y(t) = 5z^{-1}u(t),$

so its transfer function is $G(z) = \dfrac{5z^{-1}}{1 - 0.8z^{-1}}$. Negative powers are the natural convention here precisely because they make this division come out as a series in how far back you look, which is what an impulse response is.

The general pattern for a first-order plant is $\omega(k) = b\,a^{k-1}$ for $k \ge 1$, and the $\omega(0)=0$ is the one-step input delay showing up as a leading zero. A system whose $\omega(0)$ is non-zero has direct feedthrough; chapter 01 splits that off before building canonical forms.

Division with a remainder: the $k$-step predictor

Chapter 08 needs something subtler. Rather than an infinite series it wants the division stopped after exactly $k$ terms, with whatever is left over kept as an explicit remainder. Dividing a polynomial $C(z)$ by $A(z)$ and halting after $k$ steps gives the identity:

The reason chapter 08 cares is that $E$ collects exactly the noise terms arriving within the next $k$ steps, which no predictor can know, while $\tilde R$ collects what is already determined by the past and therefore is predictable. The split of $C$ into “unknowable now” and “already decided” is performed by long division and nothing else.

05 · Core idea

Signals that are random, and the one that is purely random

From chapter 04 onwards nothing in MIDA2 is deterministic. The Kalman filter, software sensing and minimum-variance control are all built on a description of disturbance as a stochastic process, with white noise as the irreducible atom. MIDA1 taught all three ideas; MIDA2 uses them from its first noise term without restating them.

Stochastic process

A family of random variables indexed by time, $\{v(t)\}$. The crucial mental shift: $v(3)$ is not a number you have not measured yet — it is a random variable with a distribution. A data record is one realisation of the process, not the process.

White noise

$e(t) \sim \mathrm{WN}(0, \lambda^2)$: zero mean, constant variance $\lambda^2$, and uncorrelated across time — $E[e(t)e(s)] = 0$ for $t \neq s$. It is the part of a signal that no model, however good, can predict.

AR(1) process

$d(t) = a\,d(t-1) + \xi(t)$ with $\xi$ white. White noise passed through a one-pole filter: correlated with its own past, and therefore partly predictable.

Why white noise is the atom

White noise is defined by what it lacks. Zero mean means it adds no bias; constant variance means it does not change character over time; being uncorrelated means yesterday’s value tells you exactly nothing about today’s. That last property is the important one, because it is what makes white noise the unpredictable residue: the best possible prediction of $e(t)$ from all of history is $0$.

Every model in chapters 04–08 is therefore built the same way — an explicable part driven by data, plus a white part that is conceded rather than modelled. When chapter 08 computes the irreducible variance of a controlled system, it is counting exactly how much white noise the controller cannot reach in time.

AR(1): correlated disturbance

The scooter’s disturbance is not white. Road slope, headwind and rider posture change slowly, so a disturbance now resembles the disturbance a moment ago. The simplest description with that property is AR(1):

So a mild white input, recycled through a pole at $0.9$, produces a disturbance with more than five times its variance. This amplification is why coloured disturbance cannot simply be ignored, and it is the direct motivation for chapter 04’s model extension: the trick of enlarging the state so that a coloured disturbance becomes white again in the enlarged description.

06 · Mechanics

Least squares, argmin and the prediction error

Three chapters — 02, 03 and 06 — solve their central problem by writing down an objective and minimising it, and chapter 03 states the recipe outright as the standard four-step parametric procedure. The audit found least squares used eight times, the $\arg\min$ notation in four chapters, and the gradient in two, none of them defined here.

Prediction error

$\varepsilon(t) = y(t) - \hat y(t\mid t-1)$ — what the plant did, minus what your model said it would do. Every identification method in this course is a rule for making these small.

Objective function

$J(\theta)$, a single number scoring how badly parameter vector $\theta$ explains the data. Almost always the mean squared prediction error.

argmin

$\hat\theta = \arg\min_\theta J(\theta)$ — not the minimum value of $J$, but the $\theta$ at which it occurs. The distinction matters: identification wants the parameters, not the score.

Gradient

$\partial J/\partial\theta$, the vector of partial derivatives, one per parameter. Setting it to zero is the first-order condition for a minimum.

The four-step recipe

Chapter 03 lays out the procedure that every parametric method in the course instantiates:

Methods differ only in the choices they make at steps 2 and 3. Chapter 03 picks a frequency-domain index; chapter 06 picks a neural family; chapter 02 sidesteps the whole loop with a direct algebraic construction. Recognising the shared skeleton is what makes them feel like one course.

Least squares, worked on the scooter

Suppose you have logged throttle and speed and want the two coefficients. Write the model as an inner product of a regressor and a parameter vector:

Now score a candidate $\theta$ by its mean squared prediction error, and minimise:

$J(\theta) = \frac{1}{N}\sum_{t=1}^{N}\varepsilon(t)^2 = \frac{1}{N}\sum_{t=1}^{N}\big(y(t) - \varphi(t)^{\top}\theta\big)^2, \qquad \hat\theta = \arg\min_\theta J(\theta).$

$J$ is quadratic in $\theta$ and therefore has one minimum, found where the gradient vanishes. Differentiating and setting to zero produces the normal equations, whose solution is the least-squares estimate:

07 · Worked example

Where the model came from in the first place

Every number used so far was handed to you. This section earns them: it derives the scooter model from Newton’s second law, names the three ways a model can be obtained, and shows what experiment would be needed to pin down the parts physics cannot supply.

White-box modelling

Deriving the model from physical laws, with every parameter carrying units and meaning. Natural in continuous time, so you derive first and discretise afterwards.

Parametric identification

Choosing a model family $M(\theta)$ and estimating $\theta$ from data — the recipe of section 06, with the structure fixed in advance and only the numbers unknown.

Experiment design

Choosing what to inject and for how long, so the data actually contains the information the estimator needs.

White box: from Newton to a difference equation

The scooter obeys a force balance — motor thrust minus the drag and friction that grow with speed:

$m\,\dot v(t) = \frac{1}{r}T(t) - \beta\,v(t),$

with $m$ the combined mass of vehicle and rider, $r$ the wheel radius, $T$ the motor torque and $\beta$ an aggregate drag coefficient. Rearranged, this is a first-order linear ODE with time constant $\tau = m/\beta$. Sampling it with a zero-order hold — the input held constant between samples — gives a discrete model whose coefficients are not free at all:

Check it against the running example: with $\tau \approx 2.2$ s and $\Delta T = 0.5$ s, $a = e^{-0.5/2.2} \approx 0.80$, and with a top speed of $\mu = 25$ km/h, $b = 25 \times 0.2 = 5$. The two numbers used since section 01 were never arbitrary.

Three colours of model

The scooter is the natural gray-box case, and it shows why the category exists. The structure is solid Newtonian physics, $r$ is measurable with a ruler and $\Delta T$ is a setting — but $m$ includes the rider, so it changes every trip, and $\beta$ depends on posture, tyre pressure and wind. Chapter 07 handles exactly this by treating the unknown parameter as an extra state and letting an extended Kalman filter estimate it alongside the speed. That machinery is chapter 07’s to teach; all you need here is to see why physics alone could not finish the job.

The experiment that would pin it down

Section 06 showed that a constant throttle identifies nothing. What does work follows directly from what the estimator needs — enough independent variation to separate the parameters, sustained long enough for the plant to reveal its memory:

• Vary the input richly. A step gives one time constant’s worth of information; a square wave or a pseudo-random binary sequence gives many, at many frequencies.
• Log for several time constants. For the scooter, $\tau \approx 2.2$ s, so a run of 30 s at $\Delta T = 0.5$ s gives about 60 samples spanning some 13 time constants.
• Excite what you intend to model. Chapter 03 makes this literal: to identify a frequency response at a given frequency, inject a sinusoid at that frequency.

08 · Practice

Work these before chapter 1

Six exercises, one per prerequisite cluster, each shaped like the chapter that will spend it. Attempt each before opening the solution — recognition is not the same skill as production, and the exam tests the second.

09 · Recap

Chapter 01 begins by writing a plant three ways at once — state space, transfer function, impulse response — and asking which of the six conversions between them lose information. You now have every tool it uses: the convolution sum is the third representation, the long division is the route from the second to the third, and the modulus test is how it decides whether any of them is stable.