Chapter 01

What Model Identification Is

The one problem behind the whole course — building a model of an uncertain dynamical system from data — and the vocabulary it runs on: white/grey/black-box modelling, static vs dynamical systems, prediction framed as an optimisation, and the white-noise residual that tells you a predictor is optimal.

Reading: ~16 min Interactive: 1 widgets Source: Polimi MIDA1 2025/26 — Lecture 1 (Course presentation) · Polimi MIDA1 2025/26 — Lecture 2 (Preliminary concepts)

01 · Motivation

From first principles to data

Every problem in this course — forecasting, control, filtering, fault detection — needs a mathematical model of a phenomenon. There are two ways to get one, and model identification is the second.

Direct modelling derives the equations from first principles — physical laws, prior knowledge. Model identification estimates them from experimental data. Which you can use depends on how much you already know about the system’s equations:

White box

The model comes entirely from first principles. Every variable has a physical meaning and the model generalises — but everything must be known exactly a priori, which is expensive and often infeasible for complex systems.

Black box

The model is estimated from historical data and statistical analysis. Fast and cheap, and it needs no process knowledge — but the parameters are not physically interpretable and the model is tied to the conditions under which the data were collected.

Grey box

A mix: the model structure comes from physical laws, the parameters from data. The pragmatic middle ground for most engineering systems.

Even a white-box model usually needs identification techniques, because real models carry structural uncertainty (the governing laws are not fully known) and parametric uncertainty (the coefficients need calibration). And a model is always built for a purpose: a simple model for control, a detailed one for simulation — the same system warrants different models.

why

Why identification is one pillar of AI

System identification is machine learning aimed at dynamical systems — estimating how a quantity evolves over time from observed data. Classical ML mostly models static input–output maps; this course is about the systems that have memory.

Because a computer stores a finite amount of data, continuous-time signals are sampled with a sampling time $T_s$ , giving values at discrete instants $kT_s$ . We keep writing $y(t)$ for the sampled signal, with $t$ now a discrete index. A discrete-time dynamical system is then described by a difference equation, for example

Difference eqn

$y(t) = 0.5\,y(t-1) + 3\,u(t).$

The output at time $t$ depends on the input and on the system’s own past — that memory is exactly what makes a system dynamical rather than static.

02 · Distinction

Static vs dynamical systems

A static system is one whose output is fully determined by the input at the same instant — a resistor, $i(t) = V(t)/R$ . There is no memory: yesterday’s voltage tells you nothing about today’s current. Classical regression (fit $y = f(\varphi)$ to a cloud of points) is the black-box modelling of static systems.

A dynamical system has memory: the current output depends on the current input and on the past history of inputs and outputs. We still want to learn a map $y(t) = f(\varphi(t))$ , but now the regressor $\varphi(t)$ contains past samples of both signals:

$\varphi(t) = \big[\,y(t-1),\ y(t-2),\ \dots,\ u(t-1),\ u(t-2),\ \dots\,\big].$

tip

Where dynamical models earn their keep

A GSM receiver identifies the channel $\hat G(z)$ on every call from a known preamble, so it can undo the echoes that smear the transmitted signal. A CD-player servo needs a model accurate well beyond its 1 kHz control bandwidth because of high-frequency mechanical modes that first-principles modelling misses. In both cases the model is estimated from data, because the true equations are unknown or change constantly.

03 · Setup

The prediction problem

Given past observations of a signal, predict its next value. This single problem organises the whole course — and it turns out to be an identification problem in disguise.

Given a time series $v(1), v(2), \dots, v(t-1)$ , a predictor is a function of the available data that estimates the next value $v(t)$ . We write $\hat v(t\mid t-1)$ for “the estimate of $v(t)$ given data up to $t-1$ ”. In practice the predictor is usually linear, uses a finite memory, and has constant parameters:

Predictor

$\hat v(t\mid t-1) = a_1 v(t-1) + a_2 v(t-2) + \dots + a_n v(t-n),\qquad \vartheta = [a_1\ \dots\ a_n]^\top.$

A specific predictor is one specific parameter vector $\vartheta$ . To measure how good it is, form the prediction error (or residual) at each time, $\varepsilon(i) = v(i) - \hat v(i\mid i-1)$ , and add up the squares:

Cost

$J(\vartheta) = \sum_i \varepsilon(i)^2.$

The best predictor is the $\vartheta$ that makes $J(\vartheta)$ smallest on the data we already have. So predicting is really identifying: we are searching for the model (a value of $\vartheta$ ) that best explains the observed signal. That is why the rest of the course alternates between predicting and identifying — they are two views of the same optimisation.

04 · Key idea

A white-noise residual means you’re done

Is the $\vartheta$ that minimises $J$ automatically a good model? Not necessarily — you have to look at the error sequence itself. If the residual has any leftover regularity — a persistent bias, a sign that flips every step — the predictor can still be improved by absorbing that structure.

A predictor is optimal exactly when the error it leaves has zero mean and no structure at all: a signal whose value at one instant is uncorrelated with every other instant. That signal is white noise.

key

The load-bearing idea of the whole course

White noise is completely unpredictable — there is no information in its past that helps predict its future. So if the residual is white, there is nothing left to extract, and the predictor is optimal. Conversely, since $v(t) = \hat v(t\mid t-1) + \varepsilon(t)$ , anything the predictor does not describe is just white noise. “Make the residual white” is the goal of every method that follows — and the whiteness test in the validation chapter is how you check it.

Writing the same relation with the unit-delay operator $z^{-1}$ turns the recursive predictor into a transfer function from a white-noise input $\varepsilon(\cdot)$ to the signal:

$v(t) = W(z)\,\varepsilon(t), \qquad W(z) = \frac{1}{1 - a_1 z^{-1} - \dots - a_n z^{-n}}.$

So a signal worth predicting is modelled as white noise shaped by a dynamical filter — the exact object the next chapters study as AR, MA and ARMA processes.

identification-loop-smieThe identification loop (S/M/I/E)easy4 pts

Describe the four ingredients of a parametric identification problem (system, model, method, experiment) and explain why a white-noise residual signals an optimal predictor.

05 · Framework

System, Model, Method, Experiment

Every identification problem has four ingredients. Keep them straight — the rest of the course is about getting each one right. Three of them are yours to choose; one is not, and the whole loop has exactly one exit test.

S — System

The mechanism that generates the data. It is uncertain, so we model it as a stochastic dynamical system.

M — Model

A parametric family $M(\vartheta)$ . Choosing the family — the structure — is itself a hard decision (too simple misses dynamics; too rich cannot be identified).

I — Method

The tool that picks the best model in the family, typically an optimisation of the prediction-error cost.

E — Experiment

The data-collection setup. The experiment must excite the system — a model can never say more than the data say.

✓

Kalman: "Let the data speak"

The experiment is not an afterthought. If the input never excites a mode of the system, no method can recover it — a point we make precise with persistent excitation and identifiability later. Design the experiment as carefully as the model.

A residual sequence from your best-fit predictor still alternates sign at every time step. What does that tell you?

Load-bearing ideas

Identification estimates a model from data — white box (all physics), black box (all data), grey box (structure from physics, parameters from data). Real models carry structural and parametric uncertainty.
Dynamical systems have memory: the output depends on past inputs and outputs, so we work in discrete time with difference equations and regressors of past samples.
Prediction is identification: choose the predictor $\vartheta$ that minimises the sum of squared prediction errors $J(\vartheta)$ .
A white-noise residual means the predictor is optimal — nothing predictable is left. This is the thread running through the entire course; every method aims to make the residual white.
Four ingredients — S / M / I / E — the System, the Model family, the Method, and the Experiment that must excite the system.

Exam radar. This chapter is framing, not a graded topic on its own — but the ideas here (the white-noise-residual criterion, the S/M/I/E loop, prediction-as-optimisation) are the vocabulary every later exam problem is written in. Make sure “why must the residual be white?” is a reflex.