Chapter 07

The Extended Kalman Filter and Gray-Box Identification

Two ideas that turn out to be one. Linearising a nonlinear model at every step gives the extended Kalman filter; promoting an unknown physical parameter to a state lets the same filter identify it. The second trick forces the first — appending a parameter to the state multiplies two unknowns together, so a linear plant stops being linear the moment you try this.

Reading: ~29 min Interactive: 1 widgets Source: Polimi MIDA2 2025/26 — Lecture 8, slides 4–18 (Ch. 2.3 — the EKF and an end-to-end design) · Polimi MIDA2 2025/26 — Lecture 25 (Ch. 2.5 — gray-box identification via state extension)

01 · Extension 5

The extended Kalman filter

Chapter 04 listed five extensions to the basic Kalman filter and deferred the fifth. Here it is — and unlike the other four, it is not free.

The plant is now nonlinear in the state:

A nonlinear state-space model

$\begin{cases}x(t+1)=f\big(x(t),u(t)\big)+v_1(t)\\ y(t)=h\big(x(t)\big)+v_2(t)\end{cases}$

with $f$ and $h$ of class $C^1$ or better — differentiable, which is the only property the construction actually needs. Something like $x(t+1)=\tfrac12x^{5}(t)+u^{3}(t)+v_1(t)$ with $y(t)=e^{x(t)}+v_2(t)$ qualifies.

The strategy is unchanged: run a replica of the plant and correct it with the output error. The only open question is what the gain should be, and there are two candidate answers.

Option 1 — a nonlinear gain

The correction block outputs $K\big(e(t)\big)$ , with $K(\cdot)$ itself a nonlinear function of the innovation. The most intuitive and natural answer — and it discards the entire linear theory.

Option 2 — a linear time-varying gain

The block outputs $K(t)\cdot e(t)$ , exactly as before, with $K(t)$ recomputed each step. Less intuitive, and it re-uses almost all of the Kalman theory. This is what the EKF does.

Linearise, then apply what you already have

The gain and the Riccati recursion keep their usual shape; only $F$ and $H$ become time-varying:

The EKF gain and DRE

K(t)=\big(F(t)P(t)H(t)^{\top}+V_{12}\big)\big(H(t)P(t)H(t)^{\top}+V_2\big)^{-1}

with $P(t+1)$ given by the same STATE − MIX·OUTPUT⁻¹·MIXᵀ expression, and the two matrices obtained as Jacobian matrices, recomputed at every sampling instant:

Local linearization around the predicted state

F(t)=\left.\frac{\partial f\big(x(t),u(t)\big)}{\partial x(t)}\right|_{x(t)=\hat x(t|t-1)}, \qquad H(t)=\left.\frac{\partial h\big(x(t)\big)}{\partial x(t)}\right|_{x(t)=\hat x(t|t-1)}

key

The EKF turns extension 5 into extension 4

The whole idea in one sentence: approximate a nonlinear time-invariant system by a linear time-varying one, so that chapter 04’s fourth extension — which cost nothing at all — applies.

Note where the Jacobians are evaluated: at $\hat x(t|t-1)$ , the predicted state, not the filtered one and certainly not the true state, which is unknown. The linearisation is taken about the best guess available at the moment it is needed.

The loop One EKF step

Take the latest prediction

Start from $\hat x(t|t-1)$ , carried over from the previous iteration.

Linearise about it

Evaluate the two Jacobians at $\hat x(t|t-1)$ to get $F(t)$ and $H(t)$ .

Update the Riccati recursion and the gain

Feed $F(t)$ and $H(t)$ into the DRE to get $P(t+1)$ , and into the gain formula to get $K(t)$ .

Advance the estimate

Compute $\hat x(t+1|t)$ from the correction equation, and loop.

What you gave up

There is no steady state, and therefore no shortcut and no guarantee

The EKF is nonlinear and time-varying, so the whole of chapter 05 does not apply: no algebraic Riccati equation, no $\bar P$ , no $\bar K$ , no asymptotic theorems. Two consequences follow, and neither is small.

Asymptotic stability is impossible or very difficult to guarantee. There is no $F-\bar KH$ to test. In practice only extensive empirical validation is available — you run it on a lot of data and hope.
The computational load is high. $F(t)$ , $H(t)$ , $P(t)$ and $K(t)$ must all be recomputed at every sampling instant, where a steady-state linear filter would use one precomputed constant.

Despite that, the EKF is used heavily in real applications — with reservations exactly where the missing guarantee matters most: safety-critical systems (velocity estimation inside ABS) and mission-critical ones (a rover on Mars).

02 · End to end

Designing a filter for a real plant

Every worked example so far arrived with $F$ , $H$ and the covariances already supplied. Getting them is four steps, and only the last one is genuinely hard.

The problem: a mass–spring–damper with known stiffness $K$ , mass $M$ and friction coefficient $C$ . The input is the applied force $F(t)$ ; the measured output is the position $x(t)$ . Estimate the velocity $\dot x$ in software.

why

The obvious answer, and why it is not the right one

Differentiate the position numerically and low-pass the result. The filter is needed because numerical differentiation amplifies high-frequency noise — but it introduces phase shift, which can be critical inside a control loop.

It is not optimal. A Kalman filter gives an estimate that is both less noisy and less phase-shifted, because it is using a model of how position and velocity are related rather than treating the signal as an arbitrary time series.

Four steps From physics to a running filter

Step 1 — build the white-box model, in continuous time

Newton’s second law on the mass:

$M\ddot x(t)=\underbrace{-Kx(t)}_{\text{spring}}\underbrace{-C\dot x(t)}_{\text{friction}}+F(t)$

A linear second-order ODE. Physical white-box modelling comes naturally in continuous time, so start there and convert later. Split it into two first-order equations by taking the two natural mechanical states — position and velocity:

$\begin{cases}\dot x_1(t)=x_2(t)\\ \dot x_2(t)=-\tfrac{K}{M}x_1(t)-\tfrac{C}{M}x_2(t)+\tfrac1MF(t)\\ y(t)=x_1(t)\end{cases}$

Step 2 — discretise

Euler forward, $\dot x(t)\approx\big(x(t+1)-x(t)\big)/\Delta$ with $\Delta$ the sampling time — say 10 ms. Substituting and rearranging gives a discrete-time model whose matrices are

F=\begin{bmatrix}1&\Delta\\ -\tfrac{K\Delta}{M}&1-\tfrac{C\Delta}{M}\end{bmatrix},\quad G=\begin{bmatrix}0\\ \tfrac{\Delta}{M}\end{bmatrix},\quad H=\begin{bmatrix}1&0\end{bmatrix}

This is an approximation, and $\Delta$ is the knob that controls how good it is.

Step 3 — add the noise

The model so far is deterministic; the Kalman filter needs the stochastic version. Add $v_1(t)$ to the state equation and $v_2(t)$ to the output:

\begin{cases} x_1(t+1)=x_1(t)+\Delta x_2(t)+v_{11}(t)\\ x_2(t+1)=-\tfrac{K\Delta}{M}x_1(t)+\big(1-\tfrac{C\Delta}{M}\big)x_2(t)+\tfrac{\Delta}{M}F(t)+v_{12}(t)\\ y(t)=x_1(t)+v_2(t) \end{cases}

Step 4 — choose the covariances

The only genuinely open step, handled empirically:

$V_{12}=0$ — there is no reason for the process and measurement noises to be correlated here.
$V_2$ — from the position sensor’s datasheet, or by measuring its noise directly.
$V_1$ — the hard one. Simplify by assuming it diagonal, $V_1=\mathrm{diag}(\lambda_{11}^{2},\lambda_{12}^{2})$ , which turns an $n\times n$ unknown into two scalars to be tuned.

key

Where the hard step went

Chapter 04 flagged $V_1$ as the Kalman filter’s real weak point, and this is where the flag is redeemed. Modern practice treats $\lambda_{11}^2$ and $\lambda_{12}^2$ as meta-parameters estimated from a dataset in a black-box way: collect a training set and minimise the state estimation error over them.

Which is worth noticing for what it is. A model-based method, at its very last step, reaches for a machine-learning technique — the same convergence chapter 06 approached from the other side.

03 · The problem

Known structure, unknown parameters

The Kalman filter was built to estimate states. It turns out to estimate physical constants too, with no new theory at all — a side benefit that chapter 04 promised and this section collects.

Two stems, three sittings — and nothing since July 2024

Gray-box identification has been set twice with distinct stems, spanning three sittings: “explain how the Kalman filter can be used to solve grey-box identification problems” (15/07/2023 and again, identically, 25/06/2024) and “describe what offline grey-box identification is, with two example input signals” (26/07/2024). Three marks each.

It is also the module’s fading topic: it has not appeared in any of the seven sittings since 26/07/2024. Read that as a reason to know it rather than to drill it — the recorded weight is low and the trend is downward, but a 3-mark theory question is cheap to secure and both stems are answerable from this chapter and the next section.

The setting. A model is available, built white-box from physics:

A gray-box model

$\begin{cases}x(t+1)=f\big(x(t),u(t);\theta\big)+v_1(t)\\ y(t)=h\big(x(t);\theta\big)+v_2(t)\end{cases}$

with $f$ and $h$ known functions containing a few unknown parameters $\theta$ — and $\theta$ has physical meaning: a friction coefficient, a mass, a time constant. Given a dataset $\{u(1),\dots,u(N)\}$ and $\{y(1),\dots,y(N)\}$ , estimate $\hat\theta_N$ .

key

This is the precise definition of gray box

Not “partly known” in a vague sense. The structure is fully known and a handful of physical constants are not. Place it against the rest of the module:

	structure	parameters	physical meaning
chapters 02–03, black box	unknown	fitted	none
chapter 04, white box	known	known	full
this chapter, gray box	known	a few unknown	full

The payoff is that the identified numbers mean something — which is exactly what chapter 06’s black-box sensor could not offer.

The trick, stated

State extension

\begin{cases}x(t+1)=f\big(x(t),u(t);\theta(t)\big)+v_1(t)\\ \theta(t+1)=\theta(t)+v_\theta(t)\\ y(t)=h\big(x(t);\theta(t)\big)+v_2(t)\end{cases} \qquad x_E(t)=\begin{bmatrix}x(t)\\ \theta(t)\end{bmatrix}

Append a state equation for $\theta$ and stack it under the physical state. An unknown parameter has been reclassified as an unknown state, and the entire Kalman machinery — built for states — now estimates it. No new theory; a change of bookkeeping.

2023-07-q52023Q05Gray-box identificationmedium3 pts

Explain and discuss how the Kalman Filter can be used to solve grey-box system-identification problems.

04 · The trick

Promoting parameters to states

The appended equation looks trivial and is not. Each of its three features — the constant dynamics, the marginal stability, the added noise — is doing work, and one of them is the chapter’s most commonly missed point.

The fictitious equation

$\theta(t+1)=\theta(t)+v_\theta(t)$

The core relation is $\theta(t+1)=\theta(t)$ — the equation of a constant quantity, which is right, because constant parameters are what we are estimating. Adding the noise makes it a random walk.

✓

Marginally stable, and that is fine

The appended mode sits at $1$ : simply stable, not asymptotically stable. That is not a defect to be repaired. Chapter 05 showed the filter’s stability lives in $F-\bar KH$ , not in $F$ , and the lecture states plainly that marginal stability here is not a problem for the Kalman filter.

Why the fictitious noise is there — the most missed point in the chapter

The parameter really is constant, so $v_\theta$ is not a claim that it fluctuates. It is a deliberate design device.

Without it, the parameter channel has no process noise, its error covariance collapses towards zero, and the filter concludes it already knows $\theta$ perfectly. It then trusts its initial condition and stops searching — the estimate freezes on whatever was guessed at $t=0$ , no matter what the data says.

Injecting $v_\theta$ keeps $P$ from collapsing, which keeps the gain on that channel alive, which keeps the estimator willing to move. Setting $V_\theta=0$ “because the parameter is constant” is the single most reliable way to get a confident wrong answer.

One knob

The assumptions are the usual ones — $v_\theta\sim\mathrm{WN}(0,V_\theta)$ with $v_1\perp v_\theta$ and $v_2\perp v_\theta$ — and $V_\theta$ is then collapsed, hard:

Everything condensed into one meta-parameter

$V_\theta=\lambda_\theta^{2}\,I$

All the structure of $V_\theta$ becomes a single tuning knob $\lambda_\theta^{2}$ , and it controls a trade-off with two clean ends:

$\lambda_\theta^{2}$	convergence	steady-state variance
large	fast — the estimate climbs to the true value quickly	large — it then rattles around it
small	slow — the estimate creeps up over many samples	small — it settles tightly

why

Which end to pick, and the case that settles it

The right choice depends on the length of the dataset — a short record cannot afford slow convergence — and on whether the parameter is genuinely constant.

If it is time-varying and the filter runs always-on, you want the larger value, because the estimator must keep chasing. The lecture’s example is the mass of fuel in a rocket: that “parameter” drops continuously through the flight, and a filter tuned for a constant would lag it badly.

Two estimates for the price of one — within limits

Run the extended filter and it delivers $\hat x(t|t)$ and $\hat\theta(t|t)$ simultaneously, from the same measurements. That is genuinely free, and it invites over-reach.

Do not ask too much

The number of unknowns you can identify is bounded by the number of things you actually measure. Compare the sensors-versus-unknowns budget:

sensors	states	parameters	ratio	verdict
3	10	15	$3/25$	hopeless
3	5	2	$3/7$	realistic

Extending the state is cheap to write and expensive to afford. The formal version of this check is observability of the extended system — and identifiability is structural, not statistical: if two parameters only ever appear as a product, no quantity of data will separate them. A filter given more unknowns than it can support still runs, and produces confident nonsense.

You extend the state with an unknown constant parameter and set its fictitious process-noise variance to zero, reasoning that the parameter genuinely does not change. What happens?

05 · The consequence

State extension makes the model nonlinear

The trick has a hidden cost, and it is the reason this chapter carries the extended Kalman filter rather than referring to it.

Same move, same consequence, every time: one more state, one more product, one linearisation per step.

Take §02’s mass–spring–damper and change one thing: the friction coefficient $C$ is now unknown. So $\theta=C$ , and state extension promotes it to a third state $x_3(t)=C(t)$ :

The extended mass–spring–damper

\begin{cases} x_1(t+1)=x_1(t)+\Delta x_2(t)+v_{11}(t)\\ x_2(t+1)=-\tfrac{K\Delta}{M}x_1(t)+\Big(1-\tfrac{x_3(t)\,\Delta}{M}\Big)x_2(t)+\tfrac{\Delta}{M}F(t)+v_{12}(t)\\ x_3(t+1)=x_3(t)+v_{13}(t)\\ y(t)=x_1(t)+v_2(t) \end{cases}

with $V_1=\mathrm{diag}\big(\lambda_{11}^{2},\lambda_{12}^{2},\lambda_\theta^{2}\big)$ , $V_2$ from the sensor, and $V_{12}=0$ .

Look at the second equation

It contains the product $x_3(t)\cdot x_2(t)$ — an unknown state multiplied by another unknown state. The system has become nonlinear.

The original plant was perfectly linear. Nothing about the physics changed; the nonlinearity was manufactured by the trick itself, and it is unavoidable, because a parameter that multiplies a state in the original model will multiply that state again once it is promoted.

So gray-box identification via state extension is almost always an EKF problem, never a plain KF one. Answering the exam question with “run a Kalman filter on the extended model” and stopping there is the single most-missed point in the topic.

✓

Two extensions, one pattern

This is the second time in the module that appending states to make a problem tractable has cost something structural. Chapter 04 extended the state with a noise shaping filter and the extended $V_1$ came out singular, which put chapter 05’s second theorem in question. Here the extension costs linearity, which puts all of chapter 05 out of reach at once.

Both are worth remembering as a general lesson: state extension is a cheap thing to write down and its price is always paid somewhere in the hypotheses.

Once that is accepted, the extended model is ready: run the EKF on it and it returns the physical states and the friction coefficient together, online, from position measurements alone.

06 · The alternative

Offline identification by simulation error

There is a second route to the same parameters — more intuitive, entirely classical, and far more expensive.

Set once, and it wants example inputs

“Describe what offline grey-box identification is and make two examples of input signals which can be used for the identification dataset” — 3 marks, 26/07/2024. The second clause is half the marks: an answer that explains batch optimisation and never names a signal is incomplete.

Treat the physical model as a simulator $M(\theta)$ : pick a candidate $\theta$ , drive the model with the recorded input, compare its output against the recorded one, and minimise the simulation-error optimisation index:

The simulation-error index

J_N(\theta)=\frac1N\sum_{t=1}^{N}\Big(y(t)-\hat y\big(t;\theta\big)\Big)^{2}, \qquad \hat\theta_N=\arg\min_\theta J_N(\theta)

where $\hat y(t;\theta)$ is the output of the simulator $M(\theta)$ driven by $u$ — and writing it that way makes the cost visible.

Why this is extremely more expensive than the filter

Every candidate $\theta$ requires a full simulation of the model over the whole dataset, and that simulation sits inside a non-convex outer optimisation which will evaluate many candidates.

The Kalman route gets the same parameter as a by-product of one forward pass over the data. That is the honest comparison, and it is not close.

Choosing the input

Both routes need data that actually contains information about $\theta$ , which is the requirement chapter 02 called persistent excitation: the input must excite every mode the unknown parameters influence, across the band where the model has to be accurate. Otherwise $J_N(\theta)$ is flat in some directions and those parameters are simply not identifiable.

Multi-sine or swept sinusoid (chirp)

A sum of sinusoids, or one whose frequency sweeps the band of interest. Gives precise control over which frequencies get excited and how much energy each receives, and keeps amplitude inside actuator limits — chapter 03’s $A_i\propto1/\sqrt{\omega_i}$ schedule applies directly.

PRBS — pseudo-random binary sequence

A deterministic, repeatable two-level signal with an approximately flat spectrum over a wide band. Excites everything at once, is trivial to generate, drives on/off actuators natively, and delivers maximum energy for a bounded amplitude.

A step is not an identification input

It is the reflex answer and it is wrong. A step excites essentially one region of the spectrum, so it pins down the static gain well and the dynamics badly. Whatever else the experiment does, the plant must also stay inside its linear operating region, the actuators must not saturate, and an unstable plant must be tested inside a stabilising loop — chapter 03’s closed-loop identification.

Deep dive Choosing between the two routes

They are not ranked; they fail differently.

State extension is recursive and cheap, runs online, tracks a drifting parameter, and delivers states and parameters together. It requires the fictitious noise and its tuning knob, it is bounded by the sensors-versus-unknowns budget, and it drags in the EKF with all of §01’s missing guarantees.

Simulation-error optimisation uses the whole dataset at once, needs no fictitious noise, and is not restricted to models a Kalman filter can handle — any simulator will do. It cannot track a drifting parameter, it is not recursive, and it is enormously more expensive.

The practical split: online, drifting, cheap points to state extension; offline, constant, awkward model points to batch optimisation.

2024-07-q52024Q05Gray-box identificationmedium3 pts

Describe what offline grey-box identification is and make two examples of input signals which can be used for the identification dataset.

Load-bearing ideas

The EKF keeps the linear gain structure and recomputes $F(t)$ , $H(t)$ as Jacobians evaluated at $\hat x(t\lvert t-1)$ , the predicted state. It converts a nonlinear time-invariant problem into a linear time-varying one — chapter 04’s extension 4, which was free.
What that costs: no steady state, so no ARE, no $\bar P$ , no asymptotic theorems; stability can only be validated empirically; and every step recomputes four matrices. Used widely anyway, warily in safety- and mission-critical settings.
Designing a filter for a real plant is four steps: white-box ODE in continuous time → state space → Euler discretisation → add noise, then choose $V_{12}=0$ , $V_2$ from a datasheet, and $V_1$ diagonal with its entries tuned (today, from data — a machine-learning step inside a model-based method).
Gray box means the structure is fully known and a few physically meaningful parameters are not — distinct from black box (structure unknown, parameters meaningless) and white box (all known).
State extension: append $\theta(t+1)=\theta(t)+v_\theta(t)$ and stack $\theta$ under $x$ . The parameter becomes a state and the ordinary machinery estimates it.
The fictitious noise is a design device, not a modelling claim. Without it $P$ collapses, the gain on the parameter channel dies, and the estimate freezes on its initial guess. $V_\theta$ collapses to one knob $\lambda_\theta^{2}$ : large = fast and noisy, small = slow and clean; large is right for a genuinely drifting parameter.
Budget the unknowns against the sensors. More parameters than the measurements support gives a filter that runs and lies. Identifiability is structural — parameters appearing only as a product are never separable.
The catch: extending the state multiplies unknown by unknown ( $x_3x_2$ ), so a linear plant becomes nonlinear and the EKF is required. Same pattern as chapter 04’s coloured-noise extension, which cost a singular $V_1$ .
The alternative is batch simulation-error optimisation: intuitive, no fictitious noise, any simulator — but one full simulation per candidate $\theta$ inside a non-convex search, and no tracking. Inputs must be persistently exciting: multi-sine/chirp or PRBS, never a step.

Exam radar. Two 3-mark theory stems, three sittings, nothing since July 2024 — the module’s fading topic, and cheap to secure. For the first stem, be able to go: gray-box setting → state extension → why the fictitious noise exists → $\lambda_\theta^{2}$ trade-off → and the extended model is nonlinear, so EKF. That last clause is what most answers omit. For the second, define offline batch optimisation against the recursive route, then name two concrete inputs and say what persistent excitation buys.