The Kalman Filter

01 · Motivation

Deleting a sensor and keeping the signal

A plant has $n$ internal states and $p$ measured outputs, and almost always $n\gg p$ — for one reason, which is that physical sensors cost money. Software sensing is the business of getting the missing $n-p$ signals anyway.

The states you cannot afford to measure are exactly the ones you want. They feed the control algorithm, and they feed monitoring: fault detection and predictive maintenance both need to watch quantities that no instrument is reading.

There are two ways out, and choosing between them is an economics problem before it is a control problem. Adding a physical sensor costs a fixed amount per unit produced. Developing a software sensing algorithm costs a large amount once and nothing thereafter. Break-even sits where the development cost equals $N$ times the unit hardware cost, so:

That is the clean version. In practice three things bend it. First, redundancy: safety-critical aerospace and automotive systems run both, at low volume, precisely because two independent estimates of the same quantity are worth more than either. Second, some sensors are not merely expensive but infeasible — nothing can be installed where the signal lives. Third, hardware carries costs that do not appear on the unit price: installation complexity, space and weight, energy consumption. The net trend across industry is steadily towards more software sensors and fewer physical ones.

The two questions every software-sensing design must answer

Before any algorithm, two questions decide whether the project is possible and whether it is worth doing.

1. Is it feasible? Only if the state can be reconstructed from the output at all — which is exactly the observability test from chapter 01. That rank condition stops being a classification exercise here and becomes a go/no-go gate.
2. How good will it be? The estimation error $\operatorname{var}\big[x(t)-\hat x(t)\big]$ is the software sensor’s noise specification — the direct analogue of the noise figure printed on a physical sensor’s datasheet. §05 shows that this number is computable in advance.

What the Kalman filter is asked to deliver

The chapter’s model-based approach sets out three problems and one bonus:

Problem 2 is where input/output models run out. An ARMAX model has no state to predict; it describes the plant from the outside. Getting at $x$ requires the internal description, which is why the whole of this chapter is written in state space.

02 · The model

The basic system and its two noises

Everything that follows is derived for the simplest useful case — no exogenous input, linear, time-invariant — and then extended. The extensions in §06 turn out to be nearly free; the real content is here.

The basic system is a state-space model driven by two noises instead of an input:

with $x\in\mathbb R^{n}$, $y\in\mathbb R^{p}$ and — once §06 restores the input — $u\in\mathbb R^{m}$. The SISO special case $p=m=1$ is what every exam question uses, and it is worth noticing early that the general formulas below never assume it. Dropping $Gu(t)$ makes the input/output pair a pure time series; §06 puts the input back at no cost.

v₁ — process noise

Also called state noise or model noise. An $n$-component vector white noise, $v_1\sim\mathrm{WN}(0,V_1)$. It represents internal disturbances and small modelling errors — the model is never exact, and $v_1$ is where that inexactness is booked.

v₂ — measurement noise

Also called output error or sensor error. A $p$-component vector white noise, $v_2\sim\mathrm{WN}(0,V_2)$. It models the noise on the physical sensors, which is why $V_2$ is usually the one you can look up.

V₁ — process noise covariance

$n\times n$, symmetric, semi-definite positive ($V_1\ge0$). Semi-definite is enough: some state directions may be driven by no noise at all.

V₂ — measurement noise covariance

$p\times p$, symmetric, and strictly definite positive ($V_2>0$). The strictness is not cosmetic — §03 shows it is what guarantees the filter exists.

V₁₂ — cross-covariance

$n\times p$. Non-zero only at lag zero. In practice it is usually $0$, but several formulas below are stated in the general case precisely because the exam is not.

Each noise carries the three standard white-noise properties — zero mean, the covariance above at lag zero, and zero correlation at every other lag. The cross-correlation is assumed to follow the same pattern:

Because the system is dynamic, it also needs an initial condition — and being stochastic, that condition is itself probabilistic: $\mathbb E[x(1)]=X_0$ and $\operatorname{var}\big[x(1)-X_0\big]=P_0$. The special case $P_0=0$ says the initial state is known exactly. Finally, both noises are assumed uncorrelated with $x(1)$, a technical assumption that the optimality proof needs.

Reading the covariances off a system

Exam systems never hand you $V_1$, $V_2$ and $V_{12}$ directly. They hand you a model written in terms of named scalar noises and let you assemble the matrices — and they arrange for one noise to appear in both equations, so that the cross-covariance cannot be zero.

03 · The recursion

The filter, and the equation that feeds it

The Kalman filter is four equations plus a recursion for a matrix. Three of the four are obvious. The fourth — the gain — is Kalman’s entire contribution, and the recursion that produces it is the only hard object in the chapter.

Two dynamic equations means two initial conditions: $\hat x(1|0)=X_0$ and $P(1)=P_0$ — the two pieces of prior knowledge from §02, arriving exactly where they are needed.

The mnemonic that rebuilds both formulas

The gain and the DRE look unmemorable until you notice they are built from three repeated blocks, each of the shape (matrix) $P(t)$ (matrix) $+$ (noise covariance):

The DRE runs without data

Written that way, one thing is conspicuous: there is no input. The DRE is a non-linear matrix difference equation that is autonomous — its trajectory depends on $F$, $H$, $V_1$, $V_2$, $V_{12}$ and $P_0$, and on nothing that is measured. Two consequences follow, and both matter:

• $P(t)$ and $K(t)$ can be computed entirely offline, before the plant is switched on. The online filter is then three cheap equations with a pre-computed gain schedule.
• The accuracy of the estimate is known in advance. §05 makes this precise.

The equation is named for Jacopo Riccati, who studied its continuous scalar ancestor in the early 1700s; Kalman rediscovered and used the matrix difference version in 1960.

Why $V_2>0$ was worth insisting on

The gain inverts $\big(HP(t)H^{\top}+V_2\big)$, so the filter exists only if that matrix is invertible at every step. The argument is short:

• $HP(t)H^{\top}$ is semi-definite positive by construction, for any $P(t)\ge0$. On its own it may well be singular.
• $V_2>0$ is definite positive, by assumption.
• Semi-definite plus definite is definite, and definite positive matrices are invertible. ∎

That is the whole reason §02 demanded strictness for $V_2$ and allowed slackness for $V_1$: a state direction with no noise is harmless, an output direction with no noise is not.

What the structure is doing

The block diagram is worth holding in your head, because every extension in §06 and §07 is a modification of it. The filter contains a replica of the plant — a digital twin running the same $F$ and $H$, differing only in that it cannot see the two noises. It compares the real measurement $y(t)$ against its own prediction $\hat y(t|t-1)$, and feeds the difference back into its state equation through $K(t)$.

The plant’s two noise arrows are exactly what the filter lacks; the innovation feedback is what it has instead.

This idea long predates Kalman: a replica with output-error feedback is the classical state observer. Kalman’s contribution was narrower and much deeper — a closed-form optimal choice of the correction gain. Without it you are guessing, and the failure modes are symmetric:

• $K(t)$ too small — the filter under-exploits the information in $e(t)$ and trusts a model it should be correcting.
• $K(t)$ too big — the filter over-amplifies measurement noise, and can go unstable outright.

Nor can it be tuned by hand, because $K(t)$ is not a knob. It is an $n\times p$ matrix: for a modest $n=10$, $p=3$ plant that is 30 numbers, all time-varying, all coupled.

04 · The distinction that is graded

Predictor, filter, and the step between them

$\hat x(t+1\lvert t)$ and $\hat x(t\lvert t)$ both use every measurement up to time $t$. They differ in which state they estimate — tomorrow’s or today’s — and converting between them is one multiplication. The exam asks for that conversion more often than for anything else in chapter 2.3.

What each estimate is

The notation $\hat x(a|b)$ means: estimate of the state at time $a$, using measurements up to time $b$. Both objects below use the same data.

Same information, different target instant — which is why one multiplication converts between them.

Route A — invert $F$

The filter’s own state equation is $\hat x(t+1|t)=F\hat x(t|t)$: to predict tomorrow from today’s best estimate, push it through the model — the noise contributes nothing because it is zero-mean. Reading that backwards gives the conversion:

One matrix inverse, applied to the whole predictor equation. That last point is where marks are lost; see the mistake callout below.

Route B — the filter form of the recursion

When $F$ is singular, route A is unavailable and the filter has to be generated directly. It has its own gain:

with $\hat y(t|t-1)$ and $e(t)$ unchanged — the innovation is still built from the prediction, in both forms — and the DRE identical. Nothing about $P(t)$ changes; only how its output is used.

The one-symbol difference

Set $V_{12}=0$ and put the two gains side by side:

They differ by exactly the leading $F$, which is to say $K(t)=F\,K_0(t)$. This is not a coincidence — it is route A’s $F^{-1}$ seen from the other side. It is also the single most testable line in the chapter, and a one-symbol slip changes the answer without changing its shape.

Worked example — the exam’s own format

05 · What you get for free

An accuracy figure you can compute before any data

The Riccati recursion was introduced as machinery for producing the gain. Its solution turns out to be the answer to the question §01 said every software-sensing project must answer second.

$P(t)$ is the covariance matrix of the one-step state prediction error — $n\times n$, symmetric, semi-definite positive. Its diagonal entries are the variances of the individual state estimates; its off-diagonal entries say how those errors are correlated.

Now combine that with the fact from §03 that the DRE is autonomous:

The output prediction error variance

$P(t)$ measures the error on the state. The exam sometimes wants the error on the output, which is not the same thing and is a three-line derivation away.

06 · Extensions 1, 3 and 4

Many steps ahead, a known input, a drifting plant

The basic case was deliberately stripped down. Putting the missing pieces back costs almost nothing — which is a statement about the structure of the derivation, not about luck.

The lecture numbers five extensions. Extension 2 — the filter — was promoted to §04 because it carries the exam weight, and extension 5 — the nonlinear case — is deferred to chapter 07, where the Extended Kalman Filter is the machinery behind gray-box identification. The remaining three are here.

Extension 1 — prediction $k$ steps ahead

Once $\hat x(t+1|t)$ is available, going further needs no new data, so it needs no new correction: just iterate the model.

Extension 3 — an exogenous input $Gu(t)$

The estimate equations pick up the input term in the obvious place, and then:

Extension 4 — a time-varying plant

For $x(t+1)=F(t)x(t)+G(t)u(t)+v_1(t)$ with $y(t)=H(t)x(t)+v_2(t)$, the Kalman equations are exactly the same — replace $F$, $G$, $H$ by $F(t)$, $G(t)$, $H(t)$ wherever they appear. The typical physical cause is ageing: plant dynamics that drift slowly over the equipment’s life.

This extension is also the door to the Extended Kalman Filter in chapter 07, whose whole strategy is to approximate a nonlinear time-invariant system by a linear time-varying one — precisely so that this extension applies.

07 · When the assumption fails

Coloured noise and the model-extension trick

Every formula so far assumed $v_1$ and $v_2$ are white. Real disturbances frequently are not — a sensor’s own noise is often visibly correlated in time. The repair does not modify the filter; it modifies the model until the filter applies again.

Why whiteness was load-bearing

The Kalman derivation assumes the innovation is white. If $v_1$ is correlated in time, then part of “the part the model failed to anticipate” was in fact predictable from past innovations — the orthogonality argument behind the DRE fails, and the filter systematically mis-weights the model against the measurement. It still runs; it is simply no longer optimal, and worse, $P(t)$ no longer reports the true error covariance, so its own accuracy claim becomes a lie.

The trick

A coloured noise is, by definition, white noise pushed through a filter. So model that filter explicitly, append its state to the plant’s, and the augmented system is driven by white noise once more. The standard Kalman filter then applies to the extended model unchanged — you are including the noise dynamics into the system dynamics.

The first-order example the question asks for

Take a first-order plant whose process noise is an AR(1) process:

with $e\sim\mathrm{WN}(0,1)$, $v_2\sim\mathrm{WN}(0,1)$ and $v_2\perp e$. The Kalman formulas cannot be applied as they stand, because $\eta$ is not white.

Expand the noise model back into the time domain: $\eta(t+1)=c\,\eta(t)+e(t+1)$. Define $v(t)=e(t+1)$, which is still white with the same variance, and the shaping filter becomes a first-order state equation, $\eta(t+1)=c\,\eta(t)+v(t)$.

Now promote $\eta$ to a state. With $x_1=x$ and $x_2=\eta$:

which in standard form is

The driving noise is white again, so the standard Kalman filter applies. Note where the coupling went: the $1$ in the top-right corner of $F_E$ is the coloured noise entering the plant, now a perfectly ordinary state interaction.

The same construction handles a coloured measurement noise, and — with unknown model parameters appended as extra states instead of a noise filter — it is exactly the mechanism behind gray-box identification in chapter 07. Learning the trick once pays three times.

08 · In practice

The Kalman filter in MATLAB

Two functions cover the whole chapter: kalman for the filter and idare for the steady-state Riccati solution. Both are stated in a different parameterisation from the course’s, and both have a trap in the translation.

No archived question has yet asked specifically for kalman or idare. But MATLAB questions appear in 8 of the last 9 papers (see the exam analysis), the lecturer states in the exercise sessions that the exam will contain simple MATLAB questions, and these are the only two functions in the course that touch chapter 2.3 — so the cost of learning them is three lines and the downside of not is a whole question.

The two parameterisations

MATLAB does not write the noises the way the course does. It routes a single process noise $w$ through matrices into both equations:

Matching $v_1=\tilde Gw$ and $v_2=\tilde Hw+v$ gives the general translation:

Nobody wants to compute that. The trick is to choose the routing matrices so it collapses: set $\tilde G=I$ and $\tilde H=0$, and the three equivalences become simply $V_1=Q$, $V_2=R$, $V_{12}=N$ — the course’s covariances pass straight through.

That choice is implemented by declaring the noise as an extra bank of inputs:

% dt = sampling TIME; n states, p outputs
SYS = ss(F, [G eye(n)], H, [D zeros(p,n)], dt);

[KALMAN_SYS, K, P] = kalman(SYS, V1, V2, V12, 'current');   % 'current' -> FILTER
                                                            % 'delayed' -> PREDICTOR

SYS and KALMAN_SYS are both state-space objects; K and P are the asymptotic gain and Riccati solution.

The algebraic Riccati equation — idare

For the steady-state solution of chapter 05, MATLAB provides the implicit discrete algebraic Riccati equation solver. It is stated for the dual control problem, so the plant matrices go in transposed:

[P, Ktilde, EIGS] = idare(F', H', V1, V2, V12, eye(n));
K = Ktilde';        % <-- transpose back
% EIGS already contains eig(F - K*H)