Chapter 09

Time-Series Analysis & Practical Aspects

The dedicated time-series toolkit and the engineering that makes identification work in practice: Yule–Walker and the Durbin–Levinson recursion, the PARCOR function that reads off AR order, differencing a non-stationary series into an ARIMA model, and designing an informative experiment (input richness, sampling time, pre-filtering).

Reading: ~24 min Interactive: 1 widgets Source: Polimi MIDA1 2025/26 — Lecture 15 (Time-series analysis) · Polimi MIDA1 2025/26 — Lecture 14 (Practical aspects of identification)

01 · Toolkit

Order identification: Yule–Walker, Durbin–Levinson, PARCOR

For pure time series (no exogenous input) there is a dedicated toolkit for reading the model order straight off the data’s correlations.

Whichever one dies abruptly names the family; where it dies names the order.

The Yule–Walker estimation of an AR( $n$ ) solves the $n\times n$ Toeplitz system $\gamma(\tau)=\sum_i a_i\gamma(\tau-i)$ for the coefficients from the sample covariances — the right-to-left reading of Yule–Walker from Chapter 3, and asymptotically equivalent to least squares. Solving it order by order is the Durbin–Levinson recursion: it upgrades the order- $n$ solution to order $n+1$ without re-inverting the matrix, producing at each step a reflection coefficient. That last coefficient is the PARCOR (partial autocorrelation) at lag $n$ — and it is the order-identification tool, the dual of the covariance cut-off:

MA(n): γ cuts off

The autocovariance is exactly zero beyond lag $n$ ; the PARCOR tails off. Reads MA order.

AR(n): PARCOR cuts off

The PARCOR is exactly zero beyond lag $n$ ; the covariance tails off. Reads AR order.

ARMA: both tail off

Neither cuts off sharply — a sign you need both an AR and an MA part.

key

The order-identification table

Two fingerprints, read together, name the model and its order: $\gamma$ cutting off at lag $n$ ⇒ MA( $n$ ); PARCOR cutting off at lag $n$ ⇒ AR( $n$ ); both merely decaying ⇒ ARMA. It is the practical counterpart to the covariance fingerprints of Chapter 3.

02 · Non-stationary

Differencing & ARIMA models

Real series often drift — their mean is not constant, so they are not stationary and the machinery so far does not apply directly. The remedy is differencing: model the changes rather than the levels. A random walk $w(t)=w(t-1)+\eta(t)$ has a variance that grows without bound ( $\mathrm{Var}[w(t)]=\lambda^2 t$ ), but its first difference $w(t)-w(t-1)=\eta(t)$ is white and stationary.

Building this into the model gives the ARIMA family (also written CARIMA/ARIMAX with an input): an ARMA model on the differenced series. If a monthly economic series is non-stationary in the mean, differencing once often yields a stationary series well described by, say, an IMA(1,1) — an integrated moving average. The workflow: difference until stationary, identify an ARMA on the result, then integrate back to forecast the levels.

03 · Practice

Designing an experiment that identifies the system

The best algorithm cannot rescue an uninformative experiment. Three practical levers decide whether the data actually pin down the model:

Input richness

The input must persistently excite all modes. A PRBS approximates white noise’s flat spectrum and is the standard choice; a step or impulse can leave parameters biased regardless of $N$ .

Sampling time

Too fast pushes poles toward the unit circle (numerical trouble); too slow aliases and loses dynamics. Choose $T_s$ for the bandwidth you care about.

Pre-filtering & outliers

Pre-filter to emphasise the frequency band that matters for the model’s purpose; guard against outliers that a squared cost over-weights.

✓

Noise is good — but excitation is essential

Recall from Chapter 7 that noise helps identification by exciting the dynamics. The failure mode is not too much noise but too little excitation from the input: if $u$ never moves a mode, no amount of data identifies it. Experiment design is where identifiability is won or lost.

experiment-design-excitationExperiment design & persistent excitationmedium5 pts

Why does the choice of input signal matter in system identification? Discuss persistent excitation, why "noise is good for identification", and the role of signals such as the PRBS.

A time series has an autocovariance that decays geometrically but never hits zero, while its PARCOR is essentially zero beyond lag 2. What model does this point to?

Load-bearing ideas

Yule–Walker estimates AR coefficients from sample covariances; Durbin–Levinson solves it order by order, yielding the PARCOR reflection coefficients.
Order-identification table: $\gamma$ cuts off ⇒ MA( $n$ ), PARCOR cuts off ⇒ AR( $n$ ), both tail off ⇒ ARMA.
Differencing turns a drifting (non-stationary) series into a stationary one; modelling the difference gives ARIMA/IMA.
Experiment design decides identifiability: persistent excitation (PRBS), a sensible sampling time, pre-filtering, and outlier care.

Exam radar. This material surfaces occasionally (Yule–Walker for AR(1), September 2025) rather than every session, but the γ/PARCOR table and the “why persistent excitation matters” argument are the most likely to be asked. Weight your revision accordingly — the heavy marks are in Chapters 2, 5, 6 and 7.