Chapter 04

Frequency Analysis & Spectral Factorization

The frequency-domain view of a stochastic process: the power spectral density and its properties, the master formula that turns a filter into a spectrum, the model-zoo spectra, why the periodogram never converges, and the spectral factorization that puts a process into the canonical form prediction needs.

Reading: ~34 min Interactive: 1 widgets Source: Polimi MIDA1 2025/26 — Lecture 5 (Frequency analysis) · Polimi MIDA1 2025/26 — Lecture 6 (Estimation of process characteristics) · Polimi MIDA1 2025/26 — Lecture 7 (Spectral factorization)

01 · Definition

The spectrum: definition and properties

The power spectral density is the covariance seen in the frequency domain — it says how a process’s variability is distributed across frequencies, and it carries exactly the same information as $\gamma(\tau)$ .

The power spectral density (PSD, or spectrum) of a zero-mean stationary process is the Fourier transform of its correlation function:

Spectrum

$\Gamma(\omega) = \mathcal{F}[\tilde\gamma(\tau)] = \sum_{\tau=-\infty}^{\infty} \tilde\gamma(\tau)\,e^{-j\omega\tau}, \qquad \tilde\gamma(\tau) = \frac{1}{2\pi}\int_{-\pi}^{\pi}\Gamma(\omega)\,e^{j\omega\tau}\,d\omega.$

The transform pair is one-to-one, so $\gamma$ and $\Gamma$ are two encodings of the same process. Four properties fall straight out of the definition and matter on every exam:

Real & even

Since $\gamma$ is even, $\Gamma(\omega)=\gamma(0)+2\sum_{\tau\ge1}\gamma(\tau)\cos(\omega\tau)$ — a real, even function.

Periodic

$2\pi$ -periodic (it is the transform of a discrete sequence); we plot it on $[-\pi,\pi]$ , with $\omega_{\max}=\pi$ .

Non-negative

$\Gamma(\omega)\ge 0$ for all $\omega$ — a power density cannot be negative.

Area = variance

$\gamma(0)=\frac{1}{2\pi}\int_{-\pi}^{\pi}\Gamma(\omega)\,d\omega$ — the total area is the variance; the area in a band is the variance carried by those frequencies.

It is often easier to compute the complex spectrum $\Phi(z)=\sum_\tau\gamma(\tau)z^{-\tau}$ and then set $z=e^{j\omega}$ : $\Gamma(\omega)=\Phi(e^{j\omega})$ . The spectrum of white noise is flat, $\Gamma(\omega)=\lambda^2$ — every frequency contributes equally, the frequency-domain face of “completely unpredictable”.

Defined and its properties — a recurring open question

“Define the power spectral density and discuss its main properties” is asked as a standalone theory question (e.g. June 2026, Problem 4.2), and its computation is the last part of every Problem 1. Learn the definition, the transform pair, and the four properties as a block — the derivation $\Gamma=\gamma(0)+2\sum\gamma(\tau)\cos(\omega\tau)$ is worth writing out.

2026-06-q42026Power spectral densitymedium5 pts

Define the power spectral density of a stationary stochastic process and discuss its main properties.

02 · Master formula

Spectrum of a filtered process

Here is the formula that makes the whole model zoo computable. Push a process $u$ with spectrum $\Gamma_{uu}$ through a stable filter $W(z)$ ; the output spectrum is

Filtering formula

$\Gamma_{yy}(\omega) = \big\lvert W(e^{j\omega})\big\rvert^2\,\Gamma_{uu}(\omega).$

In particular, for a white-noise input $\Gamma_{uu}=\lambda^2$ , the output spectrum is $\lvert W(e^{j\omega})\rvert^2\lambda^2$ — a rational spectrum, and this proves it is non-negative. Applying it to the model zoo:

MA(1)

$W=1+cz^{-1}$ , so $\Gamma(\omega)=(1+c^2+2c\cos\omega)\lambda^2$ : low frequencies dominate for $c>0$ , high for $c<0$ .

AR(1)

$W=1/(1-az^{-1})$ , so $\Gamma(\omega)=\dfrac{\lambda^2}{1+a^2-2a\cos\omega}$ : a peak at low frequency for $a>0$ , at $\omega=\pi$ for $a<0$ .

ARMA(1,1)

$\Gamma(\omega)=\lambda^2\dfrac{1+c^2+2c\cos\omega}{1+a^2-2a\cos\omega}$ — numerator and denominator shapes multiply.

Zero-blocking, and the DC vs variance mix-up

Two staples: (1) An MA zero blocks a frequency. For $v=\eta(t)+\eta(t-2)$ , $W=1+z^{-2}$ vanishes at $\omega=\pm\pi/2$ , so $\Gamma(\pm\pi/2)=0$ — that band is filtered out entirely. (2) At $\omega=0$ the AR(1) spectrum is $\lambda^2/(1-a)^2$ (a squared DC gain), not the variance $\lambda^2/(1-a^2)$ . Confusing $\Gamma(0)$ with $\gamma(0)$ is the classic slip in the last part of Problem 1.

Purely deterministic components show up as impulses in the spectrum: a constant gives $\lambda^2\delta(\omega)$ , an alternating sequence $\lambda^2\delta(\omega-\pi)$ , and a sinusoid a pair $\tfrac{\lambda^2}{2}[\delta(\omega-\omega_0)+\delta(\omega+\omega_0)]$ . A mixed process’s spectrum is the sum of a continuous part (non-deterministic) and spikes (deterministic).

03 · From data

Estimating the spectrum: the periodogram

In practice you have data, not $\gamma$ . The natural nonparametric estimate is the periodogram, the squared magnitude of the data’s Fourier transform:

Periodogram

$\hat\Gamma_N(\omega) = \frac{1}{N}\left\lvert \sum_{t=1}^{N} v(t)\,e^{-j\omega t}\right\rvert^2.$

It has a notorious flaw:

✓

The periodogram is inconsistent

$\hat\Gamma_N$ is asymptotically unbiased but not consistent — its variance does not shrink as $N\to\infty$ . Collecting more data makes the estimate no less noisy, just more finely sampled in frequency. This is the pedagogical punchline of the estimation lecture.

The fixes trade resolution for variance: Bartlett’s method averages the periodograms of $K$ disjoint segments (variance drops by roughly $1/K$ ), and the windowing method smooths the correlogram with a lag window that down-weights the poorly-estimated high-lag covariances. Estimating $\mu$ and $\gamma$ themselves is gentler: the sample mean $\hat\mu_N=\frac1N\sum v(t)$ and sample covariance $\hat\gamma_N(\tau)=\frac1N\sum(v(t)-\hat\mu)(v(t+\tau)-\hat\mu)$ are consistent for ARMA processes (the biased $1/N$ form is preferred — lower variance and guaranteed PSD).

2023-07-q42023Nonparametric spectral estimationmedium5 pts

Discuss nonparametric estimation of a process's power spectral density from data — the periodogram, its limitation, and how to fix it.

04 · Canonical form

Spectral factorization

The filter that generates a given spectrum is not unique: multiplying $W(z)$ by a constant, a delay $z^{-n}$ , or an all-pass factor, or reflecting a pole/zero across the unit circle, all leave $\gamma$ (and $\Gamma$ ) unchanged. Prediction needs a single, well-defined representative — the canonical one.

key

The spectral factorization theorem

Every stationary process with a rational spectrum has a unique $W^*(z)=C(z)/A(z)$ such that: (1) $C,A$ are monic; (2) they have the same degree; (3) they are coprime (no shared roots); and (4) $C$ has all roots in the closed unit disk and $A$ in the open unit disk. $W^*$ is the canonical spectral factor, and $(W^*,\ \eta\sim\mathrm{WN}(0,\lambda^2))$ is the canonical representation.

The practical move you will use constantly: if a zero (or pole) lies outside the unit circle, reflect it to its reciprocal and rescale the noise variance so that $\gamma$ is preserved. That turns any representation into the canonical one — and, as the next chapter shows, only the canonical form yields a stationary predictor.

For the AR(1) x(t)=0.5·x(t−1)+e(t) with Var(e)=1, what is the spectrum at ω=0, Γ_x(0)?

Load-bearing ideas

The PSD $\Gamma(\omega)=\mathcal{F}[\gamma]$ is real, even, $2\pi$ -periodic, non-negative, and its area is the variance. Compute it via the complex spectrum $\Phi(z)$ evaluated at $z=e^{j\omega}$ .
Master formula: $\Gamma_{yy}(\omega)=\lvert W(e^{j\omega})\rvert^2\Gamma_{uu}(\omega)$ ; for white input, $\lvert W\rvert^2\lambda^2$ . MA zeros block frequencies; AR poles peak them.
Watch $\Gamma(0)=\lambda^2/(1-a)^2$ vs the variance $\gamma(0)=\lambda^2/(1-a^2)$ — different objects.
The periodogram is unbiased but inconsistent; Bartlett averaging and windowing trade resolution for variance.
Spectral factorization gives the unique canonical $C/A$ (monic, equal degree, coprime, roots inside/on the unit circle). Reflect out-of-circle singularities and rescale $\lambda^2$ — this is the precondition for prediction.

Exam radar. The spectrum computation closes Problem 1; the “define the PSD + properties” theory recurs; and canonicalisation is the gateway to Problem 2. Practise: given a process, get $\Gamma(\omega)$ via $\lvert W\rvert^2\lambda^2$ , evaluate it at $\omega=0$ or $\pi$ , and canonicalise a non-canonical $C/A$ .