Model Identification and Data Analysis — Module 1 — Practice

Explain and discuss the main results of the asymptotic analysis of PEM methods.

Explain how you would approach the identification of a dynamical model — what determines whether you can use least squares or need an iterative method, and what the workflow is.

Define the power spectral density of a stationary stochastic process and discuss its main properties.

Explain the whiteness test (Anderson test) for model validation — what it checks, the statistic used, and the confidence band.

A canonical process gives the two-step-ahead predictor filter with quotient $1+\tfrac34 z^{-1}$ (so $w_0=1,\ w_1=\tfrac34$) and $e(\cdot)\sim\mathrm{WN}(0,1)$. Compute the variance of the two-step-ahead prediction error. Give the decimal value.

Consider $x(t)=\tfrac12 x(t-1)+e(t)$ with $e(\cdot)\sim\mathrm{WN}(0,1)$, and $y(t)=x(t)+\eta(t)$ with $\eta(\cdot)\sim\mathrm{WN}(0,3)$, $e\perp\eta$. (a) Is $y(t)$ stationary? (b) Compute $E[y(t)]$. (c) Compute $\gamma_x(0)$. (d) Compute $\gamma_y(1)$. (e) Compute the power spectral density $\Gamma_y(\omega)$ at $\omega=0$.

Consider the stochastic process generated by the system $S$: $$x(t)=\tfrac13 x(t-1)+e(t),\qquad y(t)=x(t)+x(t-1)+\eta(t)$$ with $e(\cdot)\sim\mathrm{WN}(2,2)$, $\eta(\cdot)\sim\mathrm{WN}(-1,1)$, and $e\perp\eta$. (a) Is $y(t)$ stationary? (b) Compute $E[y(t)]$. (c) Compute $\gamma_x(0)$. (d) Compute $\gamma_y(1)$. (e) Compute the power spectral density $\Gamma_y(\omega)$ at $\omega=\pi$.

Consider $y(t)=\tfrac12 y(t-1)-\tfrac18 y(t-2)+e(t)+\tfrac14 e(t-1)$, $e(\cdot)\sim\mathrm{WN}(0,1)$, i.e. $A(z)y(t)=C(z)e(t)$ with $A(z)=1-\tfrac12 z^{-1}+\tfrac18 z^{-2}$, $C(z)=1+\tfrac14 z^{-1}$. (a) Show the process is in canonical form. (b) Perform the long division of $C/A$ for two steps, writing $\tfrac{C(z)}{A(z)}=1+\alpha z^{-1}+z^{-2}\tfrac{\beta+\gamma z^{-1}}{A(z)}$. (c) Give the optimal two-step-ahead predictor. (d) Compute the two-step prediction-error variance.

The data are generated by an MA(1) $S:\;y(t)=e(t)+\tfrac12 e(t-1)$, $e\sim\mathrm{WN}(0,1)$, and fitted with $M:\;y(t)=a\,y(t-1)+\xi(t)$. (a) Compute $\gamma(0),\gamma(1)$. (b) Find the PEM estimate $\hat a$. (c) Find $\hat\lambda^2$.

The data are generated by $S:\;y(t)=e(t)+e(t-1)-\tfrac12 e(t-2)$, $e(\cdot)\sim\mathrm{WN}(0,1)$ (an MA(2)), and are fitted with the model class $M:\;y(t)=a\,y(t-1)+b\,y(t-2)+\xi(t)$, $\xi\sim\mathrm{WN}(0,\lambda^2)$. (a) Compute $\gamma(0),\gamma(1),\gamma(2)$ of the true process. (b) Write the PEM criterion $J(a,b)$. (c) Find the PEM estimates $\hat a,\hat b$. (d) Find the estimated noise variance $\hat\lambda^2$.

In ARMA modelling, describe the cross-validation approach to model selection and compare it with alternatives (whiteness test, information criteria).

Explain the least-squares estimation procedure for a linear regression model, including the normal equations, when the solution is unique, and its statistical properties.

A zero-mean stationary process has autocovariance $\gamma(\tau)\ne 0$ only for $\tau=0$ and $\tau=\pm1$, and $\gamma(\tau)=0$ for $|\tau|\ge 2$. Which model class does it belong to?

Discuss the concept of a stationary stochastic process, then elaborate on the different ways it can be equivalently represented.

Explain in detail the methodology for identifying a model whose noise enters through a moving-average (i.e. ARMAX / MAX), and why least squares does not suffice.

Write the Yule-Walker equations for an AR(1) process and explain what they represent.

The true system is $S:\;y(t)=-\tfrac12 y(t-1)+e(t)$, $e\sim\mathrm{WN}(0,1)$. (a) Compute $\gamma(\tau)$ for $\tau=0,1,2$. (b) How would (a) change if $E[e(t)]=1$? (c) With model class $M_1:\;y(t)=a\,y(t-2)+\xi(t)$, find $\hat a$ by PEM. (d) With $M_2:\;y(t)=-\tfrac12 y(t-1)+\xi(t)+b\,\xi(t-1)$, find $\hat b$ by PEM. (e) Which model is best? (f) Interpret via the transfer functions.

Prove qualitatively that when the model set contains the true system dynamics, the PEM estimate converges to the true system, and explain what happens otherwise.

When data are scarce and cross-validation is impractical, discuss at least two criteria for choosing the model order.

A process $y(t)$ is generated by the scheme: white noise $e(\cdot)\sim\mathrm{WN}(0,1)$ is filtered by $\dfrac{1+5z^{-1}}{1+\tfrac15 z^{-1}}$ to give $\eta(t)$; then $\delta(t)=\eta(t)+w(t)$ with $w(\cdot)\sim\mathrm{WN}(0,1)$ and $e\perp w$; finally $y(t)=\dfrac{1}{1+\tfrac12 z^{-1}}\,\delta(t)$. (a) Is $y(t)$ stationary? (b) Write $y(t)$ in canonical form. (c) Compute the optimal two-step-ahead predictor $\hat y(t+2\mid t)$. (d) Compute the variance of the two-step prediction error.

Consider the model class $M:\;y(t)=V\,y(t-1)+u(t-1)+e(t)$, $e\sim\mathrm{WN}(0,\lambda^2)$, $V\in\mathbb{R}$, with measured outputs $y(1)=0.1,\ y(2)=-0.4,\ y(3)=2,\ y(4)=-1$. (a) For which $V$ is the process stationary (for stationary $u$)? (b) Write the time-domain one-step-ahead predictor. (c) Identify $V$ by least squares with $u(t)=0$. (d) Would the answer change if $u(t)=\sin(3t)$?

Discuss nonparametric estimation of a process's power spectral density from data — the periodogram, its limitation, and how to fix it.

For the AR(1) process $x(t)=\tfrac12 x(t-1)+e(t)$ with $e(\cdot)\sim\mathrm{WN}(0,1)$, compute the variance $\gamma_x(0)$. Give the decimal value.

Consider the ARMA(1,1) process $y(t)=\tfrac12 y(t-1)+e(t)+\tfrac14 e(t-1)$, $e(\cdot)\sim\mathrm{WN}(0,1)$, i.e. $A(z)=1-\tfrac12 z^{-1}$, $C(z)=1+\tfrac14 z^{-1}$. (a) Show the process is in canonical form. (b) Derive the optimal one-step-ahead predictor $\hat y(t\mid t-1)$. (c) Give the variance of the one-step prediction error.

Why does the choice of input signal matter in system identification? Discuss persistent excitation, why "noise is good for identification", and the role of signals such as the PRBS.

Describe the four ingredients of a parametric identification problem (system, model, method, experiment) and explain why a white-noise residual signals an optimal predictor.

For the MA(2) process $y(t)=e(t)+e(t-1)-\tfrac12 e(t-2)$ with $e(\cdot)\sim\mathrm{WN}(0,1)$, compute the variance $\gamma(0)$. Give the decimal value.