Statistical Analysis & Time Series

A process {Xₜ} is weakly stationary if (1) E[Xₜ] is constant, (2) Var(Xₜ) is finite and constant, and (3) Cov(Xₜ, Xₜ₊ₖ) depends only on lag k, not on t.

The joint distribution of (X_(t₁),…,X_(tₖ)) equals that of (X_(t₁+h),…,X_(tₖ+h)) for all h, k, and time indices. Strict stationarity implies weak stationarity when second moments are finite.

ρ(k) = Cov(Xₜ, Xₜ₊ₖ) / Var(Xₜ). Measures linear dependence between observations separated by lag k. Always satisfies |ρ(k)| ≤ 1 and ρ(0) = 1.

PACF at lag k is the correlation between Xₜ and Xₜ₊ₖ after removing the linear effect of the intermediate lags Xₜ₊₁,…,Xₜ₊ₖ₋₁. Equivalent to the last coefficient in an AR(k) regression.

AR(p): ACF tails off (decays); PACF cuts off after lag p. MA(q): ACF cuts off after lag q; PACF tails off. ARMA(p,q): both tail off.

A sequence {εₜ} is white noise if E[εₜ] = 0, Var(εₜ) = σ², and Cov(εₜ, ε_s) = 0 for t ≠ s. Gaussian white noise additionally requires εₜ ~ N(0, σ²) i.i.d.