Machine Learning for Finance

Supervised uses labeled (x, y) pairs to learn a mapping; unsupervised finds structure (clusters, components) in unlabeled x; reinforcement learns a policy by interacting with an environment and maximizing cumulative reward.

OLS minimizes the sum of squared residuals Σ(yᵢ − xᵢᵀβ)². Closed-form solution: β̂ = (XᵀX)⁻¹Xᵀy, assuming XᵀX is invertible.

R² = 1 − SSres/SStot, the fraction of variance in y explained by the model. R² ∈ (−∞, 1] out-of-sample; in-sample for OLS with intercept it lies in [0, 1].

Check residuals for zero mean, homoscedasticity (constant variance), no autocorrelation (Durbin–Watson), normality (Q–Q plot, Jarque–Bera), and no remaining structure vs fitted values or predictors.

Minimizes Σ(yᵢ − xᵢᵀβ)² + λ‖β‖₂². Shrinks coefficients toward zero but does not set them exactly to zero. Solution: β̂ = (XᵀX + λI)⁻¹Xᵀy.

Minimizes Σ(yᵢ − xᵢᵀβ)² + λ‖β‖₁. The L1 penalty produces sparse solutions, performing automatic feature selection by driving some coefficients exactly to zero.