Linear Models

A model is linear when its prediction is linear in learned coefficients, even if the input is first transformed into nonlinear basis features:

$$f(x)=\mathbf{w}^{\top}\phi(x)+b.$$

Choose a member of the family by answering four questions:

Question	Common choice
Is the target numerical?	least squares, robust loss, quantile regression, or an appropriate generalized linear model
Is the target categorical?	logistic/softmax regression, or a discriminant model with explicit class distributions
Are coefficients unstable or overfit?	Ridge ($\ell_2$), Lasso ($\ell_1$), or Elastic Net (both)
Is the relationship nonlinear in raw features?	specify basis functions or interactions while retaining linear coefficients

Site Path

• Ordinary Least Squares and Regularization covers OLS, Ridge, Lasso, and Elastic Net.
• Linear and Quadratic Discriminant Analysis compares Gaussian generative classifiers.
• Linear Regression presents the same predictor as a shallow neural model.
• Softmax Regression develops multiclass linear classification.

Regularization parameters and feature transformations must be selected inside the training/validation process. Coefficients are not automatically causal, stable, or directly comparable across differently scaled features.

Use the scikit-learn linear-model guide for maintained implementation details; this page owns the conceptual selection map.