Skip to main content

ML and deep-learning foundations

This shelf is for readers who train their own models. It covers classical learning and training, the building blocks of neural networks, generative models, and where a dataset comes from and how it shifts.

Reading Order​

StepArticleYour job
1Machine Learning General ConceptsThe vocabulary: model, loss, empirical risk, likelihood, regularization and generalization.
2Linear Regression: Modeling and Gradient DescentFit a line, compute residuals and follow gradient-descent updates.
3Ordinary Least Squares and RegularizationThe closed-form least-squares solution, and Ridge, Lasso and Elastic Net.
4Softmax RegressionMulticlass classification with logits, softmax probabilities and cross-entropy.
5Linear and Quadratic Discriminant AnalysisGaussian classifiers whose covariance assumption gives a linear or quadratic boundary.
6Training a Model: Gradients, Optimizers, and Learning RatesFollow one parameter update, then minibatches, optimizer state, learning rates and checkpoints.
7Decision Trees, Random Forests, and Gradient BoostingDerive a tree split and a boosting update, then compare their biases and validation needs.
8Clustering and Dimensionality ReductionFind groups with k-means and compress features with PCA.
9Reinforcement Learning: Rewards, Value, and Sequential DecisionsRewards, value and exploration on a two-state example.
10Multilayer PerceptronTensor shapes, backpropagation and a worked XOR network.
11Activations and Gated Feed-Forward NetworksReLU, GELU, SiLU and the GLU variants used in modern feed-forward layers.
12Convolutional Neural NetworksConvolution, channels, padding, stride and pooling; equivariance versus invariance.
13Recurrent Neural NetworksRecurrent state, backpropagation through time, LSTM gates and state-space models.
14Reading “Attention Is All You Need”: The Original Transformer PaperThe 2017 paper read closely: architecture, training setup and results.
15Variational AutoencodersLatent-variable generation with the ELBO and the reparameterization trick.
16Diffusion ModelsForward noising, learned denoising and iterative sampling.
17Datasets: Provenance, Leakage, and EvaluationWhere a dataset comes from, and what its collection process lets you claim.
18Model Evaluation Under Distribution ShiftEvaluate for the population you will deploy on, not for one test set.
Explore connectionsOpen network