Machine learning and training

Learning changes a model's parameters using data or interaction. Start by identifying the input, target, loss and evaluation split; then ask which assumptions let a trained pattern carry over to a new example.

Linear regression introduces a model and gradient updates; least squares develops the closed-form solution, statistical assumptions and regularization. Classification and training mechanics lead into adaptation. Trees, clustering and reinforcement learning show alternatives to fitting a neural network to fixed labels.

Reading Order

Step	Article	What it explains
1	Machine Learning General Concepts	A compact vocabulary for models, losses, empirical risk, likelihood, regularization, optimization, and generalization.
2	Linear Regression: Modeling and Gradient Descent	Build a linear predictor, calculate residuals, and follow gradient updates; closed-form estimation and statistical inference are covered separately.
3	Ordinary Least Squares and Regularization	Least-squares estimation and the Ridge, Lasso, and Elastic Net penalties used to control unstable or overly complex coefficients.
4	Softmax Regression	Multiclass linear classification with logits, softmax probabilities, cross-entropy, and clear boundaries around multilabel tasks.
5	Linear and Quadratic Discriminant Analysis	Gaussian generative classifiers whose shared or class-specific covariance assumptions produce linear or quadratic decision boundaries.
6	Training a Model: Gradients, Optimizers, and Learning Rates	Follow one parameter update, then connect minibatches, optimizer state, validation, and reproducible checkpoints.
7	Pretraining, Supervised Fine-Tuning, and Preference Optimization	Explain how training targets shape representations, instruction following, and preferred behavior without equating preference with truth.
8	Transfer Learning, LoRA, and Distillation	Choose between a frozen encoder, full fine-tuning, low-rank adaptation, and a smaller student for a defined task.
9	Decision Trees, Random Forests, and Gradient Boosting	Derive a tree split and a boosting update, then compare their inductive biases, validation needs, and limits.
10	Clustering and Dimensionality Reduction	Separate discovering groups from compressing representations, using worked k-means and PCA examples.
11	Reinforcement Learning: Rewards, Value, and Sequential Decisions	Use a two-state example to distinguish immediate rewards, long-term value, exploration, and learned agent policies.

Use What You Read

Use one small example to connect prediction, loss and parameter update. Keep data splitting and evaluation separate from optimization: a falling training loss alone does not establish useful generalization.

Return to the AI reading paths to choose a neighboring topic.

My reading · Opened notes are remembered in this browser.