AI & Data Foundations
This branch contains compact course-style notes on machine learning, neural architectures, generative models, and data work. It is a reference shelf under cultivation, not a complete or neutral AI curriculum.
Current Branches
| Branch | Current strength | Known gap |
|---|---|---|
| Machine Learning | supervised-learning vocabulary and classical models | causal inference, uncertainty, calibration, decision theory, unsupervised/RL coverage |
| Deep Learning | short architecture maps from MLPs through transformers and diffusion | training systems, scaling, alignment, robustness, multimodal and modern evaluation detail |
| Data Science | dataset and workflow reminders | governance, provenance, leakage, privacy, representativeness, and production quality |
Agent runtimes, MCP, skills, tools, and harnesses belong in AI Systems. Current model/API choices belong in the dated Frontier Radar.
LLM Relevance, Not a New Validity Test
| Material | Relevance to LLMs now |
|---|---|
| probability, linear algebra, optimization, losses | still foundational; scale does not remove their assumptions |
| MLPs and modern activations | directly relevant: Transformer FFNs commonly use GELU/SwiGLU rather than only textbook ReLU |
| CNNs and classic RNNs | useful contrasts and deployment alternatives, but not the usual text-LLM core |
| attention and Transformers | directly relevant, including RoPE, GQA, RMSNorm, KV cache, and efficient kernels |
| VAE and diffusion notes | adjacent generative foundations, not a sufficient explanation of text LLMs |
| leakage and evaluation | essential: contamination and benchmark reuse get worse, not less important, at scale |
This shelf still lacks focused foundations for tokenization/embeddings, pretraining data and objectives, scaling, instruction/preference post-training, open-ended generation evaluation, and inference systems. Do not infer those topics from older architecture pages; use the maintained AI Systems and Frontier notes until dedicated foundations exist.
How to Read the Legacy Notes
Most child pages are concise learning aids derived from a small number of textbooks or references. Their formulas and definitions can be useful, but brevity hides assumptions and competing schools. The branch has several biases:
- textbook and architecture bias over deployment and sociotechnical context;
- supervised-learning bias over causal, unsupervised, and sequential decision methods;
- English-language and benchmark-dataset bias;
- clean-data examples over shift, leakage, feedback, and operational failure;
- historical sequence presented as a smooth progression rather than contested design choices.
Do not infer current state of the art from this branch.
Promotion Rule
A child note should move from compressed reference to maintained concept only when it is used. A substantive revision should add:
- the problem and assumptions, not only a definition;
- a derivation or executable example with observable behavior;
- failure modes, counterexamples, and comparisons;
- primary or authoritative sources plus provenance;
- links to mathematical prerequisites and practical evaluation;
- a clear distinction between durable theory and dated implementation.
Use Evidence and Bias in AI Notes as the review lens. This avoids mass-generating longer pages that remain shallow.
Supporting Network
- Mathematics: probability, calculus, linear algebra, and optimization.
- Computer Science: algorithms, data structures, and systems.
- Tools & Workflows: environments, notebooks, version control, and reproducibility.