Skip to main content

How LLMs work

Follow one prompt through a large language model. The prompt becomes tokens; the model decodes one token at a time through attention and the Transformer block, keeping a key–value cache. Some models also route each token through a few experts or take images as input. The last two notes explain how post-training and LoRA shape the behaviour you get.

Reading Order​

StepArticleYour job
1Tokenization and Text Representations Inside a ModelThe prompt becomes token IDs, embeddings and contextual states.
2Autoregressive Generation and DecodingThe model picks one next token at a time: temperature, top-k, top-p, stopping and structured output.
3Attention MechanismHow each token looks up the others with Q, K and V, worked through in numbers.
4Transformers: Original Architecture and Modern BlocksThe block a model stacks many times, from the 2017 design to modern components.
5Attention Variants and KV-Cache CompressionWhy memory grows with context, and how MQA, GQA, MLA and cache compression shrink it.
6Mixture of Experts: Sparse Computation, Routing, and Deployment CostA router sends each token to a few experts: active parameters, total weights, memory and speed.
7Multimodal Models: Alignment, Fusion, and Visual QuestionsHow images join the input: patches, modality connectors and visual questions.
8Encoder, Decoder, and Encoder–Decoder: How Information FlowsWhich models read and which write: classifiers, generators and sequence converters.
9Pretraining, Supervised Fine-Tuning, and Preference OptimizationHow pretraining, supervised fine-tuning and preference optimization shape what the model does.
10Transfer Learning, LoRA, and DistillationAdapt a model to your task with a frozen encoder, full fine-tuning, LoRA or a smaller student.
Explore connectionsOpen network