Latest publications
New notes and blog posts, newest first.
View revisionsReading “Attention Is All You Need”: The Original Transformer Paper
A section-by-section reading of Vaswani et al. 2017 covering the parallelism and long-range-dependency problem, scaled dot-product and multi-head attention, positional encoding, the training
Speculative Decoding: Drafting, Verification, and Real Speed Gains
Check how to turn on speculative decoding in a local runtime, then work through probability correction and timing examples to see when it really speeds things up, and compare chains, trees,
Vector Indexes: Exact Search, HNSW, and IVF
Start with a metric and an exact baseline, then see how HNSW and IVF reduce comparisons and measure the neighbors they miss.
Balanced Search Trees and Ordered Queries
Understand search-tree queries, rotations, and successor deletion, then use red-black invariants to bound height and compare ordered containers.
Graph Representations: Edge Lists, Adjacency Lists, and Matrices
Compare storage and operation costs on one small graph, handle duplicates, self-loops, and isolated vertices, and choose a representation for each algorithm.
Hash Tables: Collisions, Load, and Resizing
Understand key constraints, collision handling, load, and expected and amortized costs through a trace with deletion and rebuilding.
Following a Network Request: DNS, Connections, TLS, and HTTP
Follow a hostname through resolution and an HTTP response, understand ports, certificates, proxies, and localhost, and diagnose failures by layer.
Processes, Address Spaces, and Memory
How processes run, use virtual memory, inherit resources, and differ from containers and virtual machines.
Relational Data, SQL, and Transactions
Use orders and inventory to connect tables, keys, joins, NULL, constraints, indexes, concurrent transactions, and external side effects.
Concurrency and Asyncio: Ownership, Waiting, and Cancellation
Choose coroutines, threads, or processes, understand cooperative scheduling and shared state, and handle timeouts and failures with a bounded queue.
Constrained Optimality and Dual Certificates
Solve a two-variable quadratic program, check its active constraints and KKT multipliers, and certify optimality with a primal–dual gap.
Counting and Combinatorial Proofs
Define the objects being counted, then use bijections, double counting, inclusion–exclusion, and pigeonhole arguments to solve finite problems and bound search and output costs.