6 docs tagged with "neural-networks"

Distinguish ReLU, GELU, SiLU/Swish, and GLU variants, then select nonlinearities with controlled modern-network experiments.

Understand content addressing, Q/K/V, masks, multi-head variants, efficient implementations, and interpretation limits through a numerical example.

Forward propagation, loss gradients, backpropagation, and parameter updates in a small multilayer classifier.

Binary classification with a linear score, sigmoid output, log loss, and chain-rule parameter gradients.

Understand MLP capacity, backpropagation, optimization failures, and inductive bias through tensor shapes and a worked XOR construction.

A single linear unit for regression, including squared loss, chain-rule gradients, and parameter updates.