Skip to content
Cook ML
GitHubRSS

Tutorials

Each series is a sequence of chapters meant to be read in order. Chapters open with a plain example, then build the mechanism, then compute it on real numbers. A short concept summary closes every chapter.

9 chaptersLLM FundamentalsHow language models work, from next-token prediction to training, inference, and evaluation.Start reading →
PlannedTransformers in DepthEfficient attention, long context, scaling laws, and modern architectures.
PlannedRetrieval and RAGChunking, vector search, reranking, and evaluating retrieval systems.
PlannedML SystemsDistributed training, serving, batching, and hardware budgets.

A large language model reads text as small pieces and repeatedly predicts the next one. This series follows that one prediction step from a single probability to a released assistant. Start at the series overview for the reading plan and a formula quick reference.

  1. 01Language Modeling and Next Token PredictionTurn a sentence into a product of next-token probabilities with the chain rule, then see why counting word sequences breaks down and how a neural network shares statistical strength across contexts.
  2. 02Tokenization and EmbeddingsChoose between characters, words, and subwords, then run byte pair encoding merge by merge and compare it with WordPiece and Unigram.
  3. 03Attention and Positional InformationCompute attention from queries, keys, and values on a three-token example, explain the square-root scaling and the causal mask, and split the width into heads.
  4. 04The Transformer Block and Model AnatomyFollow the residual stream through normalization, attention, and the feed-forward network, and compare LayerNorm with RMSNorm and pre-norm with post-norm.
  5. 05Pretraining at ScaleBuild a corpus with deduplication, filtering, and mixing, then pack documents into training windows.
  6. 06Post Training from Base Model to AssistantExplain what a base model does, then add supervised fine-tuning with loss only on response tokens.
  7. 07Inference and DecodingSeparate prompt processing from token generation, size the key-value cache, and show why generation is limited by memory bandwidth.
  8. 08Using LLMs in ApplicationsSteer a frozen model with system prompts, in-context examples, and chain of thought.
  9. 09Evaluation Reliability and SafetySeparate loss, benchmark accuracy, and human preference, estimate pass@k without bias, and size an evaluation set for the difference you need to detect.