LLM systems architecture: chapter index
LLM Systems Architecture and Strategy is my lecture series on the production LLM inference pipeline, in 8 chapters and 40 topics, numbered 1.1 to 8.5. The pipeline itself is drawn in the six zones page, and each chapter opens up one part of it: the physical limits of a model, test-time compute, retrieval, agents, customisation, operations, security and the economics of the whole system. This page maps every topic to the docs page that covers it, so you can read the series in order or jump to one topic.
Chapter 1: the physical limits of a model
| Topic | Page |
|---|---|
| 1.1 How does text become tokens | Tokens |
| 1.2 The internationalization tax | Tokens |
| 1.3 Prefill against decode | Inference optimization |
| 1.4 The KV cache and paged attention | The KV cache |
| 1.5 Sizing GPU memory | Capacity and cost modelling |
Chapter 2: test-time compute
| Topic | Page |
|---|---|
| 2.1 What changes when a model thinks | Reasoning models |
| 2.2 How a reasoning model searches | Reasoning models |
| 2.3 Streaming the thinking tokens | Latency and streaming |
| 2.4 Process supervision against outcomes | Reasoning models |
| 2.5 Where the hardware budget goes | Capacity and cost modelling |
Chapter 3: context and retrieval
| Topic | Page |
|---|---|
| 3.1 Where simple vector search breaks | RAG failure modes |
| 3.2 Hybrid search and rank fusion | Hybrid search |
| 3.3 Cross encoder reranking | Reranking |
| 3.4 Parent and child chunking | Chunking strategies |
| 3.5 Prompt compression | Context compression |
Chapter 4: orchestration and agents
| Topic | Page |
|---|---|
| 4.1 The model as a kernel | Agents and tool use |
| 4.2 The reason and act loop | Planning patterns |
| 4.3 Sandboxing tool calls | Tool registry and sandbox |
| 4.4 Multi agent patterns | Multi-agent systems |
| 4.5 Where autonomy breaks | Human in the loop |
Chapter 5: model customisation
| Topic | Page |
|---|---|
| 5.1 Low rank adaptation | LoRA and QLoRA |
| 5.2 Quantized fine tuning | LoRA and QLoRA |
| 5.3 Quantization formats | Quantization |
| 5.4 Speculative decoding | Inference optimization |
| 5.5 Swapping adapters in production | Serving adapters |
Chapter 6: operations and evaluation
| Topic | Page |
|---|---|
| 6.1 Perplexity and loss | Evaluating fine-tuned models |
| 6.2 Task metrics and model judges | Check your LLM judge |
| 6.3 The dual guardrail pattern | Guardrail sandwich |
| 6.4 Semantic caching | Semantic caching |
| 6.5 Evaluation bias | Judge prompt overfitting |
Chapter 7: security and governance
| Topic | Page |
|---|---|
| 7.1 Direct prompt injection | Prompt injection |
| 7.2 Indirect prompt injection | Prompt injection |
| 7.3 The instruction hierarchy | System prompts |
| 7.4 System prompt leakage | Data leakage and PII |
| 7.5 Refusal bias | Test abstention |
Chapter 8: economics and strategy
| Topic | Page |
|---|---|
| 8.1 Build against buy | Build vs buy vs partner |
| 8.2 Dynamic model routing | Model routing and gateways |
| 8.3 One request traced end to end | LLM inference pipeline |
| 8.4 A refund agent post mortem | Incident response |
| 8.5 Three rules for production | Notebook to production checklist |
How to use this index
If you're following the lecture series, read each chapter's pages after the lecture, in order. If you're designing a system, start at the six zones, find the zone you're working on, and use the chapter that covers it.
From the floor
The series started from the first AI System Design Engineering session, where I walked one query through the full production pipeline in about 90 minutes and said each part deserved its own deeper lecture. Those deeper lectures are the 40 topics above.