Data structures and algorithms for AI engineers
Classic data structures (arrays, linked lists, stacks, queues, hash tables, trees, graphs) still run underneath every AI system. What changes in AI is the kind of data and the cost model: instead of discrete values that are either equal or not, you work with continuous values (decimal numbers in vectors and tensors) where distance carries meaning, and instead of counting operations, you count bytes, because moving data costs more than computing on it. DSA for AI Engineers is a 12-module roadmap that bridges the two, from the classical structures you already know to tensors, embeddings, similarity search, attention, the KV cache, decoding, agent graphs and serving, with every module running on a laptop.
Classical DSA is the prerequisite, not the past
You already use classic data structures every day without naming them. The browser's back button is a stack: the last page you visited is on top. Your call log keeps the most recent call at the top of the list. A video buffers in order, packet after packet, even though the packets sit at different places in memory, which is how a linked list works.
None of that goes away in AI. Tokenizers use hash maps and tries, top-k search uses a heap, caches use a hash map plus a linked list, agent workflows are graphs. What AI adds is a new kind of value to store, and a new reason for structures to exist.
From discrete to continuous
Classic structures store discrete values: integers, booleans, exact keys. The only question you can ask is "is it the same?".
AI represents meaning as continuous values: vectors of decimal numbers, where "cat" and "kitten" sit close together. The question becomes "how close is it?", and that changes every structure built on top: indexes, caches, search, memory.
Counting bytes, not operations
Classical complexity counts operations, because it assumes every memory access costs the same. On a GPU, fetching data from main memory costs roughly what hundreds or thousands of arithmetic operations cost. So for AI systems, you count bytes moved as well as operations, and many structures in this roadmap (the KV cache, paged memory, batching, quantization) exist because of that.
Roadmap
| # | Module | Level | Classical structure it reuses | Build | Pages |
|---|---|---|---|---|---|
| 0 | Prerequisites | 1 | Arrays, hash maps, heaps, tries, graphs, queues | dsa-bridge |
Discrete to continuous |
| 1 | Cost model | 2 | Big-O and its blind spot | flopcheck |
Floating point precision |
| 2 | Tensors as memory | 3 | Flat index arithmetic | microtensor |
Tensors |
| 3 | Embeddings | 4 | Hash map, heap | embedstore |
Tokenizer data structures, Embedding tables |
| 4 | Similarity search | 5 | Binary search, graph traversal | annlab |
Nearest neighbour search |
| 5 | Retrieval and context | 5 | Knapsack, sorting | hybridrag |
Hybrid search |
| 6 | Attention | 7 | Streaming aggregate | attention-lab |
Logits and softmax, Compute graphs |
| 7 | KV cache and paging | 8 | Trie, virtual memory, LRU | kvsim |
KV cache |
| 8 | Decoding and speculation | 7 | Heap, priority queue, tree search | decodelab |
Sampling |
| 9 | Graphs and agents | 8 | Topological sort, state machines | dagrun |
Graphs for agents |
| 10 | Serving economics | 9 | LRU, queueing | servesim |
Hashing and caching, Queues and batching |
| 11 | Distributed | 10 | Hash partitioning | ringsim |
Inference optimization |
Difficulty rises across modules, not inside them: module 1 is level 2 of 10, module 11 is level 10, and everything inside a module is roughly the same level. Every module ends with an artifact you build and can show in a portfolio.
Bridge map
| Classic DSA | AI equivalent | Where it shows up |
|---|---|---|
| Array and flat indexing | Tensor strides | Tensors, module 2 |
| Hash map | Embedding lookup, exact-match cache | Embeddings, module 3, serving, module 10 |
| Heap or priority queue | Top-k search, beam search | Similarity search, module 4, decoding, module 8 |
| Trie | Shared prompt prefixes in the KV cache | KV cache, module 7 |
| Virtual memory pages | PagedAttention | KV cache, module 7 |
| LRU cache | Cache eviction | Serving, module 10 |
| Graph and DAG | Agent workflows, task plans | Agents, module 9 |
| Queue | Request batching | Serving, module 10 |
| Running aggregate | Online softmax in attention | Attention, module 6 |
Tracks
- Build: modules 0 to 5, plus 7 and 10, for engineers building AI applications.
- Serve: adds modules 6, 8 and 9, for engineers serving and scaling models.
- Systems: adds module 11, for distributed serving.
From the floor
In the AI data structures lecture for cohort 2, I started from structures everyone uses without naming them, the browser's back button as a stack, the call log, a video buffering in order like a linked list, and then made the shift explicit: classical structures store discrete values, and AI needs continuous values, because a single number for a word like "upgrade" can't carry its meaning, while a vector of decimal numbers can. I laid out the stages that follow from that: the bridge from discrete to continuous, then tensors, embeddings, similarity search, the KV cache and compute graphs.