The practical course

Practical LLM in Rust

Extend a separate copy of the completed Rust language model. Chapter 0 explains the structure; Chapter 1 onward introduces practical mechanisms with runnable examples. The index lists available lessons; later chapters remain part of the roadmap.

The structure of Practical LLM in Rust

Connect the practical text model's data, learning, generation and saved state, then follow every published and planned chapter from 0 to 32.
Content revision: 15
Keep a fixed reference before extending the decoder

Identify the tiny scalar decoder's fixed inputs and interpret its replay, window-slot scores, checkpoint reload and generated tokens without mistaking them for laptop-scale capability.
Content revision: 1
Prepare text externally, then load it in Rust

Use NVIDIA NeMo Curator to apply a word-count rule, remove repeated text and preserve known source-group splits, then load prepared JSONL in Rust.
Content revision: 1
Extend BPE with bounded local updates

Extend the existing Rust BPE trainer and tokenizer with bounded local updates, preserve their learned decisions and inspect a chapter-local GPT-2 segmentation example.
Content revision: 2
Pad variable-length batches without adding predictions

Extend the Rust batch and decoder APIs with right padding, valid attention prefixes and a loss averaged over real targets.
Content revision: 2
Pack sequences without joining their contexts

Use next-fit placement, occurrence segments and reset positions to preserve document-local attention, targets and token-weighted loss.
Content revision: 1
Initialize residual projection weights for decoder depth

Scale each block's two output projection weight matrices once at initialization, then inspect residual activation RMS and completed mean-loss adjoints on fixed CPU decoders.
Content revision: 4
Configure one decoder core before construction

Validate model and run records, plan resources before constructing the existing CPU decoder, then preserve full and cached text through one shared decoder path.
Content revision: 4
Make optional dropout explicit and replayable

Apply optional residual dropout with saved-mask gradients, exact evaluation identity and checked random counters. Keep local replay distinct from durable job resume.
Content revision: 3
Keep model operations and failures explicit

Read configuration refusals, optimizer failures and checkpoint publication failures at their actual state boundaries. Distinguish local evidence from future backend integration.
Content revision: 5
Separate input encoding from the transformer

Compose text lookup, the existing causal backbone and a tied text head. Follow vector shapes, live parameter checks and the two gradient paths into one shared table.
Content revision: 3
Run tensor primitives on a GPU

Select CUDA for existing matrix products and sum/mean reductions. Follow host/device copies and compare the same text model's logits, loss and gradients with a CPU reference.
Content revision: 3
Train with deliberate precision choices

Inspect FP16 and BF16 rounding in the existing decoder, retain higher-precision weights and reductions, and skip overflowing loss-scaled updates safely.
Content revision: 2
Accumulate gradients and recompute activations

Preserve token-weighted loss across smaller batches, replay decoder blocks with the same dropout choices, and measure retained tensor payload at a declared point.
Content revision: 1
Schedule AdamW updates and clip global gradients

Build a warmup and cosine schedule around existing AdamW, inspect a shared clipping factor, and keep successful updates separate from refused or skipped attempts.
Content revision: 1
Save, load and resume the composed model

Preserve a tied text model for fresh-process CPU inference, then restore the extra state for bounded exact training continuation.
Content revision: 3
Compare training and held-out language-model loss

Score valid next-token targets, compare a training-only bigram and three initialization seeds, and evaluate the saved text model on CPU.
Content revision: 1
Store weights in four bits and measure the error

Fit row-group scales, pack signed weight codes, and compare reconstructed products, saved bytes and the same text model's likelihood on CPU.
Content revision: 1
Share key/value heads and make context explicit

Trace shared key/value groups and local causal windows, count cache payload and compare each initialized model's full-prefix and cached CPU logits.
Content revision: 3
Accumulate attention without keeping the full grid

Follow a stable running attention summary, reconstruct tiled gradients and compare numerical results and logical storage with the dense reference.
Content revision: 1
Keep token order across cache pages

Trace logical positions through a KV page table, separate live and unused buffer payload, and compare the same restored model under contiguous and paged placement.
Content revision: 1