Return after a break

Coming back to Failed Star

You have a working next-token scorer. The next step is to turn it into a generator. You do not need to reread the entire repository first.

M0–M2 complete. M3 is next. Tokenization, weight inspection, and all 28 CPU transformer blocks work. Generation, KV caching, and Metal are not implemented. Three of eight milestones is a count, not a percentage of the remaining effort.

Read this page in order. Follow a refresher only when its checkpoint feels unclear. The longer working guide includes the code-reading route; the September audit records findings and verification limits.

First sitting · roughly 45–60 minutes

Read the milestones; use notes to fill gaps

First skim the map and the curriculum. Prerequisites is a repair kit, not an entrance exam. Then take this route:

M0: text → IDs. Refresher: BPE.
Can you explain why a leading space changes token IDs, and why inference replays rather than learns merges?
M1: load and inspect. Refreshers: shapes → config → bf16.
Can you explain why a weight [out,in] maps [S,in] to [S,out], and what architecture code adds to the config and tensor header?
M2: IDs → logits. Refreshers: layout → block anatomy → attention.
Can you trace [S,H] through both residual halves, then explain why the final row alone produces the next-token logits [V]?

Optional detours: file formats and mmap when reading the loader; radix trees and embedding models for terminology. Note numbers record publication order, not reading order.

Second sitting · connect prose to code

Follow the caller, then inspect the proof

Start at src/main.rs. Follow Tokenizer::encode, inspect::cross_check, then forward::forward. Read helpers when a call raises a question, not in file order.

S = sequence length; H=1024, V=151936. The model has 28 blocks, 16 query heads, 8 KV heads, and head width 128. Query width is 2048, not H.

The complete M2 checkpoints are embedding [5,1024], block 0 [5,1024], final norm [5,1024], and logits [151936]. The recorded August 20 development-Mac run passed all eight asset-backed tests. This proves the tested pinned-model path, not every prompt or future GPU execution.

Use the testing guide to verify existing golden files and compare against them. Regenerating expected output is not a way to fix a failing comparison. The historical session log is useful for decisions, but only its newest status is today's queue.

Next build · one complete teaching arc

A bounded greedy continuation

Load weights once → forward the prefix → choose the highest logit → append its ID → repeat. M3 recomputes the growing prefix. M4 will remove old-token recomputation; even then attention reads cached history.

Agree on raw-text input, empty input, exact-tie selection, EOS IDs, output inclusion of EOS, token budget, and context-limit behavior first.
Prove every generated ID against short official greedy continuations. Force EOS, ties, zero budget, and context edges in small toy tests.
Teach with one two-step trace: prefix IDs → logits [V] → selected ID → longer prefix. Explain why argmax needs no softmax.
Keep IDs as truth; decode accumulated IDs without corrupting split UTF-8 characters. Leave chat templates and the KV cache out of this arc.

After greedy parity, take a second arc for temperature, top-k, top-p, renormalization, seeded sampling, and safe streaming. Greedy alone does not close the full M3 promise. The working guide spells out the contract and tests to settle before coding.

The explanation budget: one reason, one explicit shape or contract, one worked example, one test that catches a plausible mistake. Reuse existing notes. Stop expanding when the reader can predict the next value or shape and explain the failure being prevented.