# Coming back to Failed Star

**Start here after a break.** This is a reading route, not another prerequisite
course. The [HTML reading guide](reorientation.html) gives the same route on the
learning site. For this review's findings and evidence, see the
[September 2026 audit](audit-2026-09.md).

## Where we actually are

We have a **verified, uncached CPU next-token scorer for Qwen3-0.6B**. We do not
yet have a text generator or a Metal engine. Three of eight curriculum milestones
are complete; that is a milestone count, **not 37.5% of the effort**.

| Milestone | Working artifact | What completion establishes |
|---|---|---|
| M0: tokenizer | `fs tokenize`, `fs detokenize` | Official token-ID parity on committed cases, including whitespace, Unicode, and special literals |
| M1: weights | `fs inspect` | Config-derived names, shapes, dtypes, complete directory, tied head, parameter counts and selected byte checks |
| M2: forward | `fs logits` | All 28 CPU transformer blocks; complete official embedding, block-0, final-norm and last-token logit checkpoints for one fixed prompt |
| M3: generation | Not implemented | Current planning target: deterministic continuation, then sampling |
| M4–M7 | Not implemented | KV cache → Metal correctness → measured optimization → conditional quantization |

M2's development-Mac gate passed on August 20, 2026 (recorded in
[PROGRESS](../PROGRESS.md)). That result is real evidence, but not proof of all
prompts, other architectures, GPU behavior, speed, or sampling quality.

## First sitting: recover the story, not every detail

Allow roughly 45–60 minutes; skip a refresher when you can answer its checkpoint.
Do not read the session log from the beginning. Its older “next” instructions are
history, not today's work queue.

1. **Get the map:** skim [the site overview](index.html), then
   [the milestone plan](../PLAN.md). Use [prerequisites](prerequisites.html) only
   to repair a gap. Checkpoint: *Where does a forward pass end, and what must a
   generation loop add?*
2. **M0 — text becomes IDs:** read [the tokenizer walkthrough](m0-tokenizer.html).
   Open [BPE](learnings/03-bpe.html) if merge priority or byte encoding is fuzzy.
   Checkpoint: *Why can a leading space change an ID, and why don't we learn new
   merges at inference time?*
3. **M1 — IDs meet a model:** read [the weights walkthrough](m1-weights.html).
   Refresher order: [shapes](learnings/05-reading-shapes.html) →
   [config](learnings/09-config.html) → [bf16](learnings/07-bf16.html).
   Checkpoint: *Why does a Linear weight stored as `[out,in]` map `[S,in]` to
   `[S,out]`, and why do we need both config and architecture code?*
4. **M2 — the numerical core:** read [the forward-pass walkthrough](m2-forward-pass.html).
   If needed, follow [layout/strides](learnings/08-row-major-strides.html) →
   [block anatomy](learnings/10-transformer-block-anatomy.html) →
   [attention](learnings/11-attention.html).
   Checkpoint: *Trace `[S,H]` through both residual halves and explain why only
   the final row is projected to `[V]` for the next token.*
5. **Confirm today's boundary:** read only the top of [PROGRESS](../PROGRESS.md)
   and the M3 section below. You are ready to continue when you can explain the
   loop in words; memorizing the attention derivation is unnecessary.

**Optional, not gates:** [file-format comparison](learnings/01-safetensors-vs-gguf.html)
and [mmap](learnings/06-mmap.html) when inspecting the loader;
[radix trees](learnings/02-radix-tree.html) and
[embedding models](learnings/04-embedding-models.html) as terminology detours.
The note numbers record publication order, not prerequisite order. The
[resource catalog](RESOURCES.md) is a reference shelf, not assigned reading.

## Second sitting: connect the story to the code

Read callers before helper details. There is no need to read every test first.

| Question | Start here | Then follow |
|---|---|---|
| What can I run? | [`src/main.rs`](../src/main.rs) | Command dispatch to each module's `run` |
| How does text become IDs? | [`Tokenizer::encode`](../src/tokenizer.rs) | Special carving → pretokenization → byte mapping → BPE → vocabulary lookup |
| How do we trust the directory? | [`inspect::cross_check`](../src/inspect.rs) | [`model_schema`](../src/model_schema.rs), [`Config`](../src/config.rs), [`SafeTensors`](../src/safetensors.rs) |
| Where is the actual model? | [`forward::forward`](../src/forward.rs) | `forward_to_final_norm` → `transformer_block` → attention and FFN; [`Matrix`](../src/tensor.rs) for storage |
| Why believe the result? | [Testing guide](testing.md) | Toy tests beside the math, then the [forward manifest](../tests/golden/forward/manifest.json) and ignored golden tests |

For Qwen: `S` = prompt length, `V=151936`, `H=1024`, `L=28`, query heads = 16,
KV heads = 8, `d=128`, `I=3072`. In particular **16 × 128 = 2048, not H**.

```text
text → ids[S] → embedding[S,H] → 28 blocks[S,H] → final norm[S,H]
                                                        ↓ last row[H]
                                                   logits[V]
                                                        ↓ M3 adds this
                                           choose ID → append → repeat
```

Use [dev-loop](dev-loop.md) for commands and [the Mac runner guide](mac-amp-runner.md)
for target-machine setup. Do not regenerate goldens merely to run tests: verify
the committed fixture hashes, then compare the engine to those unchanged fixtures.

## Next: one complete deterministic generation arc

The next useful chunk is **a bounded greedy continuation end to end**, not one
helper per session. Use existing `Weights` and `forward`; load weights once,
rerun the growing prefix for each token, and leave KV caching to M4.

Before implementation, settle and record a short contract:

- **Input:** raw text, no implicit chat template or automatic BOS; define empty
  input handling and `max_new_tokens` (including zero).
- **Selection:** argmax of finite `logits[V]`; lowest token ID wins an exact tie.
  No softmax is needed to rank logits for greedy selection.
- **Stopping:** decide the explicit EOS ID set from the pinned model's config
  and generation config; do not assume every special token is EOS. Record
  whether the stop ID appears in returned IDs and displayed text.
- **Budget:** never forward beyond `max_position_embeddings`; distinguish EOS,
  token budget, and context exhaustion as stop reasons. Specify behavior when a
  last allowed forward predicts one more token but no further forward can run.
- **Decoding:** keep generated IDs as the source of truth. Decode accumulated IDs
  for final text; single-token lossy decoding can split a UTF-8 character.

**The proof:** commit short official greedy continuations with provenance and
explicit decoding/stopping settings. Compare every generated ID, not just the
rendered sentence. Use toy tests to force EOS on the first step, an exact logit
tie, zero budget, and both sides of the context boundary; ordinary prose prompts
rarely exercise those cases. Run the final gate on Apple Silicon.

**The teaching deliverable:** begin `m3-generation.md` alongside the code, with a
single two-step trace: prefix IDs → logits `[V]` → selected ID → longer prefix.
Explain logits versus probabilities, greedy selection, stop/budget rules, and
why this first loop recomputes history. Link the existing tokenizer and forward
notes instead of repeating them. Distill the working arc to HTML in the same
change; finish the milestone writeup when sampling lands.

After greedy parity, take a second arc for **temperature → top-k → top-p →
renormalize → seeded sampling**, with explicit filter order and known-answer
tests. Streaming must preserve UTF-8 boundaries. Chat formatting remains optional.
Do not mark all of M3 done after greedy alone while sampling remains promised.

## How much explanation is enough?

Assume loops, arrays, basic Rust reading, and scalar arithmetic. Introduce an
inference-specific concept before it first changes code or a correctness choice.
For each new concept aim for **one reason, one explicit shape/contract, one
worked example, and one discriminating test**. Link deeper derivations.

Stop expanding when you can predict the next intermediate value or shape and
explain the failure the test prevents. Open a new learning note only for a concept
with a reusable identity; otherwise improve the owning milestone or existing
note. More documentation is useful only when it answers a new question.