The build, in order
Failed Star is built as a curriculum: M0 → M7, in order, each one a runnable artifact plus a doc that teaches it. This is the spine of the whole project — the home page tells the story, the Learnings capture the why, and this page is the where-we-are and the map to every milestone writeup.
Failed Star ☑ done · ◐ now ·
☐ todo. A milestone is done when it runs and its output is
verified against a known-good reference. Done milestones link to their full doc; upcoming
ones get theirs written as we build them.
Where we are
3 / 8 core milestones done · currently M3 — deterministic generation
One artifact + one doc, each
MILESTONE M0 · ☑ DONE
Text ↔ token IDs. BPE encode/decode against the model's real vocabulary — no GPU, no weights, the model's front door.
fs tokenize "hello world" → IDs, and decode back.
doc → m0-tokenizer
MILESTONE M1 · ☑ DONE
Parse the model file, map every tensor (names, shapes, dtypes) into memory, and prove the file matches the architecture the config implies.
fs inspect models/qwen3-0.6b → architecture + tensor table.
doc → m1-weights
MILESTONE M2 · ☑ DONE
Embeddings → N transformer blocks (RMSNorm, RoPE, attention, SwiGLU) → final norm → logits. CPU/Rust first — correct before fast. The "it understands" milestone.
Finished in three teaching arcs: completed the block math; connected real weights and proved block 0; then ran all 28 layers, matched every final logit, and restored the CLI.
The complete model-free suite and all 8 asset-backed release tests pass on the development Apple Silicon Mac against the pinned, hash-verified Qwen assets.
fs logits "The capital of France is" → top-k next tokens.
doc → m2-forward-pass
MILESTONE M3 · ◐ NOW
Build the autoregressive loop and greedy oracle parity first; only then add temperature / top-k / top-p and streaming. Chat templating remains optional.
Two complete teaching arcs: bounded greedy generation with explicit stop/context rules and official ID parity; then ordered filters, seeded sampling, and UTF-8-safe streaming. Each includes its Markdown/HTML lesson. Read the starting brief →
fs generate "..." reproduces a greedy continuation.
doc → lands with the milestone
MILESTONE M4 · ☐ TODO
Cache K/V per layer so decode avoids recomputing prior K/V and computes the new token's K/V once. Attention still reads the growing cached history; work is not constant. RAM-only first. The "I made it faster" milestone.
cached/uncached outputs agree; benchmark both as a baseline.
doc → lands with the milestone
MILESTONE M5 · ☐ TODO
Bring up device/queue/buffers/pipelines and move the complete path to MSL through bounded raw ObjC/Metal FFI. CPU remains the correctness oracle.
End-to-end GPU generation on the local Apple Silicon Mac.
doc → lands with the milestone
MILESTONE M6 · ☐ TODO
Profile the correct GPU path, optimize measured bottlenecks, and fuse only where reproducible before/after measurements justify it.
Local-Mac benchmarks with CPU/unfused agreement.
doc → lands with the milestone
MILESTONE M7 · ☐ TODO
Use the benchmark baseline to decide whether low-bit work earns its complexity. GGUF remains a separate optional interoperability choice. Earlier stretch ideas are explicitly optional post-core experiments, not promised curriculum.
A documented decision; if go, measured quality/memory/speed deltas.
doc → per goal, if we take it
m0-tokenizer, M1 → m1-weights,
M2 → m2-forward-pass, and so on. Docs without an mN-
prefix (the map, the prerequisites)
are context, not milestones.
The full curriculum — verify steps, book/page
cross-links, and ds4 references for each milestone — lives in
PLAN.md;
the running "where are we" is
PROGRESS.md.