Failed Star / fs

Milestone M2 · it understands

Forward pass → logits

Text now travels through the complete 28-layer Qwen3-0.6B network in clear CPU Rust. The result is 151,936 next-token scores—and every one matches the pinned official implementation.

Status: ✅ done. Four complete fp32 checkpoints pass at atol=rtol=1e-4: embedding, block 0, final norm, and every logit. The top result for “The capital of France is” is token 12095, “ Paris”.
$ fs logits "The capital of France is"
prompt tokens: 5 · residual [seq=5, H=1024]
next-token logits: [V=151936] · top 10
  TOKEN ID        LOGIT  DECODED PIECE
     12095     17.498833  " Paris"
      7407     14.337203  " located"
       279     14.169322  " the"

One prefill, all shapes visible

The complete path

TOKENIZE

Text → IDs

[785,6722,315,9625,374]
shape [seq=5]

GATHER

IDs → residual

Copy embedding rows.
[5,1024]

× 28

Transformer blocks

Attention + SwiGLU, each added back to the H-wide bus.

FINAL NORM

Stabilize the bus

Row-wise RMSNorm.
[5,1024]

LAST ROW

Project to vocab

Tied embedding table as [V,H] linear.
[151936]

x [seq,H] residual bus RMSNorm QK-norm · RoPE · GQAcausal attention → o_proj + RMSNorm SwiGLU→ down +
Each sublayer reads a normalized copy and adds its contribution to the unnormalized bus. Shapes remain [seq,H] across both residual additions.

Correct before fast

One matrix, no hidden machinery

Compute uses one contiguous row-major Matrix<f32>. BF16 weights widen once at a strict name/shape/dtype boundary. Linear layers dot contiguous input and stored [out,in] weight rows; embedding is a gather, not a matmul.

Qwen's file stores a byte-identical tied lm_head. M1 proves the identity; M2 keeps only one f32 table and borrows it for output projection, avoiding roughly 622 MB of redundant allocation. The representation matches the math.

Deep dives: row-major & strides, block anatomy, and attention worked through.

Numbers, not resemblance

Four boundaries bisect the whole model

CHECKPOINT 1

Embedding

[5,1024] · 5,120 values
Proves IDs, row offsets, and BF16 widening.

CHECKPOINT 2

Block 0

[5,1024] · 5,120 values
Proves attention, SwiGLU, norms, and residuals once.

CHECKPOINT 3

Final norm

[5,1024] · 5,120 values
Proves the complete 28-layer fold.

CHECKPOINT 4

Logits

[151936] · every score
Proves last-row selection and tied projection.

Oracle contract. Pinned Qwen revision, eager CPU fp32, one thread, eval/inference mode, no cache. Every value must satisfy |got−want| ≤ 1e−4 + 1e−4·|want|.

What M2 does not do

Logits are not generation

fs logits ranks the model's beliefs but does not select or append a token. M3 will first build a deterministic greedy loop and reproduce an official continuation; sampling comes only after parity. M4 then avoids recomputing the prompt with a KV cache.

The CLI rejects empty prompts and inputs beyond the configured 40,960-token context before loading weights; internal calls assert that bound. This does not promise practical CPU performance at maximum context.

Compatibility means the pinned Qwen3-0.6B checkpoint, not arbitrary model or tokenizer configurations. Keep mapped assets immutable and run verify_golden.py before the asset-backed suite; the CLI does not automatically check provenance. See verification commands.

📖 Inference Engineering · §2.1, §2.2.2–2.2.3🔧 ds4 · dense, norm, RoPE, attention, GLU🧭 Raschka · architecture comparison