One grid, an ordered buffer

Shape is not layout

A shape [3,4] gives three rows and four columns. The same logical grid could be stored row-first as a,b,c,d,e,… or column-first as a,e,i,b,…. Safetensors is C-contiguous, and our Matrix keeps that row-major order after widening to f32.

flat_index(row, col) = row · C + col

Matrix::row(r) = data[r · cols .. (r + 1) · cols]

That slice exposes the invariant: one logical row is one uninterrupted run of cols values.

The indexing multipliers

Strides turn coordinates into offsets

A stride is the number of elements skipped when one axis advances. For row-major [R,C], strides are [C,1]. In N dimensions, the last stride is 1 and every stride to its left is the product of dimensions to its right:

offset(i₀,…,iₙ) = i₀·stride₀ + … + iₙ·strideₙ

A general tensor library can make transpose “free” by swapping shape and strides while leaving the bytes alone. Failed Star's M2 Matrix intentionally cannot: it always means len = rows·cols and stride (cols,1).

A real offset

Token 785 in embed[V,H]

The safetensors reader gives a tensor-relative range. Add its start to the row-major element offset, then multiply by bf16's two bytes:

file_byte(row,col)
  = data_start + tensor.start + (row·C + col)·2

For the real embedding table [V,H] = [151936,1024], the oracle prompt begins with token id 785 (“The”). Relative to the tensor start:

M2 widens that row into 1,024 adjacent f32 values: 4,096 owned bytes. Width changes; row-major order does not. This also explains M1's validation tensor bytes = R·C·dtype.size(): if it fails, no stride formula can make header and blob agree.

Gather copies one contiguous row

Embedding is not multiplication. For embed[V,H] and ids[seq], gather selects each id's H-wide row, preserving token order:

ids [2,0,2] · embed [V=4,H=3] → output [seq=3,H=3]
row 2 [20,21,22]                 [20,21,22]
row 0 [ 0, 1, 2]                 [ 0, 1, 2]
row 2 [20,21,22]                 [20,21,22]

embedding_gather asserts every id is below V, finds the row start, and copy_from_slices one contiguous width — the CPU counterpart of ds4's get_rows.

Reading the triple loop

Matmul: three indices, three flat offsets

For A[M,K] · B[K,N] → C[M,N], one output cell is:

C[i,j] = Σₖ A[i,k] · B[k,j]

A[i,k] → A.data[i·K + k]
B[k,j] → B.data[k·N + j]
C[i,j] → C.data[i·N + j]

M2's naive (i,j,k) loop makes the access pattern visible: A advances contiguously while B jumps by N. Tiling, SIMD, transposition, and Metal can improve that later; first we need a baseline we understand and can measure.

The shape convention pays off

[out,in] means two contiguous rows

Neural-network weights are stored W[out,in], while the equation is Y = X·Wᵀ. The superscript T is mathematical bookkeeping: for output feature o, linear dots input row X[t,:] directly with weight row W[o,:]. Both are contiguous and exactly in wide.

Numerical neighbors

Rows stay explicit elsewhere, too

Every helper asserts its shape contract before indexing. Known-answer tests check both numbers and loud failures; RoPE also checks that rotation preserves vector length.