Learning 01 · where weights live on disk
This is the first thing we learned: where do a model's weights actually live on disk, and in what shape? A model is, physically, two things — a pile of numbers (the weights) and a description of how they're arranged (names, shapes, dtypes). A file format is just an agreed way to write those two down. Two formats dominate, built for different jobs.
HuggingFace's native format
This is what Qwen3-0.6B ships as on Hugging Face. It is almost shockingly simple — which is exactly why it's a great first format to parse by hand. The whole file is three concatenated regions:
Failed Star u64 length, then that
many bytes of JSON describing every tensor, then one contiguous blob of raw bytes.u64 giving the length
N of the header.N bytes: a UTF-8 JSON object. Each key
is a tensor name; each value is
{ "dtype": …, "shape": […], "data_offsets": [start, end] }.
(An optional __metadata__ key holds arbitrary string key/values.)[start, end) within that blob, row-major (C-contiguous).That's the entire format. The properties that matter to us:
This Qwen3 export is bf16, which need not be the dtype used throughout training. There is no quantization scheme: the bytes are the stored numbers.
Unlike Python pickle/.bin checkpoints there's no embedded
code to execute — just data. Hence the name.
Map the file and point tensors straight at the bytes. (ds4 mmaps too;
the idea is the same.)
Those live in sibling files: config.json (layers, dims, heads, vocab)
and tokenizer.json (vocab + merges, our M0). We need those anyway.
llama.cpp's format · what ds4 uses
GGUF is built for local inference distribution. Its metadata can carry architecture parameters, tokenizer data, and a chat template, so a particular file can be self-contained—but the format does not guarantee every item is present.
| section | what it holds |
|---|---|
header |
magic "GGUF" (0x46554747) · version (u32)
· n_tensors · n_metadata_kv |
metadata (key/value) |
general.architecture, hyperparameters, and optionally tokenizer/vocab,
chat template, etc. (~13 typed value kinds, arrays supported) |
tensor info |
per tensor: name · n_dims · dims[] ·
ggml_type (tensor type) · offset |
padding |
pad to alignment |
tensor data |
quantized blocks (Q4_K, Q2_K, IQ2_XXS, …)
or f16/f32 |
The key differences from safetensors:
Common files include weights plus architecture and tokenizer information. Consumers must still check that the metadata keys they require are present.
Tensors may be ordinary f16/f32 arrays or block quant formats — a block
of N weights shares a scale (sometimes a min / secondary scale), packed into far fewer
bits. Q4_K ≈ 4 bits/weight; Q2_K/IQ2_XXS ≈ 2.
This is the reason a huge model fits a laptop — a conditional M7 topic.
A binary key/value metadata section with many value types, plus the per-format block layouts you must decode to get real numbers.
ds4 is GGUF, end to end.
ds4.c:5 — “This file is deliberately vertical: it owns GGUF loading…”ds4.c:11 — “Loading is mmap based. The loader parses only the GGUF
header, metadata…”ds4.c:579 — #define DS4_GGUF_MAGIC 0x46554747u /* "GGUF", little endian. */ds4.c:1514 — the GGUF_VALUE_* type enum (UINT8…FLOAT32…).download_model.sh pulls only *.gguf; gguf-tools/
builds/quantizes GGUF.ds4 has no safetensors path at all — it's a GGUF engine.
The tradeoff, for our goals
| safetensors | GGUF | |
|---|---|---|
| Qwen3-0.6B ships it? | ✅ natively | ❌ needs conversion (llama.cpp) |
| Parse-from-scratch effort | trivial (len + JSON + bytes) | meatier (binary KV + quant blocks) |
| Weights for a first forward pass | clean bf16, no decoding | fine if f16, but quant blocks loom |
| Includes tokenizer/config? | ❌ (sibling files) | metadata can include them; presence varies |
| Quantization | ordinary declared dtypes | quantized and unquantized types |
Matches ds4 directly | ❌ | ✅ |
Our decision (and why)
The reasoning is sequencing complexity to the milestone that needs it:
safetensors makes that idea visceral in an afternoon, and Qwen ships it natively — so there's no tooling detour before we've loaded a single tensor.
Real bf16 weights mean our first-ever forward pass isn't also fighting dequantization math while we hunt numerical bugs against the golden vector.
M7 starts with a benchmark-driven low-bit go/no-go. GGUF may later earn an
optional interoperability experiment beside ds4's parser and
gguf-tools/; it is not a core-roadmap promise.
ds4 remains a GGUF-only reference; adoption
by fs is not promised.
Mental model to keep
A model on disk = (numbers) + (a table describing the numbers).
The leap from the first to the second — “the bytes need decoding” — is the door into quantization, which is most of what makes local inference possible. We walk through that door only if the benchmark-driven M7 decision says it earns its keep.
To revisit at M7, if quantization is a go
Q4_K and a 2-bit format (Q2_K /
IQ2_XXS) by hand; compare block layouts against
gguf-tools/quants.c and ds4.c's block format section
(around ds4.c:312).shape and data_offsets map to real tensors.ds4 — the GGUF-only reference engine we compare against.docs/learnings/01-safetensors-vs-gguf.md.