Learning 06 — mmap — Failed Star

Learning 06 · turning a file into memory

mmap: the file is the backing store

Context: M1, reading model.safetensors · Status: living

To load the weights we had a choice: read the 1.4 GB model.safetensors into a Vec<u8>, or mmap it. We chose mmap — and because the project's rule is no hidden abstraction, we call it with raw POSIX FFI, no libc crate. This note is what that means and why.

This note is purely about getting the bytes into our address space. See Learning 01 for the file format and Learning 05 for the tensor shapes.

📖 Inference Engineering · §4.2.2 (model loading, p.103) 🔧 ds4 · “Loading is mmap based” (ds4.c:11) — same idea, in C 🧭 POSIX mmap(2) / munmap(2) man pages

Copy vs map

What `mmap` actually does

std::fs::read copies: the kernel reads the file into a buffer you own — all 1.4 GB, up front, into your heap. mmap does something different: it maps the file into your virtual address space and hands you a pointer. No bytes are copied yet. Pages are pulled in lazily, on first touch (a page fault), and the OS can drop clean pages under memory pressure and re-read them from the file later.

file on disk heap copy (eager) mapped into address space (lazy)

read eagerly duplicates every byte into the heap; mmap hands back a pointer and lets the OS page the file in only where you touch it.

std::fs::read (eager copy)

Reads the whole file into a heap buffer up front. You own the bytes — and pay 1.4 GB of resident memory whether you touch them or not.

mmap (lazy map)

Maps the file, copies nothing. Pages fault in on first touch and the OS can reclaim clean ones. The file is the backing store.

For read-only weights this is ideal:

Zero-copy

Our Tensors are just &[u8] slices into the mapping (see SafeTensors::bytes). We never duplicate a weight — this is exactly what makes the “lazy bf16” decision work: nothing is materialized until M2 reads it.

Lazy + reclaimable

Touch only the tensors you use; the OS handles paging, dropping and re-reading clean pages under pressure.

It's what ds4 does

The mmap-based loading strategy transfers straight to the bigger engine, so the mental model carries forward.

The cost. A mapping is unsafe to hold — a raw pointer with a lifetime the compiler can't see — and the file shouldn't change underneath us. We contain that unsafety in one small Mmap type (below).

Argument by argument

The POSIX call

void *mmap(void *addr, size_t len, int prot, int flags, int fd, off_t offset);
int   munmap(void *addr, size_t len);

For mapping a whole file read-only we pass:

arg	we pass	why
`addr`	`NULL`	let the kernel choose the address
`len`	file size	map the entire file (from `file.metadata().len()`)
`prot`	`PROT_READ` (`0x1`)	pages may be read, not written
`flags`	`MAP_PRIVATE` (`0x2`)	copy-on-write, private to us — we only read
`fd`	the file's descriptor	which file to map (`File::as_raw_fd()`)
`offset`	`0`	start at the beginning of the file

Two gotchas worth burning in.

Failure is MAP_FAILED, not null. mmap returns (void *) -1 on error (usize::MAX as *mut c_void). Checking for null would miss every failure.
A zero-length file can't be mapped (len must be > 0) — we error out before calling.

After a successful map, the file descriptor can be closed: the mapping keeps its own reference to the file, so the bytes stay valid until we munmap.

No libc crate

Calling it from Rust

libc is just declarations for calls the kernel already provides, so we write those declarations ourselves. In edition 2024 an FFI block is unsafe extern:

use std::ffi::{c_int, c_void};

unsafe extern "C" {
    fn mmap(addr: *mut c_void, len: usize, prot: c_int,
            flags: c_int, fd: c_int, offset: i64) -> *mut c_void;
    fn munmap(addr: *mut c_void, len: usize) -> c_int;
}

const PROT_READ:   c_int = 0x1;
const MAP_PRIVATE: c_int = 0x2;
const MAP_FAILED: *mut c_void = usize::MAX as *mut c_void;

(offset is off_t, which is 64-bit on macOS/arm64, so i64.)

Containing the unsafety

One RAII wrapper

All the danger lives in a small owner that maps on construction and unmaps on Drop — so cleanup is automatic and can't be forgotten:

struct Mmap { ptr: *const u8, len: usize }

impl Mmap {
    fn as_bytes(&self) -> &[u8] {
        // SAFETY: ptr points at len valid, read-only bytes for self's lifetime.
        unsafe { std::slice::from_raw_parts(self.ptr, self.len) }
    }
}

impl Drop for Mmap {
    fn drop(&mut self) {
        // SAFETY: ptr/len came from a successful mmap; unmapped exactly once.
        unsafe { munmap(self.ptr as *mut c_void, self.len); }
    }
}

Why this is sound to expose safely:

the mapping is read-only, so &[u8] can't be used to mutate it;
the slice borrows self, so it can't outlive the mapping (no use-after-unmap);
munmap runs exactly once, when the Mmap is dropped — RAII, same shape as C's “map then munmap,” but the compiler enforces the cleanup.

SafeTensors then owns one Mmap and hands out tensor slices into it. The raw pointer never escapes; everything above this type is safe Rust.

Cross-links

⬅ Learning 01 · safetensors vs GGUF — the byte layout we're mapping ([u64 len][JSON header][blob]).
🔗 Learning 05 · reading shapes — what the mapped bytes mean once we slice them into tensors.
🔗 Learning 07 · bf16 — the 2-byte weights we keep raw in the mapping instead of materializing to f32.
🔧 src/safetensors.rs — the Mmap wrapper + SafeTensors reader this note documents.
🔧 ds4.c — the C version of the same mmap-based loading strategy. · Full note: docs/learnings/06-mmap.md.

mmap: the file is the backing store

What mmap actually does

std::fs::read (eager copy)

mmap (lazy map)

Zero-copy

Lazy + reclaimable

It's what ds4 does

The POSIX call

Calling it from Rust

One RAII wrapper

Cross-links

What `mmap` actually does