Weft
Sharded large-blob distribution model — BLAKE3-addressed chunks under a deterministic Merkle root.
What Weft is
Weft is the off-chain data model Weave uses to distribute large immutable payloads — training datasets, model weights, video, image archives — to many peers without putting bytes on-chain. Each payload is split into fixed-size chunks, each chunk is named by its BLAKE3 hash, and the chunks are committed under a single Merkle root.
The root is the only identifier consumers need. Given the root and one peer that can serve the manifest, any peer can verify and reconstruct the original bytes.
When to use Weft
| Workload | Weft fit |
|---|---|
| Distributing model weights, datasets, images, video | Strong |
| Multi-gigabyte content addressed once, served many times | Strong |
| Tiny mutable values (KV) | Use Lens |
| Append-only logs | Use Strand |
| Filesystem-shaped content | Use Locus |
| Live streaming media | Out of scope (Weft is for static commitments) |
How Weft fits the stack
Sigil chain commits the Weft root
│
▼
WeftManifest (signed, content-addressed)
│
▼
Ordered chunks (BLAKE3 ids, 4 MiB default)
│
▼
Peer storage: LocalWeftStore | WeftDhtStore<K>
│
▼
Discovery: DiscoveryPublisher trait
│
▼
Network: weave-dht, libp2p adaptersWeft owns the data model and the verification rules. Live network transport is provided by adapters above this crate.
Key invariants
- Chunks are content-addressed. A chunk's id is
BLAKE3(domain || bytes), wheredomainis the constantweave/weft/chunk/v1\n. Any byte change produces a new id. - Manifests are deterministic. Given the same bytes and the same
ChunkConfig, two producers compute the same root. - Manifests are signed. The producer's DID and signature are part of the manifest. The signing payload is the manifest bytes with the signature field cleared.
- Parity is reserved. A
ParitySpecfield exists for future erasure coding; the current build fails closed ifparityisSome(_).
At a glance
| Field | Value |
|---|---|
| Crate/package | weave-weft |
| Version | 0.1.0 |
| Source | models/weft |
| Default chunk size | 4 MiB (DEFAULT_CHUNK_SIZE) |
| Max chunk size | 64 MiB (MAX_CHUNK_SIZE) |
| Hash | BLAKE3 32-byte, hex-encoded |
| Manifest schema | weave.weft.manifest.v1 |
| Discovery protocol | weave.weft.availability.v1 |
| Async | tokio (fetch path); sync (publish path) |
Page map
Chunk, hash, and sign a corpus.
Verifiably retrieve chunks from peers.
Manifest format and signature surface.
Bind Weft to the Weave DHT.
Anchor manifest roots on Sigil.
Complete ledger of exported types, traits, and functions.
Source modules
src/chunk.rs— chunking and chunk IDssrc/tree.rs— Merkle tree and proofssrc/manifest.rs—WeftManifest, producer signaturessrc/local.rs—LocalWeftStoresrc/dht.rs—WeftDhtStore<K>andDhtKvsrc/fetch.rs—fetch_manifest,ChunkSourcesrc/discovery.rs— availability advertisements,DiscoveryPublisher
Spec
See specs/WEFT.md in the Weave repository for wire semantics and readiness constraints.