Publish

What this page covers

Publishing is the producer-side flow: take a payload, split it into chunks, build a WeftManifest, sign it, and write the chunks + manifest + availability into a LocalWeftStore.

Tip

Publishing is idempotent. Re-publishing the same bytes with the same ChunkConfig and producer DID yields the same root and the same chunk ids — re-publishes are free and safe to retry.

Inputs to publish

Input	Where it comes from
Payload bytes	Producer-supplied
`ChunkConfig`	Producer chooses chunk size; defaults to 4 MiB
`ManifestProducer`	Producer DID + signature key
`ArtifactMetadata`	Producer-supplied media type, schema, optional MARS asset id
`PeerId`	The peer that is advertising availability

Minimal local publish

use weft::{
    chunk::{ChunkConfig, DEFAULT_CHUNK_SIZE},
    discovery::PeerId,
    local::LocalWeftStore,
    manifest::{ArtifactMetadata, ManifestProducer},
};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // 12 MiB of payload — three default-sized chunks plus a final short chunk.
    let payload = vec![0u8; 12 * 1024 * 1024 + 17];

    let store = LocalWeftStore::in_memory();
    let cfg   = ChunkConfig::new(DEFAULT_CHUNK_SIZE)?;
    let peer  = PeerId::new("did:peer:demo")?;

    let producer = ManifestProducer {
        did: "did:oas:agent:producer-1".into(),
        signing_key_id: Some("did:oas:agent:producer-1#key-1".into()),
        // In production, sign manifest.signing_payload() with your Ed25519 key
        // and place the 64-byte signature here.
        signature: vec![0u8; 64],
    };

    let artifact = ArtifactMetadata {
        artifact_id:        Some("dataset-2026-05-14".into()),
        media_type:         "application/x-zip".into(),
        schema:             "weave.dataset.v1".into(),
        mars_asset_id:      None,
        sigil_commitment_cid: None,
    };

    let published = store.publish_bytes(&payload, cfg, producer, artifact, peer)?;

    println!("root         = {}", published.manifest.root);
    println!("chunk_count  = {}", published.manifest.chunk_count);
    println!("chunk_size   = {}", published.manifest.chunk_size);
    println!("total_len    = {}", published.manifest.total_len);
    println!("ad ttl       = {}s", published.advertisement.ttl_secs);

    Ok(())
}

LocalWeftStore::publish_bytes performs:

chunk_payloads(bytes, config) — splits into ordered chunks, computes BLAKE3 ids.
WeftManifest::from_bytes(...) — builds the manifest, validates internal consistency, computes the Merkle root over chunk ids.
manifest.to_canonical_bytes() — deterministic serialization of the manifest.
AvailabilityAdvertisement::new(...) — advertisement covering all chunks, default 24 h TTL.
Writes chunks, manifest bytes, and advertisement into the store (memory + optionally disk).

Publishing from a reader

For inputs that do not fit in memory:

use std::fs::File;
use weft::chunk::{ChunkConfig, DEFAULT_CHUNK_SIZE};
use weft::discovery::PeerId;
use weft::local::LocalWeftStore;
use weft::manifest::{ArtifactMetadata, ManifestProducer};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let file = File::open("/data/model.safetensors")?;
    let store = LocalWeftStore::open("/var/lib/weft-store")?;

    let cfg = ChunkConfig::new(DEFAULT_CHUNK_SIZE)?;
    let producer = ManifestProducer {
        did: "did:oas:agent:trainer-3".into(),
        signing_key_id: None,
        signature: vec![0u8; 64], // replace with a real Ed25519 signature
    };
    let artifact = ArtifactMetadata {
        artifact_id:          Some("model-v0.7".into()),
        media_type:           "application/octet-stream".into(),
        schema:               "weave.model.weights.v1".into(),
        mars_asset_id:        None,
        sigil_commitment_cid: None,
    };

    let published = store.publish_reader(file, cfg, producer, artifact, PeerId::new("did:peer:demo")?)?;
    println!("published {} chunks under root {}", published.manifest.chunk_count, published.manifest.root);
    Ok(())
}

publish_reader streams chunks through Chunker::next_chunk so it does not buffer the full payload.

Signing the manifest

The unsigned manifest is what your key signs. Use the signing_payload() helper:

use weft::manifest::WeftManifest;

fn sign(manifest: &WeftManifest, secret_key: &ed25519_dalek::SigningKey) -> weft::Result<Vec<u8>> {
    use ed25519_dalek::Signer;
    let payload = manifest.signing_payload()?;
    let sig = secret_key.sign(&payload);
    Ok(sig.to_bytes().to_vec())
}

The signing_payload() call clears the signature field in a clone, serializes the result canonically, and returns the bytes. Replace producer.signature on the original manifest with the produced bytes before publishing.

Choosing a chunk size

Chunk size	Trade-off
1 MiB	More chunks, more Merkle depth, more parallelism, more discovery overhead
4 MiB (default)	Balanced for most workloads
16 MiB	Fewer chunks, fewer fetches; each retry costs more bytes
64 MiB (max)	Useful only for very large pre-staged artifacts

ChunkConfig::new validates that the size is > 0 and ≤ MAX_CHUNK_SIZE.

Errors

Error	Cause	Recovery
`WeftError::Manifest("chunk_size … out of bounds")`	`ChunkConfig` value invalid	Use `DEFAULT_CHUNK_SIZE` or a value `≤ MAX_CHUNK_SIZE`
`WeftError::Manifest("producer signature is required")`	Empty signature	Sign the manifest payload before publishing
`WeftError::Manifest("artifact media_type and schema are required")`	Empty strings	Supply both fields
`WeftError::Discovery("ttl_secs must be positive")`	Zero TTL on advertisement	Use the default 24 h or a positive value
`WeftError::Io(...)`	Disk write failed (only for filesystem-backed stores)	Free disk; the in-memory state may be consistent but the on-disk copy is incomplete — re-publish