DHT Binding

What this page covers

WeftDhtStore<K> plugs Weft into any key-value store that implements the DhtKv trait. In production, the binding is to weave-dht (libp2p Kademlia). For tests and local development, an InMemoryDhtKv is provided.

The DhtKv trait

#[async_trait]
pub trait DhtKv: Send + Sync {
    async fn put(&self, key: Vec<u8>, value: Vec<u8>) -> Result<()>;
    async fn get(&self, key: Vec<u8>) -> Result<Option<Vec<u8>>>;
}

Two operations, both content-addressed. The store does not need to enforce ordering, replication policy, or quorum — those are concerns of the underlying DHT.

Key layout

WeftDhtStore writes three kinds of records, each under a domain-prefixed key:

Record	Key	Value
Manifest bytes	`WeftDhtStore::manifest_key(root)`	Canonical manifest bytes
Chunk bytes	`WeftDhtStore::chunk_key(chunk_id)`	Raw chunk bytes
Availability record	`WeftDhtStore::availability_key(root)`	Encoded `Vec<AvailabilityAdvertisement>`

Domain prefixes prevent key collisions with other Weave subsystems (Strand, Lens, Locus) that share the same DHT.

Publishing into the DHT

use std::sync::Arc;
use weft::{
    chunk::{ChunkConfig, DEFAULT_CHUNK_SIZE},
    dht::{InMemoryDhtKv, WeftDhtStore},
    discovery::PeerId,
    manifest::{ArtifactMetadata, ManifestProducer},
};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let kv = Arc::new(InMemoryDhtKv::default());
    let store = WeftDhtStore::new(kv.clone());

    let payload = b"hello dht".to_vec();
    let cfg     = ChunkConfig::new(DEFAULT_CHUNK_SIZE)?;
    let peer    = PeerId::new("did:peer:demo")?;

    let producer = ManifestProducer {
        did: "did:oas:agent:demo".into(),
        signing_key_id: None,
        signature: vec![0u8; 64],
    };
    let artifact = ArtifactMetadata {
        artifact_id:          None,
        media_type:           "text/plain".into(),
        schema:               "weave.text.v1".into(),
        mars_asset_id:        None,
        sigil_commitment_cid: None,
    };

    let published = store
        .publish_bytes(&payload, cfg, producer, artifact, peer)
        .await?;

    println!("manifest and chunks published under root {}", published.manifest.root);
    Ok(())
}

WeftDhtStore::publish_bytes performs exactly the same chunking and manifest construction as LocalWeftStore::publish_bytes and then writes:

Every chunk's bytes under chunk_key(chunk.id).
The manifest bytes under manifest_key(root).
The availability advertisement (as a one-element Vec) under availability_key(root).

Fetching from the DHT

use weft::WeftHash;
use weft::dht::WeftDhtStore;

async fn fetch(store: &WeftDhtStore<impl weft::dht::DhtKv + 'static>, root_hex: &str) -> weft::Result<Vec<u8>> {
    let root = WeftHash::from_hex(root_hex)?;
    let max_parallelism = 8;
    store.fetch_payload(root, max_parallelism).await
}

fetch_payload looks up the manifest by manifest_key(root), then issues parallel get calls for each chunk, verifying every returned blob against its expected id before assembling.

Wiring to weave-dht

The production binding is in the weave-dht crate. Wrap a weave_dht::Node in a thin adapter that implements DhtKv:

use async_trait::async_trait;
use std::sync::Arc;
use weft::Result;

pub struct WeaveDhtAdapter {
    node: Arc<weave_dht::Node>,
}

#[async_trait]
impl weft::dht::DhtKv for WeaveDhtAdapter {
    async fn put(&self, key: Vec<u8>, value: Vec<u8>) -> Result<()> {
        self.node
            .put(&key, &value)
            .await
            .map_err(|e| weft::WeftError::Discovery(e.to_string()))
    }

    async fn get(&self, key: Vec<u8>) -> Result<Option<Vec<u8>>> {
        self.node
            .get(&key)
            .await
            .map_err(|e| weft::WeftError::Discovery(e.to_string()))
    }
}

Once an adapter exists, WeftDhtStore::new(Arc::new(adapter)) gives you a full Weft publish/fetch path over the live DHT.

Availability handling

A chunk's chunk_key(id) value is content-addressed — any peer can serve it. The availability record at availability_key(root) exists so callers can find peers who already cache the chunks (the DHT itself answers "who has this key", but high-volume producers may want explicit announcements).

Lookup	Use
`chunk_key(id)`	"Who has this chunk?" — answered by the DHT's provider records or direct gets
`manifest_key(root)`	"Where is the manifest?" — root → manifest bytes
`availability_key(root)`	"Who advertises serving this root?" — root → producer advertisements

Failure modes

Failure	What happens	Recovery
DHT `get` returns `None` for a chunk	`fetch_payload` errors out on that chunk	Retry or fall back to a different peer set
Manifest stored, no chunks ever published	`fetch_payload` discovers missing chunks	Re-publish from the producer
Adapter not linked (no DhtKv impl)	Compile-time error	Link a `DhtKv` implementation before constructing the store
TTL expiry on a DHT key	Record disappears	Producer republishes via `publish_bytes`

On this page