Repair and GC
Repair and garbage collection for Strand Blobs: reference counting, orphan detection, and policy-aware pruning of unreachable chunks.
Purpose
Chunked blob storage with deduplication, cache, writer/reader APIs, repair, metrics, and policy hooks.
This page follows the real source shape for Strand Blobs and explains the workflow a developer is likely to use first.
Developer workflow
Start from the smallest constructor or builder, perform one meaningful operation, inspect the returned state, then add the relevant policy, storage, or network integration. The examples below should be expanded whenever the crate API changes.
Garbage collection is irreversible. Once a chunk's refcount drops to zero and the GC pass deletes it, the bytes are gone from this peer — recovery requires re-fetching from another mirror or re-publishing from the source. Always run a dry-run pass before enabling automatic GC in production.
Primary types to know
AllowAllBlobPolicy— models/strand-blobs/src/policy.rsBlobConfig— models/strand-blobs/src/config.rsBlobEntry— models/strand-blobs/src/chunk.rsBlobId— models/strand-blobs/src/blob.rsBlobInfo— models/strand-blobs/src/blob.rsBlobMetadata— models/strand-blobs/src/blob.rsBlobMetrics— models/strand-blobs/src/metrics.rsBlobMetricsSnapshot— models/strand-blobs/src/metrics.rsBlobReader— models/strand-blobs/src/reader.rsBlobStore— models/strand-blobs/src/store.rsBlobWriter— models/strand-blobs/src/writer.rsChunkCache— models/strand-blobs/src/cache.rs
Example shape
use strand::{Strand, StrandConfig};
use strand_blobs::{BlobConfig, BlobStore};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let dir = tempfile::tempdir()?;
let meta = Strand::new(StrandConfig::new().with_storage(dir.path().join("meta"))).await?;
let data = Strand::new(StrandConfig::new().with_storage(dir.path().join("data"))).await?;
let store = BlobStore::new(meta, data, BlobConfig::default()).await?;
// Stage a blob, then delete it; deletion is logical and the chunks remain
// until GC runs.
let id = store.put(b"transient".as_ref(), None).await?;
store.delete(&id).await?;
// gc_sweep computes a GcPlan that identifies orphan chunks. gc_apply is the
// separate, mutating pass that actually frees them.
let plan = store.gc_sweep().await?;
let unreachable = plan.unreachable.len();
store.gc_apply(&plan).await?;
// repair_reindex walks both strands and rebuilds the chunk index, surfacing
// any blobs whose data chunks are missing or quarantined.
let report = store.repair_reindex().await?;
println!(
"gc_unreachable={} gc_freed_bytes={} repair_missing={} repair_quarantined={}",
unreachable,
plan.bytes_to_free,
report.missing.len(),
report.quarantined.len(),
);
Ok(())
}