Fetch and Parse

Workflow and current contract

FetchOptions contains include_raw, extract_links, extract_images, and css_selector. It does not have the timeout_ms or user_agent fields. fetch_url currently sets a 30-second HTTP client timeout and the crate user agent internally.

The result includes URL, HTTP status, content type, title, description, extracted text, and word count; optional extraction adds raw HTML, links, images, or selected elements. The parser functions take &scraper::Html, and link/image extraction also needs a base URL. Pass Html::parse_document output rather than a raw &str.

The convenience fetch path treats weave:// through its local-site fallback; use the protocol resolver for node-backed Weave routing.

Source reference

• fetch declarations — libs/weave-browser-engine/src/fetch.rs.
• parse declarations — libs/weave-browser-engine/src/parse.rs.