Web operators

Public

The Web family contains 3 operators. HTTP fetch + article scraping.

Operators in this family

web.fetch
web.scrape
web.search

`web.fetch`

Fetches content from a URL.

Version 1.0.0 · Semantics side_effect · Visibility advanced

Config

Field	Type	Required	Default	Description
`extract_mode`	”html” \| “text” \| “markdown”	no	`"text"`
`timeout_ms`	integer	no	`30000`

Input ports

Port	Type	Required	Description
`url`	string	yes	URL

Output ports

Port	Type	Required	Description
`content`	string	no	Content
`metadata`	object	no	Metadata

`web.scrape`

Fetches a URL through the SSRF-guarded HTTP client and extracts the main article body in markdown / text / html, plus lightweight metadata (author, description, published, site).

Version 1.0.0 · Semantics side_effect · Visibility advanced

Config

Field	Type	Required	Default	Description
`format`	”markdown” \| “text” \| “html”	no	`"markdown"`	Output format for the extracted article body.
`include_metadata`	boolean	no	`true`	Include <meta>-tag metadata (author, description, published, site_name) in the output.
`url`	string	no	—	Absolute URL to fetch. Falls back to the upstream ‘url’ input port if omitted.

Input ports

Port	Type	Required	Description
`url`	string	no	URL

Output ports

Port	Type	Required	Description
`output`	object	no	Article

`web.search`

Searches the web using a search API.

Version 1.0.0 · Semantics side_effect · Visibility advanced · Secrets SEARCH_API_KEY

Config

Field	Type	Required	Default	Description
`engine`	”google” \| “bing” \| “serper”	no	`"serper"`
`num_results`	integer	no	`5`

Input ports

Port	Type	Required	Description
`query`	string	yes	Query

Output ports

Port	Type	Required	Description
`results`	array	no	Results

Source schemas

Every operator above is rendered from its JSON schema in /opt/glyph/schemas/operators/:

schemas/operators/web.fetch.json
schemas/operators/web.scrape.json
schemas/operators/web.search.json