agentutility(7)$ man scrape-to-json
/scrape-to-json
PRICE / CALL
$0.02
USDC · base mainnet · scheme: exact
METHOD
POST
CLUSTER
webprobeCATEGORY
uncategorized
STATUS
● live
NAME
scrape-to-json — url to structured json / web scraper with schema enforcement / scrape and extract / website to json / page to structured data / scrape wi…
SYNOPSIS
POST https://x402.agentutility.ai/scrape-to-json
Content-Type: application/json
X-PAYMENT: <signed-transferWithAuthorization>
{ ... }↳ first call →
402 Payment Required. Sign USDCtransferWithAuthorization, retry with theX-PAYMENT header.DESCRIPTION
URL to structured JSON / web scraper with schema enforcement / scrape and extract / website to JSON / page to structured data / scrape with JSON schema / Morpheus-powered extraction. Fetches a page server-side (cheerio, no headless), converts to clean markdown, then forces the output to a caller-supplied JSON schema via Morpheus mistral-31-24b. Pair with watch-page or research-brief for richer agent pipelines.
INPUT — request schema
| property | type | description | req? |
|---|---|---|---|
| url | string | Public URL to scrape (http or https). Follows redirects. | required |
| schema | object | JSON schema describing expected output. Must include a 'properties' key. | required |
| instructions | string | Optional extra extraction guidance. | optional |
| format | string | Pre-LLM body normalization. Default 'markdown'. enum: text · markdown | optional |
| user_agent | string | — | optional |
OUTPUT — response shape
| field | type | description |
|---|---|---|
| url | string | — |
| final_url | string | — |
| status_code | string | — |
| extracted | string | — |
| model | string | — |
EXAMPLES — two ways to call
EXAMPLE 1 · curl
curl -X POST https://x402.agentutility.ai/scrape-to-json \
-H 'Content-Type: application/json' \
-d '{ }'first response =
402 Payment Required with payment requirements; sign + retry with X-PAYMENT.EXAMPLE 2 · mcp
# Install the MCP package for this endpoint's cluster npx -y @agentutility/mcp-<cluster> # Required: EVM private key with USDC on Base export X402_PRIVATE_KEY=0x... # Then call the scrape-to-json tool from your MCP-aware agent.
MCP server handles payment automatically — your coding agent just calls the tool by name.
METADATA
- tags
- web-probescrapetojsonscrape-to-json
- methods
- POST
- cluster
- webprobe
- price
- $0.02 USDC per call
ADJACENT — other endpoints in webprobe
| endpoint | description | price |
|---|---|---|
| archive-snapshot | Archive snapshot lookup / Wayback Machine API / web archive URL lookup / historical webpage snapshot. | $0.02 |
| archive-url | Archive URL lookup / web archive URL builder / Wayback Machine snapshot finder. | $0.02 |
| dmarc-check | DMARC + SPF + DKIM email-authentication checker / mail-auth posture audit / spoofing-resistance grader / phishing-readiness scanner / dom… | $0.02 |
| dns-lookup | DNS lookup / DNS resolver / dig replacement / nslookup API / DoH proxy / authoritative-record fetch / MX checker / SPF + DMARC TXT inspec… | $0.02 |
| expand-url | URL expander / redirect chain tracer / link unshortener / shortlink resolver / bit.ly + t.co + lnkd.in + goo.gl + tinyurl resolver / fina… | $0.02 |
| password-strength | Password strength meter / password checker / password entropy calculator / zxcvbn-style score / HIBP breach lookup / pwned password check… | $0.02 |
| wayback-machine | Wayback Machine API / archive.org snapshot lookup / historical URL capture finder. | $0.02 |
| web-archive | Web archive lookup / Wayback Machine API / historical webpage snapshot. | $0.02 |
SEE ALSO