Scrape

Fetch one URL, render it, and return clean output in the format your agent or pipeline wants. The most common endpoint — it's what most users call first.

Endpoint

POST https://api.ilmenite.dev/v1/scrape

Request body

• url (string, required) — absolute URL to fetch.
• formats (string[], optional) — any of markdown, html, json, links, text. Defaults to ["markdown"].
• wait_for (string, optional) — CSS selector to wait for before extracting.
• timeout_ms (number, optional) — max fetch time. Default 15000, max 60000.
• js (boolean, optional) — evaluate JavaScript. Default true.
• headers (object, optional) — custom request headers.
• proxy (string, optional) — residential, datacenter, or a proxy URL.

Example

curl -X POST https://api.ilmenite.dev/v1/scrape \
  -H "Authorization: Bearer $ILMENITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://news.ycombinator.com",
    "formats": ["markdown", "links"],
    "wait_for": "tr.athing"
  }'

Response

{
  "url": "https://news.ycombinator.com",
  "status": 200,
  "title": "Hacker News",
  "markdown": "# Hacker News\n\n1. ...",
  "links": ["https://ycombinator.com", "..."],
  "metadata": {
    "lang": "en",
    "description": "...",
    "published_at": null
  },
  "latency_ms": 412,
  "render_time_ms": 0.19
}

Errors

• 400 invalid_url — URL failed parsing or scheme is unsupported.
• 408 timeout — page did not finish within timeout_ms.
• 422 wait_for_not_found — selector never appeared.
• 502 upstream_error — origin returned 5xx or refused connection.