Give Claude Access to the Web with MCP

Claude cannot browse the live web natively. By using the Model Context Protocol (MCP) and Ilmenite, you can implement claude mcp web browsing to give your AI assistant the ability to read pages, crawl sites, and extract structured data in real-time. This integration allows Claude to move beyond its training data and interact with current web content using a lightweight, Rust-based browser engine.

What we're building

We are configuring Claude Desktop to act as an MCP client that connects to the Ilmenite MCP server. This setup gives Claude a set of specialized tools—browser_scrape, browser_crawl, browser_map, browser_search, and browser_extract—which it can call autonomously to fetch web data. Instead of receiving messy HTML, Claude receives clean markdown, reducing token usage and improving the accuracy of its responses.

Prerequisites

To follow this tutorial, you will need:

• Claude Desktop installed on macOS or Windows.
• An Ilmenite API Key. You can sign up for a free account to get 500 credits.
• Node.js (v18 or higher) installed on your machine to run the MCP server.
• Basic familiarity with editing JSON configuration files.

Setting up Claude MCP Web Browsing

The Model Context Protocol (MCP) is an open standard that enables LLMs to interact with external tools and data sources. Because Ilmenite is built in pure Rust, the MCP server can trigger page renders with a 0.19ms cold start and use only 2MB of RAM per session. This ensures that Claude doesn't hang while waiting for a heavy Chrome instance to boot.

Step 1: Obtain your API Key

Log in to your Ilmenite dashboard and generate an API key. This key will be used by the MCP server to authenticate requests to the api.ilmenite.dev endpoints.

Step 2: Locate your Claude Desktop Config

Claude Desktop looks for a specific configuration file to determine which MCP servers to load. Depending on your operating system, the file is located at:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
• Windows: %APPDATA%\Claude\claude_desktop_config.json

If the file does not exist, create it.

Step 3: Configure the Ilmenite MCP Server

Open the claude_desktop_config.json file in a text editor. You need to add the Ilmenite server to the mcpServers object. We use npx to run the server directly from the registry, ensuring you always have the latest version.

Insert the following configuration, replacing YOUR_ILMENITE_API_KEY with your actual key:

{
  "mcpServers": {
    "ilmenite": {
      "command": "npx",
      "args": [
        "-y",
        "@ilmenite/mcp-server"
      ],
      "env": {
        "ILMENITE_API_KEY": "YOUR_ILMENITE_API_KEY"
      }
    }
  }
}

Step 4: Restart Claude Desktop

For the changes to take effect, you must fully quit Claude Desktop and restart it. Once restarted, you should see a small hammer icon (Tools) in the chat interface, indicating that the Ilmenite tools are active.

Using Ilmenite Tools for Claude MCP Web Browsing

Once the integration is active, you don't need to tell Claude which tool to use. You simply give it a task, and it decides which Ilmenite endpoint is most appropriate based on the request.

browser_scrape

This tool is used for single-page retrieval. When you ask Claude to "read this article," it calls the /v1/scrape endpoint.

How it works: The server sends the URL to Ilmenite, which renders the JavaScript (if necessary) and strips away navigation, ads, and cookie banners. Claude receives a clean markdown version of the main content.

Example Prompt: "Read the latest post from the Rust blog at https://blog.rust-lang.org and summarize the key changes."

browser_crawl

For tasks requiring multiple pages, Claude uses browser_crawl, which maps to the /v1/crawl endpoint. This is ideal for indexing documentation or researching a whole site.

How it works: Ilmenite starts at the provided URL and follows links up to a specified depth. Because our engine is 100x lighter than Chrome-based alternatives, crawling is faster and less prone to memory leaks.

Example Prompt: "Crawl the documentation at https://docs.example.com and tell me how to configure their authentication system."

browser_search

Instead of providing a specific URL, you can ask Claude to find information. This triggers the /v1/search endpoint.

How it works: Ilmenite performs a web search and then scrapes the top results, returning the combined markdown to Claude. This eliminates the need for you to manually find and paste links.

Example Prompt: "Search for the latest benchmarks comparing Axum and Actix-web and create a comparison table."

browser_extract

When you need specific data points rather than a summary, Claude uses browser_extract. This leverages the /v1/extract endpoint and a JSON schema.

How it works: You define what data you want (e.g., price, product name, SKU). Ilmenite uses an LLM-powered extraction layer to find those specific fields in the HTML and return a structured JSON object.

Example Prompt: "Go to this product page and extract the price, availability, and shipping weight into a JSON format."

Technical Implementation Example

While the MCP server handles the connection, it is helpful to understand the underlying API calls the server is making. If you were to implement a custom tool in Python instead of using the pre-built MCP server, your request to the scrape endpoint would look like this:

import requests

API_KEY = "YOUR_ILMENITE_API_KEY"
URL_TO_SCRAPE = "https://example.com"

headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json"
}

payload = {
    "url": URL_TO_SCRAPE,
    "format": "markdown",
    "js_render": True  # Required for React/Vue/Next.js sites
}

response = requests.post(
    "https://api.ilmenite.dev/v1/scrape", 
    json=payload, 
    headers=headers
)

if response.status_code == 200:
    print(response.json()['content'])
else:
    print(f"Error: {response.status_code}")

For a simple curl request to test your connection before configuring MCP:

curl -X POST https://api.ilmenite.dev/v1/scrape \
     -H "Authorization: Bearer YOUR_ILMENITE_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"url": "https://ilmenite.dev", "format": "markdown"}'

Full Working Configuration Summary

To ensure your setup is correct, your claude_desktop_config.json should look exactly like this (with your key):

{
  "mcpServers": {
    "ilmenite": {
      "command": "npx",
      "args": [
        "-y",
        "@ilmenite/mcp-server"
      ],
      "env": {
        "ILMENITE_API_KEY": "ilmn_live_xxxxxxxxxxxx"
      }
    }
  }
}

Next Steps

Now that you have enabled claude mcp web browsing, you can begin building more complex autonomous workflows.

• Explore Advanced Features: Read the MCP integration docs to learn how to optimize your prompts for better extraction.
• Test in the Browser: Use the playground to see how different websites are converted to markdown before asking Claude to process them.
• Scale Your Usage: If you are hitting the limits of the free tier, check our pricing for the Developer and Pro plans, which offer lower costs per credit and higher concurrency.