网页内容抓取
一个基于MCP协议的网页内容抓取服务器，支持将网页内容提取为干净的Markdown格式。它支持三层抓取策略、HTML转Markdown、图片保留、遵守robots.txt规则、双传输模式、代理支持和Cookie支持。
MCP 服务配置
复制以下 JSON 到 OPClaw 或其他 MCP 客户端的配置文件中即可使用
{
"mcpServers": {
"mcp-fetch-plus": {
"args": [
"mcp-fetch-plus",
"--ignore-robots-txt"
],
"command": "uvx"
}
}
}可用工具 （1 个）
该服务在 MCP 协议中暴露的工具，AI 可按需调用
fetch 5 个参数 需填 1 项
Fetches a URL from the internet and optionally extracts its contents as markdown. This tool provides internet access — fetch the most up-to-date information from web pages.
必填参数：url
服务介绍
mcp-server-fetch
A Model Context Protocol (MCP) server for fetching web content and converting it to clean Markdown.
一个基于 MCP 协议的网页内容抓取服务器，支持将网页内容提取为干净的 Markdown 格式。
Features / 功能特性
- Three-tier fetching strategy / 三层抓取策略:
- L1: httpx (fast, lightweight) — works without a browser
- L2: Playwright headless browser — for JavaScript-rendered pages
- L3: Playwright + stealth mode — bypasses anti-bot detection
- HTML to Markdown / HTML 转 Markdown: Extracts readable content and converts to Markdown
- Image preservation / 图片保留: Keeps image URLs in extracted content
- robots.txt compliance / robots.txt 遵守: Respects site crawling rules by default
- Dual transport / 双传输模式: Supports both STDIO and SSE transports
- Proxy support / 代理支持: HTTP/HTTPS proxy configuration
- Cookie support / Cookie 支持: Load cookies from JSON file for authenticated access
Installation / 安装
pip
# Basic installation (httpx-only, no browser needed)
pip install git+https://github.com/petal111/fetch_mcp.git
# With browser support (requires Chromium)
pip install "git+https://github.com/petal111/fetch_mcp.git[browser]"
playwright install chromium --with-deps
uvx
uvx --from git+https://github.com/petal111/fetch_mcp.git mcp-server-fetch
CLI Arguments / 命令行参数
| Argument | Default | Description |
|---|---|---|
--transport |
stdio |
Transport mode: stdio or sse |
--port |
8080 |
Port for SSE mode |
--ignore-robots-txt |
false |
Ignore robots.txt restrictions |
--user-agent |
Chrome UA | Custom User-Agent string |
--proxy-url |
None | Proxy URL for requests |
--stealth |
false |
Enable Playwright stealth mode |
--cookies |
None | Path to cookies JSON file |
MCP Tool API
fetch Tool
Fetches a URL and extracts its contents as Markdown.
Parameters:
| Parameter | Type | Default | Description |
|---|---|---|---|
url |
string (URI) | required | URL to fetch |
max_length |
integer | 5000 |
Maximum characters to return |
start_index |
integer | 0 |
Start content from this character index |
raw |
boolean | false |
Return raw content without Markdown conversion |
force_browser |
boolean | false |
Force use of headless browser |
fetch Prompt
A prompt template that fetches a URL and returns its content.
Arguments:
| Argument | Required | Description |
|---|---|---|
url |
Yes | URL to fetch |
Deployment on ModelScope MCP Hub / 魔搭 MCP 广场部署
This server supports hosted deployment on ModelScope MCP Hub.
| Config | Value |
|---|---|
| Protocol | STDIO |
| Install | pip install git+https://github.com/petal111/fetch_mcp.git |
| Start command | mcp-server-fetch |
Note: The hosted (Serverless) environment does not include Chromium, so only the httpx (L1) strategy is available. For full browser support, use Docker deployment.
Docker Deployment / Docker 部署
For full three-tier fetching support (including Playwright):
docker build -t mcp-server-fetch .
docker run -p 8080:8080 mcp-server-fetch
The container runs in SSE mode on port 8080 with Chromium pre-installed.
License
MIT Copyright 2026 petal111
