pinkpixel-dev-web-scout-mcp
Search the web and extract clean, readable text from webpages. Process multiple URLs at once to sp鈥�
可用工具 (5 个)
该服务在 MCP 协议中暴露的工具,AI 可按需调用
tavily_search 14 个参数 需填 1 项
Search the web for current information on any topic. Use for news, facts, or data beyond your knowledge cutoff. Returns snippets and source URLs.
必填参数:query
tavily_extract 6 个参数 需填 1 项
Extract content from URLs. Returns raw page content in markdown or text format.
必填参数:urls
tavily_crawl 11 个参数 需填 1 项
Crawl a website starting from a URL. Extracts content from pages with configurable depth and breadth.
必填参数:url
tavily_map 8 个参数 需填 1 项
Map a website's structure. Returns a list of URLs found starting from the base URL.
必填参数:url
tavily_research 2 个参数 需填 1 项
Perform comprehensive research on a given topic or question. Use this tool when you need to gather information from multiple sources to answer a question or complete a task. Returns a detailed response based on the research findings.
必填参数:input
服务介绍
# ✨ Features
- 🔍 DuckDuckGo Search: Fast and privacy-focused web search capability
- 📄 Content Extraction: Clean, readable text extraction from web pages
- 🚀 Parallel Processing: Support for extracting content from multiple URLs simultaneously
- 💾 Memory Optimization: Smart memory management to prevent application crashes
- ⏱️ Rate Limiting: Intelligent request throttling to avoid API blocks
- 🛡️ Error Handling: Robust error handling for reliable operation
# 📦 Installation
# # Installing via Smithery
To install Web Scout for Claude Desktop automatically via Smithery:
npx -y @smithery/cli install @pinkpixel-dev/web-scout-mcp - -client claude
# # Global Installation
npm install -g @pinkpixel/web-scout-mcp
# # Local Installation
npm install @pinkpixel/web-scout-mcp
# 🚀 Usage
# # Command Line
After installing globally, run:
web-scout-mcp
# # With MCP Clients
Add this to your MCP client's config.json (Claude Desktop, Cursor, etc.):
{
"mcpServers": {
"web-scout": {
"command": "npx",
"args": [
"-y",
"@pinkpixel/web-scout-mcp@latest"
]
}
}
}
# # Environment Variables
Set the WEB_SCOUT_DISABLE_AUTOSTART=1 environment variable when embedding the package and calling createServer() yourself. By default running the published entrypoint (for example node dist/index.js or npx @pinkpixel/web-scout-mcp) automatically bootstraps the stdio transport.
# 🧰 Tools
The server provides the following MCP tools:
# # 🔍 DuckDuckGoWebSearch
Initiates a web search query using the DuckDuckGo search engine and returns a well-structured list of findings.
Input:
query(string): The search query stringmaxResults(number, optional): Maximum number of results to return (default: 10)
Example:
{
"query": "latest advancements in AI",
"maxResults": 5
}
Output:
A formatted list of search results with titles, URLs, and snippets.
# # 📄 UrlContentExtractor
Fetches and extracts clean, readable content from web pages by removing unnecessary elements like scripts, styles, and navigation.
Input:
url: Either a single URL string or an array of URL strings
Example (single URL):
{
"url": "https://example.com/article"
}
Example (multiple URLs):
{
"url": [
"https://example.com/article1",
"https://example.com/article2"
]
}
Output:
Extracted text content from the specified URL(s).
# 🛠️ Development
# Clone the repository
git clone https://github.com/pinkpixel-dev/web-scout-mcp.git
cd web-scout-mcp
# Install dependencies
npm install
# Build
npm run build
# Run
npm start
# 📚 Documentation
For more detailed information about the project, check out these resources:
- OVERVIEW.md - Technical overview and architecture
- CONTRIBUTING.md - Guidelines for contributors
- CHANGELOG.md - Version history and changes
# 📋 Requirements
- Node.js >= 18.0.0
- npm or yarn
# 📄 License
This project is licensed under the Apache 2.0 License.