mcp-server-fetch-rag

MCP Server Fetch RAG

A Model Context Protocol (MCP) server that fetches web content and returns relevant chunks using RAG (Retrieval-Augmented Generation).

GitHub: https://github.com/attamari/mcp-server-fetch-rag

Features

• Semantic Chunking: Groups sentences into coherent chunks based on embedding similarity
• Query-based Scoring: Late Interaction with Power Mean aggregation for precise relevance scoring
• LexRank Scoring: Graph-based centrality scoring when no query is provided
• LexRank Backfill: Supplements query results with high-centrality chunks when needed
• Percentile Filtering: Dynamic threshold based on score distribution
• Multilingual Support: Uses paraphrase-multilingual-MiniLM-L12-v2 (50+ languages)
• PDF Support: Extracts text from PDF documents
• GPU Acceleration: Auto-detects CUDA, DirectML, ROCm, OpenVINO providers
• Context Efficient: Filters out irrelevant content to reduce token usage

Usage

MCP Client Configuration

Add to your MCP client configuration (e.g. claude_desktop_config.json):

{
  "mcpServers": {
    "fetch-rag": {
      "command": "uvx",
      "args": [
        "--from", "git+https://github.com/attamari/mcp-server-fetch-rag",
        "mcp-server-fetch-rag"
      ]
    }
  }
}

With CLI options:

{
  "mcpServers": {
    "fetch-rag": {
      "command": "uvx",
      "args": [
        "--from", "git+https://github.com/attamari/mcp-server-fetch-rag",
        "mcp-server-fetch-rag",
        "--ignore-robots-txt",
        "--user-agent", "your-custom-user-agent"
      ]
    }
  }
}

CLI Options

| Option | Description | |---|---| | --user-agent | Custom User-Agent string (overrides default MCP UA) | | --ignore-robots-txt | Ignore robots.txt restrictions | | --proxy-url | Proxy URL for HTTP requests |

Tool: fetch_rag

Fetches a URL and returns relevant content chunks.

| Parameter | Type | Required | Default | Description | |---|---|---|---|---| | url | string | Yes | - | URL to fetch | | query | string | No | null | Search query for relevance filtering | | max_chunks | int | No | 10 | Maximum number of chunks to return |

How It Works

1. Fetch: Downloads content from URL (HTML via trafilatura, PDF via pypdfium2)
2. Split: Segments text into sentences using wtpsplit (sat-3l-sm, 85+ languages)
3. Embed: Generates L2-normalized embeddings (paraphrase-multilingual-MiniLM-L12-v2 via FastEmbed/ONNX)
4. Chunk: Groups adjacent sentences by embedding similarity into semantic chunks
5. Score:

• With query: Late Interaction — sentence-level query similarity aggregated via Power Mean
• Without query: LexRank — sentence-level graph centrality aggregated via Power Mean

1. Filter: Applies percentile-based dynamic threshold (P30)
2. Backfill: When query scoring yields insufficient chunks, supplements with high-centrality LexRank chunks (P30 filtered)
3. Return: Top chunks sorted in original document order

License

MIT License — See LICENSE for details.