mcp-rag-assistant

MCP-Powered AI Assistant (Local — LlamaIndex + Ollama)

Privacy-first document intelligence: all models run locally via Ollama — no data leaves your machine.

---

What is MCP?

Model Context Protocol (MCP) is an open standard (by Anthropic) that defines how AI models discover and invoke tools at runtime. Think of it as a "USB-C port for AI" — any MCP-compatible client (Claude Desktop, your own agent, etc.) can connect to any MCP server and immediately use its tools.

MCP vs. Advanced RAG — What's the Difference?

| Dimension | Advanced RAG | MCP | |-----------|-------------|-----| | Purpose | Improve retrieval accuracy | Standardise tool/capability exposure | | Core idea | Better chunking, re-ranking, hybrid search | JSON-RPC tool registry with discovery | | What the LLM gets | Retrieved context injected into prompt | A menu of callable functions with schemas | | Execution | Single pipeline (query → retrieve → generate) | Multi-step agent loop (plan → pick tool → call → observe → repeat) | | Tools | Retrieval only | Any function: retrieval, APIs, databases, code | | State | Stateless per query | Stateful agent sessions possible | | This project | RAG is one tool inside the MCP server | MCP wraps 8 RAG tools, discoverable at runtime |

In short: Advanced RAG makes retrieval smarter. MCP makes the entire AI system composable and interoperable.

---

Project Architecture

mcp_rag_assistant/
├── config.py               ← Central config (LLM, embed, chunking, server)
├── rag_engine.py           ← LlamaIndex: load docs → build index → query engine
├── main.py                 ← CLI entrypoint (serve / index / query / demo)
├── mcp_client.py           ← Example client that calls server tools
│
├── mcp_server/
│   └── server.py           ← HTTP JSON-RPC server exposing all tools
│
├── tools/
│   └── rag_tools.py        ← 8 MCP tool implementations
│
├── utils/
│   └── logger.py           ← Structured logging
│
├── my_data/                ← ⬅ DROP YOUR FILES HERE (PDF, DOCX, XLSX, CSV)
├── storage/                ← ChromaDB persistence (auto-created)
├── logs/                   ← Log files (auto-created)
│
├── requirements.txt
├── .env.example
├── .gitignore
└── README.md

Data Flow

User Query
    │
    ▼
MCP Client (mcp_client.py or Claude Desktop or your agent)
    │  JSON-RPC POST /mcp  {"method": "tools/call", "params": {...}}
    ▼
MCP Server (mcp_server/server.py)
    │  dispatches to matching tool function
    ▼
Tool Function (tools/rag_tools.py)
    │  calls get_query_engine().query(...)
    ▼
LlamaIndex Query Engine (rag_engine.py)
    │  embeds query with qwen3-embedding:0.6b via Ollama
    ▼
ChromaDB Vector Store
    │  returns top-K similar chunks
    ▼
Ollama LLM (llama3 or mistral)
    │  synthesises answer from retrieved context
    ▼
JSON response back through MCP → Client

---

Available MCP Tools

| Tool | Description | |------|-------------| | query_documents | General Q&A over all indexed documents | | list_indexed_files | Show files in my_data/ | | rebuild_index | Re-index after adding/removing files | | summarize_document | Summarise a specific file by name | | analyze_data | Plain-English data analysis (CSV/XLSX) | | generate_report | Generate summary / detailed / executive report | | compare_documents | Compare two documents on a given aspect | | extract_entities | Extract people, orgs, dates, numbers |

---

Prerequisites

• Python 3.10+
• Ollama running locally — ollama.com
• Models already pulled (you have these):
• llama3:latest
• mistral:latest
• qwen3-embedding:0.6b

---

Setup

# 1. Clone / unzip the project
cd mcp_rag_assistant

# 2. Create virtual environment
python -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Configure (optional — defaults work out of the box)
cp .env.example .env
# Edit .env to change models, ports, chunk sizes etc.

# 5. Add your documents
#    Copy PDFs, DOCX, XLSX, CSV files into:
#    my_data/

# 6. Build the index
python main.py index

# 7. Start the MCP server
python main.py serve

---

Usage

Start the server

python main.py serve
# MCP server listening on http://0.0.0.0:8080

One-shot query (no server needed)

python main.py query "What are the key findings in the Q1 report?"

Run the demo client (server must be running)

# In a second terminal:
python main.py demo

Rebuild index after adding new files

python main.py index
# or via MCP tool:
# call rebuild_index tool from any client

Health check

GET http://localhost:8080/health
GET http://localhost:8080/tools

---

Switching Models

Edit config.py or your .env:

# Use mistral instead of llama3
LLM_MODEL=mistral:latest

# Use nomic-embed-text for embeddings
EMBED_MODEL=nomic-embed-text:latest

---

Tuning Chunk Size

In config.py or .env:

| Setting | Default | Notes | |---------|---------|-------| | CHUNK_SIZE | 256 | Tokens per chunk. Smaller = more precise retrieval | | CHUNK_OVERLAP | 25 | Overlap between chunks. Helps preserve context at boundaries | | SIMILARITY_TOP_K | 5 | Chunks retrieved per query | | RESPONSE_MODE | compact | compact \| tree_summarize \| refine |

---