MCP Local LLM Server

MCP Local LLM Server

A FastMCP server that exposes a locally-hosted llama.cpp language model as MCP tools, plus a suite of utility tools for weather, news, web fetching, file I/O, stock data, summarization, and more.

Overview

This server lets any MCP-compatible client (e.g. Claude Desktop, Cursor) use a local GGUF model for text generation and chat, without sending data to an external API. The model is served by a llama-server subprocess; the MCP server communicates with it over a local HTTP port.

src/
├── llm_server.py        # Entry point, argument parsing, server startup
├── model.py             # llama-server lifecycle, token generation
├── upload.py            # POST /upload endpoint for PDF and Markdown file uploads
├── resources.py         # MCP resource: llm://info
└── tools/
    ├── generate.py      # Tool: generate
    ├── chat.py          # Tool: chat
    ├── weather.py       # Tool: get_weather
    ├── date_time.py     # Tool: get_datetime
    ├── fetch_url.py     # Tool: fetch_url
    ├── news.py          # Tool: news_headlines
    ├── read_pdf.py      # Tool: read_pdf
    ├── read_markdown.py # Tool: read_markdown
    ├── create_file.py   # Tool: create_file
    ├── list_directory.py# Tool: list_directory
    ├── stock_price.py   # Tool: get_stock_price
    ├── summarize.py     # Tool: summarize_text
    ├── agent.py         # Tool: run_agent (autonomous ReAct agent)
    ├── explain_code.py  # Tool: explain_code (coding tutor)
    ├── review_code.py   # Tool: review_code  (coding tutor)
    ├── coding_tutor.py  # Tool: coding_tutor (orchestrating tutor agent)
    ├── transcribe_audio.py # Tool: transcribe_audio
    ├── text_to_speech.py   # Tool: text_to_speech
    ├── word_definition.py  # Tool: define_word
    └── random_joke.py      # Tool: get_random_joke

---

Requirements

• Python 3.10+
• A llama-server binary from llama.cpp
• A GGUF model file (e.g. models/Qwen2.5-7B-Instruct-Q4_K_M.gguf)
• CUDA-capable GPU recommended for large models

Install dependencies:

pip install -r requirements.txt

On Windows, IANA timezone data is not bundled with Python. Install it for the get_datetime tool to support non-UTC timezones:

pip install tzdata

The transcribe_audio tool requires faster-whisper, which is listed in requirements.txt but has an optional CUDA-accelerated variant. For GPU inference, install the matching PyTorch CUDA build first:

# CPU-only (default)
pip install faster-whisper

# GPU (CUDA 12)
pip install faster-whisper
pip install torch --index-url https://download.pytorch.org/whl/cu128

---

Configuration

Some tools require API keys. Create a .env file in the project root (already gitignored):

NEWSAPI_KEY=your_key_here

The server loads this file automatically on startup.

| Variable | Required by | Where to get it | |---|---|---| | NEWSAPI_KEY | news_headlines | newsapi.org — free tier available |

All other tools work without any API key.

---

Starting the Server

stdio transport (for MCP clients like Claude Desktop)

python src/llm_server.py \
  --model models/Qwen2.5-7B-Instruct-Q4_K_M.gguf \
  --llama-server /path/to/llama-server

HTTP transport (for network clients or testing with curl)

python src/llm_server.py \
  --model models/Qwen2.5-7B-Instruct-Q4_K_M.gguf \
  --llama-server /path/to/llama-server \
  --transport http \
  --port 5174

CLI flags

| Flag | Default | Description | |---|---|---| | --model | (required) | Path to a .gguf file, or a directory containing one | | --llama-server | (required) | Path to the llama-server executable | | --transport | stdio | stdio or http | | --host | 0.0.0.0 | Host to bind for HTTP transport (use 127.0.0.1 for localhost only) | | --port | 5174 | MCP server port (HTTP transport only) | | --server-port | 8080 | Port for the internal llama-server backend | | --gpu-layers | -1 | Layers to offload to GPU; -1 = all | | --context-size | 16384 | Total context window in tokens (prompt + output combined) |

---

HTTP Endpoints

These endpoints are only available when using --transport http.

POST /upload

Upload a PDF or Markdown file to the server and receive an upload_id to pass to run_agent.

Supported types: .pdf, .md, .markdown

Request: multipart/form-data with a single field named file.

Response: ``json { "upload_id": "3f8a1c...", "filename": "report.pdf", "size": 84210 } ``

Uploaded files are stored in uploads/ at the project root and deleted when the server shuts down.

Example: ```bash curl -X POST http://localhost:5174/upload \ -F "file=@/path/to/report.pdf"

curl -X POST http://localhost:5174/upload \ -F "file=@/path/to/notes.md" ```

---

Tools

generate

Generate text from a raw prompt. The input prompt is not included in the returned text.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | prompt | string | (required) | Input text to continue | | max_new_tokens | int | 512 | Maximum tokens to generate | | temperature | float | 0.7 | Sampling temperature; 0 = greedy (deterministic) | | top_p | float | 0.9 | Nucleus-sampling cumulative probability cutoff | | top_k | int | 0 | Top-k vocabulary filter; 0 = disabled | | repetition_penalty | float | 1.0 | Penalty for repeating tokens; 1.0 = no penalty | | stop_sequences | list[string] | null | Strings that halt generation when produced | | seed | int | null | RNG seed for reproducible outputs |

Returns: The generated text as a plain string.

Example

{
  "prompt": "The capital of France is",
  "max_new_tokens": 50,
  "temperature": 0
}

---

chat

Chat with the local LLM using a conversation history.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | messages | list[{"role": string, "content": string}] | (required) | Conversation history. Valid roles: "system", "user", "assistant" | | max_new_tokens | int | 512 | Maximum tokens to generate | | temperature | float | 0.7 | Sampling temperature; 0 = greedy (deterministic) | | top_p | float | 0.9 | Nucleus-sampling cumulative probability cutoff | | top_k | int | 0 | Top-k vocabulary filter; 0 = disabled | | repetition_penalty | float | 1.0 | Penalty for repeating tokens; 1.0 = no penalty | | stop_sequences | list[string] | null | Strings that halt generation when produced | | seed | int | null | RNG seed for reproducible outputs |

Returns: The assistant's reply as plain text.

Example

{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user",   "content": "What is the Eiffel Tower?"}
  ],
  "temperature": 0.7,
  "max_new_tokens": 256
}

---

get_weather

Fetch current weather for any location using the free Open-Meteo API. No API key required.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | location | string | (required) | City name or region (e.g. "London", "New York", "Tokyo") | | units | string | "metric" | "metric" (°C, km/h) or "imperial" (°F, mph) |

Returns: Current conditions, temperature, humidity, and wind speed.

Example output

Weather in London, England, United Kingdom:
  Conditions:  Partly cloudy
  Temperature: 12.3°C
  Humidity:    74%
  Wind:        18.5 km/h

---

get_datetime

Return the current date and time for any IANA timezone. No API key required.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | timezone | string | "UTC" | IANA timezone name (e.g. "America/New_York", "Europe/London", "Asia/Tokyo") |

Returns: A formatted date/time string, e.g. "2025-03-01 14:30:00 EST (UTC-0500)".

Windows note: Non-UTC timezones require pip install tzdata.

Example

{"timezone": "America/Chicago"}

---

fetch_url

Fetch the content of any URL and return it as plain text. HTML pages are stripped of tags; JSON and plain-text responses are returned as-is. No API key required.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | url | string | (required) | URL to fetch (must start with http:// or https://) | | max_chars | int | 4000 | Maximum characters to return before truncating |

Returns: Extracted page text, truncated to max_chars if needed. Returns a descriptive error string on connection failure rather than raising an exception.

Example

{
  "url": "https://en.wikipedia.org/wiki/Python_(programming_language)",
  "max_chars": 2000
}

---

news_headlines

Fetch the latest news headlines, optionally filtered by topic. Requires a free NewsAPI key.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | topic | string | "" | Keyword(s) to filter by (e.g. "AI", "climate change"). Leave blank for general top headlines | | country | string | "us" | 2-letter country code used when topic is blank (e.g. us, gb, au, de) | | max_results | int | 5 | Number of headlines to return (1–10) |

Returns: Numbered list of headlines with source, publication date, and URL.

Example

{"topic": "artificial intelligence", "max_results": 3}

Example output

1. [BBC News] OpenAI releases new model
   Published: 2025-03-01
   https://bbc.co.uk/...

2. [Reuters] ...

---

read_pdf

Extract and return the text content of a PDF file, organised by page. Image-only (scanned) PDFs return a clear message rather than empty output.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | file_path | string | (required) | Absolute or relative path to the PDF file | | max_chars | int | 8000 | Maximum characters to return before truncating |

Returns: Extracted text organised by page, truncated to max_chars if needed.

Example

{"file_path": "/home/user/documents/report.pdf", "max_chars": 10000}

---

read_markdown

Read and return the contents of a Markdown file.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | file_path | string | (required) | Absolute or relative path to the .md or .markdown file | | max_chars | int | 8000 | Maximum characters to return before truncating |

Returns: The file's text content, truncated to max_chars if needed.

Example

{"file_path": "/home/user/documents/notes.md"}

---

create_file

Create a new file with the given name and content. Rejects path traversal attempts and validates the filename before writing.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | file_name | string | (required) | Basename of the file to create (no path separators allowed) | | content | string | (required) | Text content to write to the file | | directory | string | null | Subdirectory to create the file in (relative to the server's working directory). Created if it doesn't exist | | encoding | string | "utf-8" | Character encoding for the file | | overwrite | bool | false | If true, overwrite an existing file at the same path |

Returns: A JSON object with keys:

• success — true if the file was created
• file_path — absolute path to the created file (null on failure)
• error — error message if success is false
• message — human-readable status

Examples

{"file_name": "notes.md", "content": "# My Notes\n"}

{
  "file_name": "config.json",
  "content": "{\"debug\": true}",
  "directory": "src",
  "overwrite": true
}

---

list_directory

List the files and directories at a given path, with optional glob filtering and recursive traversal.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | path | string | (required) | Absolute or relative path to the directory to list | | pattern | string | "" | Glob pattern to filter results (e.g. ".py", "data_", "/.json") | | recursive | bool | false | If true, traverse all subdirectories | | include_hidden | bool | false | If true, include files and folders whose names start with "." | | max_results | int | 200 | Maximum number of entries to return |

Returns: A formatted listing showing [DIR] and [FILE] entries with file sizes, plus a summary count. Truncation is noted if max_results is reached.

Examples

{"path": "/home/user/projects"}

{
  "path": "/home/user/projects/mcp-server",
  "pattern": "*.py",
  "recursive": true
}

Example output

Directory: /home/user/projects/mcp-server/src

  [DIR]  tools/
  [FILE] llm_server.py  (4 KB)
  [FILE] model.py       (7 KB)
  [FILE] resources.py   (1 KB)
  [FILE] upload.py      (2 KB)

4 item(s) shown

---

get_stock_price

Get the current stock price and key market data for a ticker symbol via the Yahoo Finance API. No API key required.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | ticker | string | (required) | Stock ticker symbol (e.g. "AAPL", "MSFT", "TSLA", "BTC-USD") |

Returns: Current price, daily change, day range, 52-week range, volume, and market cap.

Example

{"ticker": "NVDA"}

Example output

NVIDIA Corp (NVDA)
  Price:       875.40 USD
  Change:      +12.30 (+1.43%)
  Prev close:  863.10 USD
  Day range:   860.00 – 879.50 USD
  52-wk range: 410.00 – 974.00 USD
  Volume:      42,381,200
  Market cap:  2.16T USD

Tip: Crypto pairs are also supported (e.g. "BTC-USD", "ETH-USD").

---

summarize_text

Summarize a block of text using the local LLM. Long texts are automatically split into chunks, each summarized independently, then merged into a single coherent summary.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | text | string | (required) | The text to summarize. Can be arbitrarily long | | focus | string | "" | Optional instruction to guide the summary (e.g. "key risks", "action items", "technical details"). Leave blank for a general summary | | max_length | int | 200 | Approximate maximum length of the summary in tokens. Controls verbosity |

Returns: A concise summary of the input text.

Examples

{"text": "... (long article) ..."}

{
  "text": "... (meeting transcript) ...",
  "focus": "action items",
  "max_length": 150
}

{
  "text": "... (technical document) ...",
  "focus": "key risks",
  "max_length": 300
}

---

transcribe_audio

Transcribe an audio file to text using a local Whisper model via faster-whisper. Runs entirely on-device — no API key or internet connection required. Models are downloaded automatically on first use and cached locally.

Supported formats: mp3, mp4, wav, flac, ogg, m4a, webm, and most ffmpeg-readable formats.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | audio_path | string | (required) | Absolute or relative path to the audio file | | model_size | string | "base" | Whisper model to use: "tiny" (~150 MB), "base" (~290 MB), "small" (~970 MB), "medium" (~3.1 GB), "large-v3" (~6.2 GB) | | language | string | null | ISO-639-1 language code to force (e.g. "en", "fr"). Leave null to auto-detect | | device | string | "auto" | Inference device. "auto" selects CUDA if available, else CPU. Explicit: "cuda", "cpu" | | compute_type | string | "auto" | Precision. "auto" uses float16 on GPU and int8 on CPU. Explicit: "float16", "int8", "float32" |

Returns: A JSON object with keys:

• success — true if transcription succeeded
• text — the transcribed text (null on failure)
• language — detected or forced language code
• duration_seconds — audio duration in seconds
• error — error message if success is false

Examples

{"audio_path": "/home/user/recordings/meeting.mp3"}

{
  "audio_path": "/home/user/recordings/lecture.wav",
  "model_size": "small",
  "language": "en"
}

Tip: Use "tiny" or "base" for fast transcription of short clips. Use "small" or higher for better accuracy on noisy audio or non-English speech.

---

run_agent

Run an autonomous ReAct agent powered by the local LLM. The agent reasons step by step and calls tools as many times as needed before producing a final answer.

How it works

User goal
   ↓
LLM decides: call a tool or answer?
   ↓ (if tool)
Tool executes → result fed back to LLM
   ↓
LLM decides again … (repeats up to max_steps)
   ↓ (when done)
FINAL answer returned

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | goal | string | (required) | The task or question for the agent to solve | | max_steps | int | 10 | Maximum tool-call iterations before stopping | | max_new_tokens | int | 4096 | Token ceiling per LLM call. Mainly affects the length of the final answer | | max_history_pairs | int | 4 | Recent assistant+tool rounds to keep in full; older rounds are summarised | | summary_strategy | string | "deterministic" | "deterministic" — fast rule-based bullet points. "llm" — model-generated prose (adds an extra generation call) | | upload_id | string | "" | ID returned by POST /upload. The file's contents are injected into the agent's context before the loop starts |

Returns: The agent's final answer as plain text.

If you see Agent stopped after N steps without a FINAL answer, the agent exhausted its iterations. Either increase max_steps or simplify the goal.

Tools available to the agent

| Tool | Description | |---|---| | get_weather | Fetch current weather for any city | | get_datetime | Get the current date and time in any timezone | | fetch_url | Fetch and extract text from any URL | | news_headlines | Fetch the latest news headlines by topic | | read_pdf | Extract text from a PDF file at a given path | | read_markdown | Read the contents of a Markdown file at a given path | | get_stock_price | Get the current stock price and market data for a ticker symbol | | summarize_text | Summarize a block of text using the local LLM | | create_file | Create a new file with the given name and content | | list_directory | List files and directories at a given path | | transcribe_audio | Transcribe an audio file to text using a local Whisper model |

Examples

{"goal": "What should I wear in Paris today?"}

{"goal": "Compare the weather in London and Tokyo, then tell me which city is warmer."}

{"goal": "Summarise this document", "upload_id": "3f8a1c..."}

{
  "goal": "What are the top AI news stories today?",
  "max_steps": 5
}

---

text_to_speech

Convert text to an MP3 audio file using Google Text-to-Speech (gTTS). Requires an internet connection. No API key required.

Dependency: Install gTTS before using this tool: ``bash pip install gtts ``

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | text | string | (required) | The text to convert to speech | | output_path | string | (required) | File path where the MP3 will be saved (e.g. "output/speech.mp3"). Parent directories are created automatically | | lang | string | "en" | BCP-47 language code (e.g. "en", "fr", "es", "de", "ja") | | slow | bool | false | If true, speech is generated at a slower rate |

Returns: A JSON object with keys:

• success — true if the file was saved successfully
• output_path — absolute path to the saved MP3 (null on failure)
• error — error message if success is false

Examples

{
  "text": "Hello, world!",
  "output_path": "output/hello.mp3"
}

{
  "text": "Bonjour le monde",
  "output_path": "output/bonjour.mp3",
  "lang": "fr",
  "slow": true
}

---

define_word

Look up the definition, phonetics, synonyms, and antonyms of an English word using the free Dictionary API. No API key required.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | word | string | (required) | The English word to look up (e.g. "ephemeral", "serendipity") |

Returns: A JSON object with keys:

• success — true if the word was found
• word — the normalised word that was looked up
• results — list of meanings, each containing:
• phonetic — IPA phonetic spelling (may be null)
• part_of_speech — e.g. "noun", "verb", "adjective"
• definitions — up to 3 definitions, each with definition, example (may be null), synonyms (up to 5), antonyms (up to 5)
• synonyms — up to 5 synonyms for this part of speech
• antonyms — up to 5 antonyms for this part of speech
• error — error message if success is false

Example

{"word": "ephemeral"}

Example output (abbreviated)

{
  "success": true,
  "word": "ephemeral",
  "results": [
    {
      "phonetic": "/ɪˈfɛm(ə)r(ə)l/",
      "part_of_speech": "adjective",
      "definitions": [
        {
          "definition": "Lasting for a very short time.",
          "example": "fashions are ephemeral",
          "synonyms": ["transitory", "transient", "fleeting"],
          "antonyms": ["permanent", "eternal"]
        }
      ],
      "synonyms": ["transitory", "transient"],
      "antonyms": ["permanent"]
    }
  ],
  "error": null
}

---

get_random_joke

Fetch a random joke from the free JokeAPI. No API key required.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | category | string | "Any" | Joke category: "Any", "Programming", "Misc", "Dark", "Pun", "Spooky", "Christmas" | | joke_type | string | "any" | Format filter: "any", "single" (one-liner), "twopart" (setup + punchline) | | safe_mode | bool | true | If true, excludes explicit, racist, sexist, and religious jokes |

Returns: A JSON object with keys:

• success — true if a joke was returned
• category — the category the joke belongs to
• type — "single" or "twopart"
• joke — the joke text; two-part jokes are formatted as "setup\n\n— delivery"
• error — error message if success is false

Note: The "Dark" category is unavailable when safe_mode is true.

Examples

{"category": "Programming"}

{
  "category": "Pun",
  "joke_type": "twopart",
  "safe_mode": true
}

---

Coding Tutor Tools

Three tools that turn the server into an interactive programming tutor. The high-level entry point is coding_tutor; the two supporting tools (explain_code, review_code) can also be called directly.

coding_tutor

An autonomous ReAct agent specialised for teaching. It reasons step by step, calling explain_code, review_code, and fetch_url as needed, then produces a pedagogical response.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | question | string | (required) | Your coding question, code snippet, or error message | | max_steps | int | 8 | Maximum tool-call iterations before stopping | | max_new_tokens | int | 1024 | Token ceiling per LLM call | | max_history_pairs | int | 4 | Recent assistant+tool rounds to keep before older ones are summarised | | summary_strategy | string | "deterministic" | "deterministic" (fast) or "llm" (prose, slower) |

Returns: A teaching response as plain text.

Tools available to the tutor

| Tool | Description | |---|---| | explain_code | Explain a code snippet at the learner's skill level | | review_code | Review code for bugs, style, security, or performance issues | | fetch_url | Fetch documentation or a GitHub link referenced by the learner |

Examples

{"question": "Why does my list comprehension give the wrong result?\n\nresult = [x * 2 for x in [1, 2, 3] if x > 1]"}

{"question": "Explain the difference between a shallow copy and a deep copy in Python, with examples."}

Tip: The tutor infers skill level from your question — use plain language for beginner explanations, technical terminology for advanced ones.

---

explain_code

Explain a code snippet using the local LLM, tailored to the learner's skill level.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | code | string | (required) | The source code to explain (capped at 6 000 chars) | | language | string | "python" | Programming language of the snippet | | level | string | "beginner" | Explanation depth: "beginner", "intermediate", or "advanced" | | max_new_tokens | int | 1024 | Maximum tokens for the explanation |

Returns: A plain-text explanation of the code.

Example

{
  "code": "result = {k: v for k, v in zip(keys, values)}",
  "language": "python",
  "level": "beginner"
}

---

review_code

Review a code snippet for issues. Outputs a structured report: overall impression, numbered issues with severity, positives, and a top recommendation.

Parameters

| Parameter | Type | Default | Description | |---|---|---|---| | code | string | (required) | The source code to review (capped at 6 000 chars) | | language | string | "python" | Programming language of the snippet | | focus | string | "general" | Review focus: "general", "security", "performance", or "style" | | max_new_tokens | int | 768 | Maximum tokens for the review |

Returns: A structured review with four sections: Overall Impression, Issues Found, Positives, and Top Recommendation.

Example

{
  "code": "def get_user(id):\n    return db.execute(f'SELECT * FROM users WHERE id={id}')",
  "language": "python",
  "focus": "security"
}

---

Resources

llm://info

Returns metadata about the currently loaded model (path, context size, GPU layers).

---

Configuring with Claude Desktop

Add the server to your claude_desktop_config.json:

{
  "mcpServers": {
    "local-llm": {
      "command": "python",
      "args": [
        "src/llm_server.py",
        "--model", "models/Qwen2.5-7B-Instruct-Q4_K_M.gguf",
        "--llama-server", "/path/to/llama-server"
      ],
      "cwd": "/absolute/path/to/mcp-server"
    }
  }
}

---

Notes

• The model is loaded once at startup via llama-server and held in memory for the lifetime of the server.
• GPU offloading is controlled by --gpu-layers; -1 offloads all layers.
• --context-size sets the total token budget shared between prompt and generated output. Increase it if you experience truncation on long responses.
• The HTTP transport binds to 0.0.0.0 by default, making it accessible from other machines on the network. Use --host 127.0.0.1 to restrict to localhost. CORS is enabled for all origins — restrict allow_origins before exposing to untrusted networks.
• Uploaded files (POST /upload) are stored in uploads/ at the project root and automatically deleted on server shutdown.

Running a standalone llama-server (e.g. for opencode)

llama-server \
  --model /path/to/model.gguf \
  --port 8000 \
  --host 127.0.0.1 \
  --n-gpu-layers -1 \
  --ctx-size 16384 \
  --no-mmap