multi-model-mcp

multi-model-mcp

An MCP server that exposes tools for sub-agent style reasoning across multiple LLM providers. From Claude Code (or any MCP client), you can delegate prompts to OpenAI, Anthropic, Gemini, Groq, Ollama, OpenRouter, and any LiteLLM-supported provider — then run critique loops, debates, red-teaming, and answer ranking without leaving your conversation.

Tools

| Tool | Description | |---|---| | ask_model | Send a prompt to one configured model | | ask_many | Send the same prompt to multiple models in parallel | | reason_together | Multi-step reasoning: independent → critique, debate, or red-team | | critique_answer | Ask models to critique a draft answer | | pick_best_answer | Have a judge model rank candidate answers | | list_models | List all configured model aliases |

reason_together strategies

• independent_then_critique (default): All models answer independently → critic synthesizes
• debate: Models see each other's answers and refine over N rounds → critic synthesizes
• red_team: Proposer answers → red teamers attack → proposer revises (N rounds) → critic finalizes

Setup

1. Install

Requires Python ≥ 3.11 and uv.

git clone https://github.com/YOUR_USERNAME/multi-model-mcp
cd multi-model-mcp
uv sync

2. Configure models

Copy and edit models.yaml — it ships with common models pre-configured. Each entry is a model alias pointing to a LiteLLM model string:

models:
  gpt:
    litellm_model: gpt-4.1
    api_key_env: OPENAI_API_KEY

  claude:
    litellm_model: claude-sonnet-4-5
    api_key_env: ANTHROPIC_API_KEY

  local:
    litellm_model: ollama/qwen3:latest
    api_base: http://localhost:11434   # no key needed

Add any provider LiteLLM supports: Groq (groq/llama-3.3-70b-versatile), Mistral, Together AI, DeepSeek, OpenRouter (openrouter/...), etc.

3. Set API keys

cp .env.example .env
# edit .env with your keys

Only keys for providers you actually use are required.

4. Register with Claude Code

Add to your project's .mcp.json (or ~/.claude.json for global):

{
  "mcpServers": {
    "multi-model": {
      "command": "uv",
      "args": [
        "run",
        "--project", "/path/to/multi-model-mcp",
        "multi-model-mcp"
      ],
      "env": {
        "OPENAI_API_KEY": "sk-...",
        "ANTHROPIC_API_KEY": "sk-ant-...",
        "GEMINI_API_KEY": "...",
        "MODELS_CONFIG_PATH": "/path/to/multi-model-mcp/models.yaml"
      }
    }
  }
}

Or if you install it: ``bash uv tool install . ` Then use "command": "multi-model-mcp" without args`.

Example Claude Code usage

# Simple query
Use ask_model with alias "gpt" to explain backpressure in streaming systems.

# Parallel comparison
Use ask_many with aliases ["gpt", "claude", "gemini"] to explain the CAP theorem.
Compare their answers.

# Multi-model reasoning
Use reason_together with task "Should we use event sourcing for this service?"
model_aliases ["gpt", "gemini"], critic_model_alias "claude", strategy "independent_then_critique"

# Debate
Use reason_together with task "Is GraphQL worth the complexity over REST?"
model_aliases ["gpt", "claude"], critic_model_alias "gemini", strategy "debate", rounds 2

# Red-team a decision
Use reason_together with task "Our plan is to use a single Postgres instance for all tenants"
model_aliases ["gpt", "gemini", "groq"], strategy "red_team", rounds 2

# Critique a draft
Use critique_answer with question "What is eventual consistency?"
draft_answer "It means data will eventually be the same across nodes."
model_aliases ["claude", "gpt"]

# Pick the best
Use pick_best_answer with question "What is the best way to handle auth tokens?"
candidate_answers ["Store in localStorage", "Store in httpOnly cookies", "Store in memory only"]
judge_model_alias "claude"

Configuration reference

models.yaml fields

| Field | Required | Description | |---|---|---| | litellm_model | Yes | LiteLLM model string (e.g. gpt-4.1, gemini/gemini-2.5-pro, ollama/qwen3:latest) | | description | No | Human-readable label | | api_key_env | No | Env var name holding the API key | | api_base | No | Override base URL (needed for Ollama, proxies) | | timeout | No | Per-call timeout in seconds (default: 60) | | max_retries | No | Retry attempts on rate limit / timeout (default: 2) |

LiteLLM model strings by provider

| Provider | Example model string | |---|---| | OpenAI | gpt-4.1, gpt-4o, o4-mini | | Anthropic | claude-sonnet-4-5, claude-opus-4-8 | | Google Gemini | gemini/gemini-2.5-pro, gemini/gemini-2.5-flash | | Groq | groq/llama-3.3-70b-versatile | | Ollama | ollama/qwen3:latest, ollama/llama3.3 | | OpenRouter | openrouter/anthropic/claude-sonnet-4-5 | | Mistral | mistral/mistral-large-latest | | DeepSeek | deepseek/deepseek-chat | | Together AI | together_ai/meta-llama/Llama-3-70b-chat-hf |

See LiteLLM providers docs for the full list.

Design notes

• No key leakage: API keys are never logged; errors are sanitized before returning.
• Failure isolation: one model failing in ask_many / reason_together does not crash the call.
• Synthesis ≠ truth: reason_together presents the critic's output as a synthesized answer, not ground truth.
• No hidden reasoning exposed: traces summarize what happened (which model, which step) without exposing chain-of-thought internals.
• Easy to extend: add any LiteLLM-supported model in models.yaml with no code changes.