mcp-probe

!mcp-probe — one command to diagnose your MCP server

mcp-probe

One command to diagnose your MCP server.

Tests every tool, resource, and prompt your server exposes — then gives you a health report with a pass/fail scorecard.

Built on the Anthropic Model Context Protocol (MCP) spec.

Note: Published to npm as @incultnitollc/mcp-probe. The CLI binary is mcp-probe. The unscoped name mcp-doctor on npm is owned by an unrelated tool, so this project ships under a scope. Versions <= 0.2.1 shipped under the deprecated @incultnitostudiosllc scope — install @incultnitollc/mcp-probe instead.

<p align="center"> <img src="demo.gif" alt="mcp-probe demo" width="800" /> </p>

npx @incultnitollc/mcp-probe test "npx -y @modelcontextprotocol/server-everything"

Test your MCP server in 30 seconds

| Check | Description | |-------|-------------| | Tool calling | Calls every tool with auto-generated sample arguments based on the input schema | | Resource reading | Reads every resource and verifies content is returned | | Prompt rendering | Gets every prompt with sample arguments and verifies messages are returned | | Schema validation | Checks tool schemas for missing descriptions, broken required fields, malformed types | | Health scoring | Summarizes everything into a pass/fail scorecard |

Install

npm install -g @incultnitollc/mcp-probe

Or run directly:

npx @incultnitollc/mcp-probe test "your-server-command"

Usage

Local stdio server

npx @incultnitollc/mcp-probe test "npx -y @modelcontextprotocol/server-everything"

Remote server (Streamable HTTP)

npx @incultnitollc/mcp-probe test https://your-server.example.com/mcp

Remote server (SSE)

npx @incultnitollc/mcp-probe test https://your-server.example.com/mcp --transport sse

Authenticated remote server

npx @incultnitollc/mcp-probe test https://your-server.example.com/mcp \
  --header "Authorization: Bearer $TOKEN"

Options

| Flag | Description | |---|---| | --json | Output results as JSON | | --timeout <ms> | Per-operation timeout (default 30000) | | --transport <kind> | Force stdio, sse, or http (auto-detected from target) | | --header <Name: value> | Add header to remote transport. Repeatable. |

Exit codes

• 0 — All checks passed
• 1 — One or more checks failed (useful for CI gates)

JSON output

Use --json to get structured output for automation:

mcp-probe test --json "your-server" | jq '.score'

{
  "toolsCallable": 12,
  "toolsTotal": 13,
  "resourcesReadable": 7,
  "resourcesTotal": 7,
  "promptsGettable": 3,
  "promptsTotal": 4,
  "schemaErrors": 0,
  "schemaWarnings": 1
}

How tool calling works

mcp-probe auto-generates arguments for each tool based on its inputSchema:

• Only required fields get values (safest approach)
• Uses default values and enum first choices when available
• Infers smart defaults from field names (url → https://example.com, email → test@example.com)
• Falls back to type-appropriate defaults (string → "test", number → 1, boolean → false)

This means tools with complex required inputs may fail — and that's useful information. It tells you your tool isn't self-contained enough for automated testing.

Publishability score (v1.1.0+)

mcp-probe ships a second, complementary check: a publishability composite that scores your server 0–100 on whether its schemas, descriptions, and metadata are ready for other people to install. Run it as a shorthand:

npx @incultnitollc/mcp-probe score "npx -y @your-scope/your-server" --package ./package.json

Or fold it into a full test run with --publishability:

npx @incultnitollc/mcp-probe test "npx -y @your-scope/your-server" --publishability --package ./package.json

The composite combines three sub-scores — Protocol (does the wire format work), Edge cases (does it handle weird inputs), and Publishability (would a stranger understand your tools) — and a five-axis breakdown across the publishability dimension:

| Axis | What it checks | |---|---| | description-five-axis | Per-tool description density across purpose, mutation, side-effects, invariants, examples. Tools below 3.0/5 axes fire a ≤60 composite cap. | | enum-shape | Catches prose-only enums (e.g. "one of: open, closed" in the description with no JSON Schema enum). | | mutation-legibility | Does each tool tell a planner it mutates, or only reads? Name prefix / description signal / annotation all count. | | anti-purpose-clause | High-blast tools (delete, send, transfer) should include a "do not use for X, prefer Y" pointer to a narrower tool. | | distribution-metadata | npm package readiness — description length, keyword count, repository / license / homepage fields. Skipped without --package. |

What scores look like on real servers

The five official Anthropic MCP servers all land at 60/100 under v1.1.0 — the description-five-axis cap fires on every one. That's not a bug in the rubric; that's the bar Anthropic ships at, and the bar most servers will start from. Full scorecards in docs/publishability-scorecards/.

CI gate

- uses: incultnitollc/mcp-probe@v1
  with:
    command: 'node dist/index.js'
    publishability: 'true'
    package: './package.json'
    fail-under: '70'

Pre-publish vs install-time

mcp-probe's publishability score is the pre-publish quality lane — for server authors before they ship. For the install-time security lane — server installers before they connect a third-party server — see @stephenywilson/mcp-doctor. Different audiences, complementary tools.

Contract testing — "VCR for MCP" (v1.2.0+)

test/score tell you if your server is healthy today. Contract testing tells you what changed since last time — so a breaking schema edit or a poisoned tool description never ships silently.

Record a snapshot of your server's contract (its tools, resources, prompts, and their schemas — no traffic, no side effects), commit the .mcpvcr file, then diff or gate every PR against it.

# 1. Record a baseline (lists only — never calls a tool)
mcp-probe record "node dist/index.js" --out .mcp/contract.mcpvcr
git add .mcp/contract.mcpvcr && git commit -m "record MCP contract baseline"

# 2. See what a change did to the contract
mcp-probe diff --baseline .mcp/contract.mcpvcr "node dist/index.js"

# 3. Gate CI — exit 1 on breaking or security changes
mcp-probe gate --baseline .mcp/contract.mcpvcr "node dist/index.js"

Every change is classified so the gate is meaningful, not noisy:

| Severity | Examples | |----------|----------| | ❌ breaking | tool removed · new required argument · property removed · type changed · enum narrowed | | ⚠️ security | tool description mutated (rug-pull / tool-poisoning) · readOnlyHint dropped · tool became destructive | | ✅ additive | new tool · new optional field · enum widened | | ℹ️ info | server name · resource/prompt description text |

gate fails on breaking,security by default — tune with --fail-on breaking (or any comma-separated set). Snapshots are deterministic: re-recording an unchanged server produces a byte-identical file, so committed baselines and diffs stay clean. Compare two recorded files offline with --against <file> instead of a live target, and write a ready-to-post PR comment with --markdown <path>.

Why it matters: the 2026-07-28 MCP spec ships breaking changes (dropped initialize handshake, error-code and JSON-Schema shifts, HTTP+SSE → Streamable HTTP). A recorded .mcpvcr baseline turns "did we break our clients?" into a one-line CI check. See examples/contract-gate.yml for the full PR workflow with auto-commenting.

Use cases

• MCP server development — Run mcp-probe in your test suite to catch regressions
• CI/CD gates — Block deploys if your MCP server doesn't pass health checks
• Server evaluation — Quickly assess third-party MCP servers before integrating them
• Schema quality — Find missing descriptions and malformed schemas before users hit them

CI integration

mcp-probe exits 0 on full pass and 1 on any failure, so it drops directly into any CI pipeline:

# .github/workflows/mcp-health.yml
- name: Health-check MCP server
  run: npx @incultnitollc/mcp-probe test "$MCP_SERVER_CMD"

Use --json for structured output and jq to gate on specific metrics (e.g. fail the build if schemaWarnings > 0).

GitHub Action

Drop mcp-probe into your MCP server's GitHub Actions workflow in two lines:

- uses: incultnitollc/mcp-probe@v1
  with:
    command: 'node dist/index.js'

Gate your PRs on a publishability composite:

- uses: incultnitollc/mcp-probe@v1
  with:
    command: 'node dist/index.js'
    publishability: 'true'
    package: './package.json'
    fail-under: '70'

Inputs

| Name | Required | Default | Description | |---|---|---|---| | command | yes | — | Command that launches your MCP server (e.g. node dist/index.js or npx -y @your-scope/your-server). | | fail-under | no | 0 | Fail the job if the publishability composite drops below this value (0–100). Requires publishability: 'true'. | | publishability | no | false | Run the publishability suite — 5 checks + 0–100 composite. Requires mcp-probe >= 1.1.0 (ships 2026-05-23). | | package | no | '' | Path to package.json for the distribution-metadata check. Empty skips the distribution check. | | html-report | no | '' | Path to write the HTML scorecard. Upload via actions/upload-artifact in a follow-on step. | | mcp-probe-version | no | latest | npm version, dist-tag, or latest. Pin for reproducible builds. | | json-output | no | '' | Path to write the JSON report for downstream parsing. |

Outputs

| Name | Description | |---|---| | composite-score | Publishability composite (0–100). Only set when publishability: 'true'. | | band | Grade band: publishable / almost / rough / not-ready. Only set when publishability: 'true'. | | tools-pass-rate | tools_callable / tools_listed as a decimal (e.g. 0.83). | | schema-warnings | Total schema warning count across all tools. |

More examples: examples/basic.yml · examples/publishability-gate.yml · examples/matrix.yml.

Marketplace listing: github.com/marketplace/actions/mcp-probe-mcp-server-health-check.

Compared to MCP Inspector

The official MCP Inspector is a GUI for interactive exploration — point, click, see what a server returns. mcp-probe is a CLI for automated, repeatable diagnosis — every tool/resource/prompt called automatically, pass/fail scorecard out, exit code in. Use Inspector when you're exploring; use mcp-probe in CI, in pre-publish checks, or when you want a shareable scorecard of someone else's server.

Ecosystem

• MCP Registry — Cross-source catalog of MCP servers (~6,900 indexed across 6 upstream lists) with quality scores powered by mcp-probe. CLI: npm i -g @incultnitollc/mcpr. Built by Incultnito LLC.

Development

git clone https://github.com/incultnitollc/mcp-probe.git
cd mcp-probe
npm install
npm run dev -- test "npx -y @modelcontextprotocol/server-everything"
npm test

License

MIT - Incultnito LLC