multimodal-mcp

multimodal-mcp

Multi-provider media generation MCP server. Generate images, videos, audio, and transcriptions from text prompts using OpenAI, xAI, Gemini, ElevenLabs, and BFL (FLUX) through a single unified interface.

Features

• 🎨 Image Generation — Generate images via OpenAI (gpt-image-1), xAI (grok-imagine-image), Gemini (imagen-4), or BFL (FLUX Pro 1.1)
• ✏️ Image Editing — Edit images via OpenAI, xAI, Gemini, or BFL (FLUX Kontext)
• 🎬 Video Generation — Generate videos via OpenAI (sora-2), xAI (grok-imagine-video), or Gemini (veo-3.1)
• 🔊 Audio Generation — Text-to-speech via OpenAI (tts-1), Gemini, or ElevenLabs (Flash v2.5). Sound effects via ElevenLabs
• 🎙️ Audio Transcription — Speech-to-text via OpenAI (Whisper) or ElevenLabs (Scribe)
• 🔄 Auto-Discovery — Automatically detects configured providers from environment variables
• 🎯 Provider Selection — Auto-selects or explicitly choose a provider per request
• 📁 File Output — Saves all generated media to disk with descriptive filenames

Quick Start

Set the API key for at least one provider. Most users only need one — add more to access additional providers.

# Using OpenAI
claude mcp add multimodal-mcp -e OPENAI_API_KEY=sk-... -- npx -y @r16t/multimodal-mcp@latest

# Or using xAI
# claude mcp add multimodal-mcp -e XAI_API_KEY=xai-... -- npx -y @r16t/multimodal-mcp@latest

# Or using Gemini
# claude mcp add multimodal-mcp -e GEMINI_API_KEY=AIza... -- npx -y @r16t/multimodal-mcp@latest

# Or using ElevenLabs (audio + transcription)
# claude mcp add multimodal-mcp -e ELEVENLABS_API_KEY=xi-... -- npx -y @r16t/multimodal-mcp@latest

# Or using BFL/FLUX (images)
# claude mcp add multimodal-mcp -e BFL_API_KEY=... -- npx -y @r16t/multimodal-mcp@latest

Using a different editor? See setup instructions for Claude Desktop, Cursor, VS Code, Windsurf, and Cline.

Environment Variables

| Variable | Required | Description | |----------|----------|-------------| | OPENAI_API_KEY | At least one provider key | OpenAI API key — enables image, video, audio generation, and transcription via gpt-image-1, sora-2, tts-1, and whisper-1 | | XAI_API_KEY | At least one provider key | xAI API key — enables image and video generation via grok-imagine-image and grok-imagine-video | | GEMINI_API_KEY | At least one provider key | Gemini API key — enables image, video, and audio generation via imagen-4, veo-3.1, and gemini-2.5-flash-preview-tts | | GOOGLE_API_KEY | — | Alias for GEMINI_API_KEY; either name is accepted | | ELEVENLABS_API_KEY | At least one provider key | ElevenLabs API key — enables audio generation (TTS, sound effects) and transcription via Flash v2.5 and Scribe v1 | | BFL_API_KEY | At least one provider key | BFL API key — enables image generation and editing via FLUX Pro 1.1 and FLUX Kontext | | MEDIA_OUTPUT_DIR | No | Directory for saved media files. Defaults to the current working directory |

Available Tools

generate_image

Generate an image from a text prompt.

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | prompt | string | Yes | Text description of the image to generate | | provider | string | No | Provider to use: openai, xai, google, bfl. Auto-selects if omitted | | aspectRatio | string | No | Aspect ratio: 1:1, 16:9, 9:16, 4:3, 3:4 | | quality | string | No | Quality level: low, standard, high | | outputDirectory | string | No | Directory to save the generated file. Absolute or relative path. Defaults to MEDIA_OUTPUT_DIR or cwd | | providerOptions | object | No | Provider-specific parameters passed through directly |

generate_video

Generate a video from a text prompt. Video generation is asynchronous and may take several minutes.

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | prompt | string | Yes | Text description of the video to generate | | provider | string | No | Provider to use: openai, xai, google. Auto-selects if omitted | | duration | number | No | Video duration in seconds (provider limits apply) | | aspectRatio | string | No | Aspect ratio: 16:9, 9:16, 1:1 | | resolution | string | No | Resolution: 480p, 720p, 1080p | | outputDirectory | string | No | Directory to save the generated file. Absolute or relative path. Defaults to MEDIA_OUTPUT_DIR or cwd | | providerOptions | object | No | Provider-specific parameters passed through directly |

generate_audio

Generate audio from text. Supports text-to-speech and sound effects. Audio generation is synchronous.

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | text | string | Yes | Text to convert to speech, or a description of the sound effect to generate | | provider | string | No | Provider to use: openai, google, elevenlabs. Auto-selects if omitted | | voice | string | No | Voice name (provider-specific). OpenAI: alloy, ash, coral, echo, fable, nova, onyx, sage, shimmer. Google: Kore, Charon, Fenrir, Aoede, Puck, etc. ElevenLabs: voice ID | | speed | number | No | Speech speed multiplier (OpenAI only): 0.25 to 4.0 | | format | string | No | Output format (OpenAI only): mp3, opus, aac, flac, wav, pcm | | outputDirectory | string | No | Directory to save the generated file. Absolute or relative path. Defaults to MEDIA_OUTPUT_DIR or cwd | | providerOptions | object | No | Provider-specific parameters passed through directly. ElevenLabs: set mode: "sound-effect" for sound effects, model for TTS model selection |

transcribe_audio

Transcribe audio to text (speech-to-text).

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | audioPath | string | Yes | Absolute path to the audio file to transcribe | | provider | string | No | Provider to use: openai, elevenlabs. Auto-selects if omitted | | language | string | No | Language code (e.g., en, fr, es) to hint the transcription language | | providerOptions | object | No | Provider-specific parameters passed through directly |

list_providers

List all configured media generation providers and their capabilities. Takes no parameters.

Provider Capabilities

| Provider | Image | Image Editing | Video | Audio | Transcription | Key Models | |----------|:-----:|:------------:|:-----:|:-----:|:------------:|------------| | OpenAI | ✅ | ✅ | ✅ | ✅ | ✅ | gpt-image-1, sora-2, tts-1, whisper-1 | | xAI | ✅ | ✅ | ✅ | — | — | grok-imagine-image, grok-imagine-video | | Gemini | ✅ | ✅ | ✅ | ✅ | — | imagen-4, veo-3.1, gemini-2.5-flash-preview-tts | | ElevenLabs | — | — | — | ✅ | ✅ | eleven_flash_v2_5, scribe_v1 | | BFL | ✅ | ✅ | — | — | — | flux-pro-1.1, flux-kontext-pro |

Image Aspect Ratios

| Provider | 1:1 | 16:9 | 9:16 | 4:3 | 3:4 | |----------|:---:|:----:|:----:|:---:|:---:| | OpenAI | ✅ | ✅ | ✅ | ✅ | ✅ | | xAI | ✅ | ✅ | ✅ | ✅ | ✅ | | Gemini | ✅ | ✅ | ✅ | ✅ | ✅ | | BFL | ✅ | ✅ | ✅ | ✅ | ✅ |

Video Aspect Ratios & Resolutions

| Provider | 16:9 | 9:16 | 1:1 | 480p | 720p | 1080p | |----------|:----:|:----:|:---:|:----:|:----:|:-----:| | OpenAI | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | xAI | ✅ | ✅ | ✅ | — | ✅ | ✅ | | Gemini | ✅ | ✅ | — | — | ✅ | ✅ |

Audio Formats

| Provider | mp3 | opus | aac | flac | wav | pcm | |----------|:---:|:----:|:---:|:----:|:---:|:---:| | OpenAI | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | Gemini | — | — | — | — | ✅ | — | | ElevenLabs | ✅ | ✅ | — | — | — | ✅ |

Troubleshooting

No providers configured

[config] No provider API keys detected

Set at least one of OPENAI_API_KEY, XAI_API_KEY, GEMINI_API_KEY, ELEVENLABS_API_KEY, or BFL_API_KEY in the MCP server's env block.

Provider not available for requested media type

Each provider supports different media types (see Provider Capabilities). If you specify a provider that isn't configured (no API key) or doesn't support the requested media type, you'll receive an error. Omit the provider parameter to auto-select from configured providers.

Video generation timeout

Video generation polls for up to 10 minutes. If your video hasn't completed in that window, the request will fail with a timeout error. Try a shorter duration or a simpler prompt.

xAI image generation returned no data

This indicates the xAI API returned an empty response. Check that your XAI_API_KEY is valid and that your prompt does not violate xAI content policies.

Gemini image/video generation failed: 403

Verify your GEMINI_API_KEY has the Generative Language API enabled in Google Cloud Console.

Development

npm run build      # Compile TypeScript to build/
npm test           # Run tests with Vitest
npm run lint       # Lint and auto-fix with ESLint
npm run typecheck  # Type-check without emitting
npm run dev        # Watch mode for TypeScript compilation

Editor Setup

Replace OPENAI_API_KEY with your provider of choice (XAI_API_KEY, GEMINI_API_KEY, ELEVENLABS_API_KEY, BFL_API_KEY). You can set multiple keys to enable multiple providers.

Claude Desktop

Add to ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "multimodal-mcp": {
      "command": "npx",
      "args": ["@r16t/multimodal-mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "sk-..."
      }
    }
  }
}

Cursor

Add to .cursor/mcp.json in your project root (or ~/.cursor/mcp.json globally):

{
  "mcpServers": {
    "multimodal-mcp": {
      "command": "npx",
      "args": ["@r16t/multimodal-mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "sk-..."
      }
    }
  }
}

VS Code (GitHub Copilot)

Add to .vscode/mcp.json in your project root:

{
  "servers": {
    "multimodal-mcp": {
      "command": "npx",
      "args": ["@r16t/multimodal-mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "sk-..."
      }
    }
  }
}

Windsurf

Add to ~/.codeium/windsurf/mcp_config.json:

{
  "mcpServers": {
    "multimodal-mcp": {
      "command": "npx",
      "args": ["@r16t/multimodal-mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "sk-..."
      }
    }
  }
}

Cline

Add to ~/Library/Application Support/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json:

{
  "mcpServers": {
    "multimodal-mcp": {
      "command": "npx",
      "args": ["@r16t/multimodal-mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "sk-..."
      }
    }
  }
}

License

MIT