hy-mt

<p align="center"> <img src="imgs/hunyuanlogo.png" width="400"/> </p>

<p align="center"> <b>English</b> | <a href="README_CN.md">简体中文</a> | <a href="README_TW.md">繁體中文</a> | <a href="README_JP.md">日本語</a> </p>

<p align="center"> <a href="https://hub.docker.com/r/neosun/hy-mt"><img src="https://img.shields.io/docker/pulls/neosun/hy-mt?style=flat-square&logo=docker" alt="Docker Pulls"></a> <a href="https://github.com/neosun100/hy-mt/stargazers"><img src="https://img.shields.io/github/stars/neosun100/hy-mt?style=flat-square&logo=github" alt="Stars"></a> <a href="https://github.com/neosun100/hy-mt/blob/main/License.txt"><img src="https://img.shields.io/badge/license-Tencent_Hunyuan-blue?style=flat-square" alt="License"></a> <a href="https://huggingface.co/tencent/HY-MT1.5-1.8B"><img src="https://img.shields.io/badge/🤗-HuggingFace-yellow?style=flat-square" alt="HuggingFace"></a> </p>

HY-MT Translation Service

🚀 All-in-One Docker deployment for Tencent HunyuanMT 1.5 translation model with Web UI, REST API, and MCP Server support.

✨ Features

• 🌐 38 Languages Support - Chinese, English, Japanese, Korean, French, German, Spanish, and 31 more
• 🎨 Modern Web UI - Dark/Light theme toggle, drag & drop file upload, real-time progress display
• ⚡ Streaming Translation - Server-Sent Events (SSE) for real-time output, perfect for long texts
• 🔧 Full Parameter Control - Temperature, Top-P, Top-K, repetition penalty adjustable
• 📚 Terminology Intervention - Custom term mapping for domain-specific translations
• 🤖 MCP Server - Model Context Protocol support for AI assistants (Claude, etc.)
• 🐳 One-Click Deployment - All-in-One Docker image with all models pre-downloaded
• 🔄 Smart GPU Management - Auto GPU selection, idle timeout, memory release
• 🔀 Multi-Model Support - Switch between 4 models (1.8B/7B, base/FP8) via UI or API

🎯 Model Selection Guide

| Model | VRAM | Speed | Quality | Recommendation | |-------|------|-------|---------|----------------| | HY-MT 7B | 16GB | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | 🏆 Best Choice - Highest quality, fast speed | | HY-MT 1.8B | 6GB | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ | Good for limited VRAM | | HY-MT 1.8B FP8 | 4GB | ⭐⭐⭐ | ⭐⭐⭐⭐ | For VRAM < 6GB | | HY-MT 7B FP8 | 10GB | ⭐⭐ | ⭐⭐⭐⭐⭐ | 7B quality with less VRAM |

💡 Tip: If you have 16GB+ VRAM, use HY-MT 7B for best results. FP8 models save memory but are slower due to runtime decompression.

📸 Screenshot

<p align="center"> <img src="docs/images/ui-screenshot-v2.0.1.png" width="800"/> </p>

🚀 Quick Start

Docker Run (Recommended)

# One command to start (uses 7B model by default)
docker run -d --gpus all \
  -p 8021:8021 \
  -v ./models:/app/models \
  --name hy-mt \
  neosun/hy-mt:latest

# Access Web UI
open http://localhost:8021

The Docker image (~43GB) includes all 4 models pre-downloaded. No external downloads needed!

Docker Compose

Create docker-compose.yml:

services:
  hy-mt:
    image: neosun/hy-mt:latest
    container_name: hy-mt
    ports:
      - "8021:8021"
    environment:
      - MODEL_NAME=tencent/HY-MT1.5-7B  # Recommended for 16GB+ VRAM
      - GPU_IDLE_TIMEOUT=300
      - HF_ENDPOINT=https://huggingface.co  # Use https://hf-mirror.com for China
    volumes:
      - ./models:/app/models
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    restart: unless-stopped

docker compose up -d

📋 Requirements

| Requirement | Minimum | Recommended | |-------------|---------|-------------| | GPU | NVIDIA GPU with 6GB+ VRAM | 16GB+ VRAM (for 7B model) | | CUDA | 11.8+ | 12.4+ | | Docker | 20.10+ | 24.0+ | | nvidia-docker | Required | - |

Verify GPU Support

# Check NVIDIA driver
nvidia-smi

# Check Docker GPU support
docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

📊 Performance Benchmark

Tested on NVIDIA L40S GPU, translating English to Chinese:

| Model | Short (61 chars) | Medium (530 chars) | Long (1.8K chars) | Extra Long (4.2K chars) | |-------|------------------|--------------------|--------------------|-------------------------| | HY-MT 7B | 0.4s | 4.4s | 17.7s | 43.0s | | HY-MT 1.8B | 0.4s | 3.6s | 14.0s | 32.3s | | HY-MT 1.8B FP8 | 1.1s | 10.8s | 38.1s | 92.9s | | HY-MT 7B FP8 | 2.9s | 28.5s | 115.6s | 274.1s |

⚠️ Why are FP8 models slower?

This is counter-intuitive but technically correct:

| Comparison | Speed Change | Reason | |------------|--------------|--------| | 1.8B FP8 vs 1.8B | 2.7x slower | Runtime decompression overhead | | 7B FP8 vs 7B | 6.4x slower | More parameters = more decompression |

FP8 quantization is designed to save VRAM, not to speed up inference. The model is stored in 8-bit format but needs to be decompressed to 16-bit for GPU computation at runtime. This decompression happens for every token generation.

When to use FP8:

• ✅ When VRAM is limited (< 16GB for 7B, < 6GB for 1.8B)
• ❌ Not for speed optimization
• ❌ Not for batch processing (speed loss accumulates)

See Benchmark Report for detailed analysis.

🔑 Key Optimization: Chunk Size

Critical finding: Smaller chunk size = Better translation quality

| Chunk Size | Quality | Notes | |-----------|---------|-------| | 500 chars | ❌ Poor | Mixed languages in output | | 300 chars | ⚠️ Fair | Some untranslated residue | | 150 chars | ✅ Excellent | Complete, accurate translation |

The service uses MAX_CHUNK_LENGTH=150 by default for optimal quality.

Why? HY-MT model tends to "slack off" on long inputs, only translating part of the content. Shorter chunks force the model to fully translate each segment.

See Optimization Guide for details.

⚙️ Configuration

Environment Variables

| Variable | Default | Description | |----------|---------|-------------| | PORT | 8021 | Service port | | MODEL_NAME | tencent/HY-MT1.5-7B | HuggingFace model name | | MODEL_PATH | ./models | Local model cache path | | GPU_IDLE_TIMEOUT | 300 | Auto-release GPU after idle (seconds) | | NVIDIA_VISIBLE_DEVICES | auto | GPU ID (empty = auto select) | | HF_ENDPOINT | https://huggingface.co | HuggingFace mirror URL |

Using .env File

# Copy example config
cp .env.example .env

# Edit as needed
vim .env

📖 API Usage

Basic Translation

curl -X POST "http://localhost:8021/api/translate" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Hello, how are you?",
    "target_lang": "zh"
  }'

Response: ``json { "status": "success", "result": "你好，你好吗？", "elapsed_ms": 358, "model": "tencent/HY-MT1.5-7B", "chunks": 1 } ``

Streaming Translation (SSE)

curl -N "http://localhost:8021/api/translate" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Long article to translate...",
    "target_lang": "en",
    "stream": true
  }'

With Terminology Intervention

curl -X POST "http://localhost:8021/api/translate" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Apple released iPhone 16",
    "target_lang": "zh",
    "terms": {"Apple": "苹果公司", "iPhone": "苹果手机"}
  }'

Output: 苹果公司发布了苹果手机16

File Upload Translation

curl "http://localhost:8021/api/translate/file" \
  -F "file=@document.txt" \
  -F "target_lang=zh" \
  -F "stream=true"

Switch Model

curl -X POST "http://localhost:8021/api/models/switch" \
  -H "Content-Type: application/json" \
  -d '{"model": "tencent/HY-MT1.5-1.8B"}'

📚 API Endpoints

| Endpoint | Method | Description | |----------|--------|-------------| | / | GET | Web UI | | /api/translate | POST | Translate text (supports streaming) | | /api/translate/file | POST | Upload and translate file | | /api/translate/batch | POST | Batch translation | | /api/translate/stream | POST | Streaming translation (SSE) | | /api/languages | GET | List supported languages | | /api/models | GET | List available models | | /api/models/switch | POST | Switch translation model | | /api/gpu/status | GET | GPU status and memory info | | /api/gpu/offload | POST | Release GPU memory | | /api/config | GET | Service configuration | | /health | GET | Health check | | /docs | GET | Swagger API documentation |

🌍 Supported Languages

| Language | Code | Language | Code | Language | Code | |----------|------|----------|------|----------|------| | Chinese | zh | English | en | Japanese | ja | | Korean | ko | French | fr | German | de | | Spanish | es | Portuguese | pt | Russian | ru | | Arabic | ar | Thai | th | Vietnamese | vi | | Italian | it | Dutch | nl | Polish | pl | | Turkish | tr | Indonesian | id | Malay | ms | | Hindi | hi | Traditional Chinese | zh-Hant | Cantonese | yue |

And 17 more languages. See /api/languages for full list.

🛠️ Tech Stack

• Model: Tencent HY-MT1.5 (1.8B & 7B)
• Backend: FastAPI + Uvicorn
• Frontend: Vanilla JS with Dark/Light Mode
• Container: NVIDIA CUDA 12.4 base image
• Streaming: Server-Sent Events (SSE)
• MCP: Model Context Protocol for AI integration

📁 Project Structure

hy-mt/
├── app_fastapi.py      # Main FastAPI application
├── mcp_server.py       # MCP Server for AI assistants
├── benchmark.py        # Performance benchmark script
├── templates/
│   └── index.html      # Web UI (Dark/Light theme)
├── docs/
│   ├── BENCHMARK_REPORT.md    # Performance test report
│   ├── OPTIMIZATION_GUIDE.md  # Long text optimization guide
│   └── QUICK_REFERENCE.md     # API quick reference
├── Dockerfile          # All-in-One Docker build
├── docker-compose.yml  # Docker Compose config
├── start.sh           # Quick start script
├── test_api.sh        # API test script
└── .env.example       # Environment config template

🔧 Advanced Usage

Manual Start (Development)

# Clone repository
git clone https://github.com/neosun100/hy-mt.git
cd hy-mt

# Install dependencies
pip install torch transformers accelerate fastapi uvicorn

# Run
python -m uvicorn app_fastapi:app --host 0.0.0.0 --port 8021

MCP Server Integration

For AI assistants like Claude Desktop, add to MCP config:

{
  "mcpServers": {
    "hy-mt": {
      "command": "python",
      "args": ["/path/to/hy-mt/mcp_server.py"],
      "env": {
        "HY_MT_API": "http://localhost:8021"
      }
    }
  }
}

Available MCP tools:

• translate - Translate text
• list_languages - Get supported languages
• list_models - Get available models
• switch_model - Switch translation model

See MCP_GUIDE.md for details.

🐛 Troubleshooting

| Issue | Solution | |-------|----------| | Model download slow | Set HF_ENDPOINT=https://hf-mirror.com (China mirror) | | GPU out of memory | Use quantized model: tencent/HY-MT1.5-1.8B-FP8 | | Container won't start | Check nvidia-smi and nvidia-docker installation | | Translation incomplete | Already optimized with chunk size 150 | | Container shows unhealthy | Wait 1-2 minutes for model loading |

📝 Changelog

v2.0.1 (2026-01-03)

• 🏆 Default model changed to HY-MT 7B (best quality & speed)
• 🩺 Added Docker HEALTHCHECK for container health monitoring
• 📦 Container status now shows (healthy) when ready

v2.0.0 (2026-01-03) - True All-in-One

• 🎯 All 4 models pre-downloaded in Docker image - No external downloads needed!
• 📦 Image size: ~43GB (includes all models)
• 🏆 Recommended: HY-MT 7B for best quality and speed
• 📊 Added performance benchmark report
• 🔧 Added benchmark.py for reproducible testing

v1.2.0 (2026-01-03)

• 🔀 Multi-model support (4 models: 1.8B, 1.8B-FP8, 7B, 7B-FP8)
• 🔄 Model switching via UI and API
• 📝 MCP Server: added list_models and switch_model tools
• 🐛 Fixed model name display in translation response

v1.0.0 (2026-01-03)

• 🎉 Initial release
• ✨ All-in-One Docker image
• ⚡ Streaming translation with SSE
• 🎨 Dark/Light theme Web UI
• 🔧 Long text optimization (chunk size 150)
• 🤖 MCP Server support

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

📄 License

This project is based on Tencent HunyuanMT. See License.txt for details.

🙏 Acknowledgments

• Tencent Hunyuan - Original HY-MT model
• HuggingFace - Model hosting

---

⭐ Star History


📱 Follow Us

<p align="center"> <img src="https://img.aws.xin/uPic/扫码_搜索联合传播样式-标准色版.png" width="300"/> </p>