Augent

Augent — The Audio Layer for Agents

<p align="center"> <picture> <img src="./images/logo.png" width="600" alt="Augent"> </picture> </p>

<p align="center"> <strong>The wormhole stays open. Fully local. Fully private.</strong> </p>

<p align="center"> <a href="https://github.com/AugentDevs/Augent/actions/workflows/tests.yml"><img src="https://img.shields.io/github/actions/workflow/status/AugentDevs/Augent/tests.yml?label=build&style=for-the-badge" alt="Build"></a> <img src="https://img.shields.io/badge/dynamic/toml?url=https://raw.githubusercontent.com/AugentDevs/Augent/main/pyproject.toml&query=$.project.version&label=version&style=for-the-badge" alt="Version"> <a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.10+-3776AB.svg?style=for-the-badge" alt="Python 3.10+"></a> <a href="https://discord.com/invite/DNmaZtaE7b"><img src="https://img.shields.io/badge/Discord-Join-5865F2.svg?style=for-the-badge&logo=discord&logoColor=white" alt="Discord"></a> <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge" alt="License: MIT"></a> </p>

<p align="center"> <a href="#mcp-tools">MCP Tools</a> · <a href="#cli">CLI</a> · <a href="#claude-code-skill">Claude Code Skill</a> · <a href="#openclaw">OpenClaw</a> · <a href="#web-ui">Web UI</a> · <a href="https://augent.app">Website</a> · <a href="https://docs.augent.app">Docs</a> · <a href="CHANGELOG.md">Changelog</a> · <a href="mailto:hello@augent.app">Contact</a> </p>

If the answer is trapped in audio or video, this is the way through.

Augent turns any audio or video source into structured, searchable intelligence for agents. Give it URLs or files. It downloads, transcribes, indexes, and stores everything in persistent memory. Search by keyword or meaning, find where concepts intersect, identify speakers, generate chapters and notes, batch process entire libraries, and more. One install, full pipeline, entirely on your machine.

If you want the quality info from content without sitting through it, the fastest way, this is it.

Preferred setup: run the one-line installer in your terminal. One command installs Augent, all dependencies, and the MCP server config. Works on macOS and Linux. Windows: install via pip. Works with Claude Code, Codex, and any MCP client. New install? Start here: Getting started.

<br />

Install

curl -fsSL https://augent.app/install.sh | bash

Works on macOS and Linux. Installs everything automatically.

Windows: pip install "augent[all] @ git+https://github.com/AugentDevs/Augent.git"

<details> <summary><strong>What does the installer do?</strong></summary>

<br />

The installer is a single bash script (source). Every dependency is open source:

| Dependency | What it does | |---|---| | Python | Runtime | | FFmpeg | Audio processing | | yt-dlp | Media downloads | | aria2 | Parallel downloads | | espeak-ng | TTS phonemizer | | faster-whisper | Speech-to-text | | PyTorch | ML framework | | sentence-transformers | Semantic search | | pyannote-audio | Speaker diarization | | Kokoro | Text-to-speech | | Demucs | Audio source separation | | FastAPI | Local web UI |

No background services. No telemetry. No sudo on macOS.

| | | |---|---| | Full breakdown | What each phase installs and why | | Manual install | Step-by-step for macOS, Linux, and Windows | | Uninstall | How to fully remove Augent |

</details>

<br />

<p align="center"> <picture> <img src="./images/install-demo.svg" alt="Install demo"> </picture> </p>

<br />

How it works (short)

graph TB
    A["URL / File"] --> B["Download + Separate"]
    B --> C["Transcribe"]
    C --> D["Memory + Tag"]

    D --> E["Search"]
    D --> F["Analyze"]
    D --> G["Export"]

    style A fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px
    style B fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px
    style C fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px
    style D fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px
    style E fill:#0a0a0a,stroke:#00f060,color:#00f060,stroke-width:2px
    style F fill:#0a0a0a,stroke:#00f060,color:#00f060,stroke-width:2px
    style G fill:#0a0a0a,stroke:#00f060,color:#00f060,stroke-width:2px

    linkStyle default stroke:#00f060,stroke-width:1.5px

Full architecture →

Project Structure

augent/
├── mcp.py          # MCP server — 22 tools for agents
├── config.py       # User configuration (~/.augent/config.yaml)
├── core.py         # Transcription engine (faster-whisper)
├── search.py       # Keyword search
├── embeddings.py   # Semantic search, chapters, visual scoring
├── speakers.py     # Speaker diarization (pyannote-audio)
├── separator.py    # Audio source separation (Demucs v4)
├── tts.py          # Text-to-speech (Kokoro)
├── memory.py       # Three-layer memory (SQLite)
├── graph.py        # Obsidian graph view (wikilinks, MOCs, frontmatter)
├── clips.py        # CLI clip extraction (audio segments around matches)
├── export.py       # Export formats (JSON, CSV, SRT, VTT, MD)
├── cli.py          # CLI interface
└── web.py          # Web UI (FastAPI)

<br />

MCP Tools

The primary way to use Augent. Any MCP client gets direct access to all tools.

Add to ~/.claude.json (global) or .mcp.json (project):

{
  "mcpServers": {
    "augent": {
      "command": "augent-mcp"
    }
  }
}

Restart Claude Code. Run /mcp to verify connection.

| Tool | Description | |:-----|:------------| | download_audio | Download audio from video URLs at maximum speed (1,000+ supported sites) | | transcribe_audio | Full transcription with metadata | | search_audio | Find keywords with timestamps and context snippets | | deep_search | Search audio by meaning, not just keywords (semantic search) | | take_notes | Take notes from any URL with style presets | | chapters | Auto-detect topic chapters in audio with timestamps | | batch_search | Search multiple files in parallel, built for batch workflows and agent swarms | | text_to_speech | Convert text to natural speech audio (Kokoro TTS, 54 voices, 9 languages) | | search_proximity | Find where keywords appear near each other | | identify_speakers | Identify who speaks when in audio (speaker diarization) | | separate_audio | Isolate vocals from music and background noise (Demucs v4) | | clip_export | Export a video clip from a URL for a specific time range | | highlights | Export MP4 clips of specific moments, auto-pick the best or target exactly what you want | | tag | Add, remove, or list tags on transcriptions for organized filtering | | visual | Extract visual context from video at moments that matter (query, auto, or manual) | | rebuild_graph | Rebuild Obsidian graph view data for all transcriptions | | search_memory | Search across ALL stored transcriptions by keyword or meaning | | list_files | List media files in a directory | | list_memories | List stored transcriptions by title | | memory_stats | View transcription memory statistics | | clear_memory | Clear stored transcriptions | | spaces | Download, check, or stop X/Twitter Spaces recordings |

Full tool reference →

<details> <summary>Example prompt</summary>

"Download these 10 podcasts and find every moment a host covers a product in a positive or unique way. Not just brand mentions, only real endorsements or life-changing recommendations. Give me the timestamps and exactly what they said: url1, url2, url3, url4, url5, url6, url7, url8, url9, url10"

<p align="center"> <picture> <img src="./images/pipeline.png" alt="Augent Pipeline — From URLs to insights in one prompt" width="100%"> </picture> </p>

</details>

<br />

CLI

Full CLI for terminal-based workflows. Works standalone or with any agent.

<picture> <img src="./images/cli-help.png" alt="Augent CLI"> </picture>

| Command | Description | |:--------|:------------| | audio-downloader "URL" | Download audio from video URL (speed-optimized) | | augent search audio.mp3 "keyword" | Search for keywords | | augent transcribe audio.mp3 | Full transcription | | augent proximity audio.mp3 "A" "B" | Find keyword A near keyword B | | augent memory search "query" | Search across all stored transcriptions | | augent memory stats | View memory statistics | | augent memory list | List stored transcriptions | | augent memory clear | Clear memory |

<br />

Eyes & Ears

Someone explains their entire workflow in a video. Augent transcribes it, builds the workflow files, maps the sequencing, the decision points, the tool stack. Every piece structured into something an agent can act on.

But some steps are inherently visual. Augent detects where visual context is needed and exports multiple screenshots at those moments, giving the agent frame-by-frame context of the flow being described. Audio intelligence plus visual context equals a complete, replicable system.

graph TB
    A["Expert explains workflow or automation"] --> B["Augent transcribes + structures"]
    B --> C["Builds workflow files + sequencing"]

    C --> D["Maps decision logic"]
    C --> E["Identifies tools + platforms"]
    C --> F["Flags visual gaps"]

    F --> G["Exports screenshots for context"]
    D --> H["Ready to run"]
    E --> H
    G --> H

    style A fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px
    style B fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px
    style C fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px
    style D fill:#0a0a0a,stroke:#00f060,color:#00f060,stroke-width:2px
    style E fill:#0a0a0a,stroke:#00f060,color:#00f060,stroke-width:2px
    style F fill:#0a0a0a,stroke:#00f060,color:#00f060,stroke-width:2px
    style G fill:#0a0a0a,stroke:#00f060,color:#00f060,stroke-width:2px
    style H fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px

Read more →

<br />

X/Twitter Spaces

Download or live-record Twitter/X Spaces audio. Auto-detects whether a Space is live or ended and handles both. Live Spaces record from the current moment using ffmpeg, ended Spaces download the full recording via yt-dlp. All downloads run in the background so your agent keeps working.

One-time setup

X/Twitter requires authentication to access Space audio. Any account works, including a burner.

1. Log into x.com in any browser
2. Open DevTools (F12 or Cmd+Option+I) > Application > Cookies > https://x.com
3. Copy the auth_token and ct0 values
4. Create ~/.augent/auth.json:

{"auth_token": "PASTE_HERE", "ct0": "PASTE_HERE"}

Tokens are stored locally and only sent to Twitter's servers to fetch audio. Augent never posts, DMs, follows, or modifies anything on your account. To revoke access, log out of Twitter or delete ~/.augent/auth.json.

<br />

Claude Code Skill

Claude and Codex already know how to use Augent's tools from their descriptions. The skill adds advanced workflows on top: multi-step note-taking pipelines, auto-tagging rules, translation flows, quiz formatting, and optimal search strategies.

mkdir -p ~/.claude/skills/augent
curl -o ~/.claude/skills/augent/SKILL.md \
  https://raw.githubusercontent.com/AugentDevs/Augent/main/skills/augent/SKILL.md

Works globally across all projects. One install, every conversation benefits.

<br />

OpenClaw

Augent is available as an OpenClaw skill on ClawHub.

Install via ClawHub:

npx clawhub@latest install augent

Or set up manually:

augent setup openclaw

If you installed Augent with curl -fsSL https://augent.app/install.sh | bash, OpenClaw is detected and configured automatically. These commands are only needed for manual setup or pip installs.

<br />

Obsidian Graph View

Every transcription builds a node. Every shared tag builds a connection. Your audio memory becomes a navigable knowledge graph, entirely automatic.

<picture> <img src="./images/obsidian-graph.png" alt="Augent knowledge graph in Obsidian"> </picture>

Every take_notes call, every transcription, every tag creates structure: YAML frontmatter, [[wikilinks]] between related content, and MOC hub files that cluster topics. Run rebuild_graph once to upgrade existing memory. The graph grows on its own from there.

Use Augent inside your main Obsidian vault, alongside your personal notes, journals, and projects. Everything compounds together. Full guide.

Using Claude Code or Codex with Obsidian? Set up augent-obsidian to make every .txt and .md file on your Mac open directly in Obsidian, with automatic sync for external edits.

<br />

Multilingual

Augent transcribes audio in its original language with full accuracy, powered by OpenAI's Whisper, supporting 99 languages including Chinese, French, Spanish, Japanese, Arabic, Hindi, Korean, German, Russian, Portuguese, and many more. Language is auto-detected, no configuration needed. Translation to English is handled by Claude (or your LLM), producing far better translations than any local model.

• When a transcription returns a non-English language, the MCP response includes a translation offer
• Accepting stores a clean English (eng) sibling file in memory alongside the original
• Both the original and translated versions appear in the Memory Explorer

<br />

Model Sizes

tiny is the default. Handles everything from clean studio recordings to noisy field audio. Use small or above for heavy accents, poor audio, or lyrics.

| Model | Speed | Accuracy | |:------|:------|:---------| | tiny | Fastest | Excellent (default) | | base | Fast | Excellent | | small | Medium | Superior | | medium | Slow | Outstanding | | large | Slowest | Maximum |

<br />

Configuration

Customize defaults and disable tools you don't need via ~/.augent/config.yaml:

# ~/.augent/config.yaml
model_size: tiny           # Default Whisper model
output_dir: ~/Downloads    # Default download directory
notes_output_dir: ~/Desktop # Notes, clips, TTS output
clip_padding: 15           # Seconds of padding around clips
context_words: 25          # Words of context in search results
tts_voice: af_heart        # Default TTS voice
tts_speed: 1.0             # TTS speed multiplier
disabled_tools: []         # Hide tools from MCP clients

Per-call arguments always override config. No config file needed, all values have sensible defaults.

Configuration docs →

<br />

Web UI

Local web interface. Runs 100% locally. No internet, no API keys, no data leaves your machine.

augent-web

Open: http://127.0.0.1:8282

Search view:

1. Upload an audio file or paste a YouTube/video URL to download audio directly
2. Enter keywords separated by commas
3. Click SEARCH and results stream live with timestamps and context
4. YouTube timestamps are automatically hyperlinked when the source is YouTube

Clip export:

• Click the film icon on any search result to create a visual region on the waveform, or drag on the waveform to select any range manually
• Nudge buttons (±1s / ±5s) on each edge for precise boundary adjustment
• Preview plays only the selected range so you hear exactly what will be exported
• Export MP4 downloads only the selected segment, not the full video
• Keyboard shortcuts: Space preview, Enter export, Esc close

Memory Explorer:

• Browse all stored transcriptions, including files transcribed via MCP or CLI. Every tool writes to the same memory.
• View full transcripts with clickable YouTube timestamps
• Delete individual transcriptions from memory
• Show Audio to reveal the source audio file in Finder
• Show Transcript to reveal the .md transcript file in Finder. Drag it into a Claude Code session to run the full MCP pipeline on a previously transcribed file.
• Share as HTML to download a self-contained, shareable transcript page
• Search across all memories by keyword to find matches across every transcription in your library

Source URL persistence: When audio is downloaded from any URL (YouTube, Twitter/X, TikTok, Instagram, SoundCloud, and 1000+ sites) the source URL is permanently stored by file hash. Any future search or transcription of that file, even weeks later or from a different path, automatically links back to the original source. No need to re-enter the URL.

<details> <summary>Web UI options</summary>

| Command | Description | |:--------|:------------| | augent-web | Start on port 8282 | | augent-web --port 8585 | Custom port |

</details>

<picture> <img src="./images/webui-1.png" alt="Augent Web UI - Upload"> </picture> <picture> <img src="./images/webui-2.png" alt="Augent Web UI - Results"> </picture>

<br />

Star History

<a href="https://www.star-history.com/#AugentDevs/Augent&type=date&legend=top-left"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=AugentDevs/Augent&type=date&theme=dark&legend=top-left" /> <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=AugentDevs/Augent&type=date&legend=top-left" /> <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=AugentDevs/Augent&type=date&legend=top-left" /> </picture> </a>

<br />

Contributing

PRs welcome. Open an issue for bugs or feature requests.

<br />

License

MIT