webpilot

Webpilot

The web, through the eyes of a machine.

 

A semantic terminal browser that renders web pages as structured, numbered, interactive text. Built for LLM agents (via MCP), CLI-native developers, and automation pipelines.

Key insight: LLMs don't need to see a website — they need to understand it. Webpilot uses the accessibility tree (the same structure screen readers use) to represent any website as numbered elements that both humans and machines can interact with.

$ wpilot https://github.com

┌──────────────────────────────────────────────────────────────┐
  GitHub: Let's build from here
  https://github.com
└──────────────────────────────────────────────────────────────┘

  [1] › link         Sign in
  [2] › link         Sign up
  [3] ⌕ searchbox    Search GitHub
  [4] # h1           Build and ship software on a single platform
  [5] _ textbox      Enter your email address
  [6] ◆ button       ⟦ Sign up for GitHub ⟧

~ › click 1

  Navigated: github.com -> github.com/login

  [1] # h1           Sign in to GitHub
  [2] _ textbox      Username or email address
  [3] _ textbox      Password
  [4] ◆ button       ⟦ Sign in ⟧
  [5] › link         Forgot password?
  [6] › link         Create an account

~ › type 2 octocat@github.com
~ › type 3 ••••••••

~ › ss github-login.png

  Screenshot saved to github-login.png

The screenshot webpilot saves:

<img src="assets/github-example.png" alt="Webpilot screenshot of GitHub login with filled credentials" width="700" />

Why Webpilot?

| | Browsh | Lynx | Carbonyl | Webpilot | |---|---|---|---|---| | JavaScript support | Yes | No | Yes | Yes | | SPAs (React, Next.js, Vue) | Yes | No | Yes | Yes | | LLM-parseable output | No | No | No | Yes | | MCP server for AI agents | No | No | No | Yes | | Element interaction by ID | No | No | No | Yes | | State diffs | No | No | No | Yes | | Semantic (a11y tree) | No | Partial | No | Yes | | No pixels, no rendering | No | Yes | No | Yes | | Zero cost (runs locally) | Yes | Yes | Yes | Yes |

Install

npm install -g webpilot-cli
npx playwright install chromium   # one-time browser setup

Quick Start

# Interactive REPL
wpilot https://google.com

# JSON output for LLM agents
wpilot --agent https://google.com

# Pipe mode for scripting
echo 'goto https://example.com
extract --links' | wpilot --pipe

# Shorthand URLs
wpilot :3000              # → http://localhost:3000
wpilot google.com         # → https://google.com

MCP Server (for Claude, ChatGPT, etc.)

Webpilot ships as an MCP server — any LLM agent that supports the Model Context Protocol can browse the web through it.

Setup with Claude Desktop

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "webpilot": {
      "command": "npx",
      "args": ["-y", "webpilot-cli", "--mcp"]
    }
  }
}

Setup with VS Code / Copilot

Add to your .vscode/mcp.json:

{
  "servers": {
    "webpilot": {
      "command": "npx",
      "args": ["-y", "webpilot-cli", "--mcp"]
    }
  }
}

Available MCP Tools

| Tool | Description | |------|-------------| | web_navigate | Open a URL and get page state | | web_snapshot | Get current page as numbered elements | | web_click | Click element by [n] ID | | web_type | Type into input/textarea by [n] ID | | web_select | Select dropdown option | | web_scroll | Scroll up/down/top/bottom | | web_back | Go back in history | | web_extract | Extract text, links, tables, forms, or metadata | | web_eval | Execute JavaScript in page context | | web_screenshot | Capture page as PNG image | | web_tabs | List open tabs | | web_newtab | Open new tab | | web_close | Close browser session |

Example Agent Interaction

Agent: web_navigate("https://news.ycombinator.com")
→ 280 elements: [1] link "Hacker News", [2] link "new", [3] link "past" ...

Agent: web_click(5)
→ Navigated to article page, 42 elements

Agent: web_extract({ type: "text" })
→ Full article text extracted

Agent: web_back()
→ Back to Hacker News front page

Commands Reference

| Command | Description | |---------|-------------| | goto <url> | Navigate to URL | | click [n] | Click element by ID | | type [n] "text" | Type into form field | | select [n] "option" | Select dropdown option | | check [n] / hover [n] | Toggle checkbox / hover | | press <key> | Press keyboard key (Enter, Tab, etc.) | | back / forward | Browser history | | refresh | Reload current page | | scroll down\|up\|top\|bottom | Scroll the page | | find "text" | Search for text in elements | | show | Re-display current page state | | extract --text\|--links\|--tables\|--forms\|--meta | Extract structured content | | eval "js" | Execute JavaScript | | screenshot [path] | Save screenshot | | source | View page HTML | | tabs / tab [n] / newtab / closetab | Tab management | | help | Show all commands |

Three Output Modes

• Human (default) — Colored, formatted for terminal reading
• Agent (--agent) — JSON structured output for LLM consumption
• Pipe (auto-detected) — Plain text for grep, awk, scripting

How It Works

Website → Playwright (headless Chromium) → CDP Accessibility Tree → Numbered Elements → You

1. Playwright launches headless Chromium — full JS, cookies, SPAs, everything works
2. The accessibility tree is extracted via Chrome DevTools Protocol — semantic structure, not pixels
3. Elements get numbered IDs — [1], [2], [3]... for easy targeting
4. After each action, a state diff shows what changed, not the entire page
5. You interact with simple commands: click [3], type [5] "hello"

Works With Everything

• localhost:3000 — your dev server
• Public websites — Google, GitHub, HN, anything
• React / Next.js / Vue / Angular / Svelte — full JS execution
• SPAs with client-side routing
• Sites behind login — cookies persist in session
• Dynamic content — JS runs before each snapshot
• Forms, dropdowns, checkboxes — full interaction
• Multi-tab browsing

Use Cases

• LLM agents browsing the web — Claude/ChatGPT navigate, fill forms, extract data via MCP
• E2E testing in CI — pipe commands, assert output, no flaky selectors
• Web scraping — extract links, tables, text from any JS-rendered page
• Accessibility auditing — see exactly what the a11y tree exposes
• SSH/headless environments — browse from any terminal, no GUI needed

Development

git clone https://github.com/luckysolanki902/webpilot.git
cd webpilot
npm install
npx playwright install chromium
npm run build    # → dist/index.js
npm run dev      # Watch mode

License

MIT