macOS Desktop Control

macOS Control Skill

A high-fidelity automation bridge for macOS (Darwin) that enables agents to perceive the desktop state and execute precise mouse and keyboard interactions.

🛠 Included Scripts

This skill leverages specialized wrappers located in the /scripts directory to interface with system-level binaries:

1. cliclick_wrapper.sh

A dedicated wrapper for the cliclick utility to handle synthetic input events.

• Path: scripts/cliclick_wrapper.sh
• Logic: Executes /opt/homebrew/bin/cliclick with passed arguments.
• Capabilities: Left/Right clicks, mouse movement, and keyboard emulation.

2. vision_wrapper.sh

The visual perception engine for the skill.

• Path: scripts/vision_wrapper.sh
• Logic: Utilizes the native macOS screencapture utility in silent mode (-x).
• Output: Generates a standard PNG at /tmp/claw_view.png.

🚀 Tool Specifications

see

Captures the current screen state for visual analysis.

• Returns: A string confirming the filepath of the capture.
• Use Case: Identifies UI elements, window positions, and application states.

click

Sends precise commands to the mouse and keyboard.

• Usage: click "c:x,y" (Click) or click "m:x,y" (Move).
• Syntax: Supports all cliclick standard notation including w: (wait) and t: (type).

⚙️ Requirements & Setup

1. Binary Dependency:

   brew install cliclick