GGua REFERENCE
TOOL / MCP SERVER

MCP Server

gui-mcp is a thin stdio server that forwards an AI agent's tool calls to the WebSocket bridge running inside the game.

Start the game first

AI clientstdio →gui-mcpWebSocket →Game bridge→Gua adapter

The MCP server does not own UI state. The Godot application must start a bridge listening at ws://127.0.0.1:8765.

Register it with an MCP client

In a Bun environment, launch the published package with bunx. Adapt the following command and arguments to your client's configuration format.

Example MCP configurationjson
{
  "mcpServers": {
    "gua": {
      "command": "bunx",
      "args": ["gui-mcp@latest", "mcp"],
      "env": {
        "GUA_BRIDGE_URL": "ws://127.0.0.1:8765",
        "GUA_ARTIFACT_DIR": ".gua"
      }
    }
  }
}

Inspect and operate semantic UI

get_ui_tree / wait_for_node

Read the current tree and wait for node state changes.

click_node / focus_node

Click or focus a Control by stable semantic ID.

set_value / set_checked / select

Operate inputs, checkboxes, and selectable Controls semantically.

scroll / press_key

Send scrolling or key input to a semantic target.

get_screenshot / get_logs

Read the published PNG and runtime logs.

run_test

Execute a small wait/click sequence.

Observe game-world objects

get_world_object_tree, find_world_objects, and wait_for_world_object read the explicitly opted-in World Object Tree. World v1 remains observation-only; use semantic UI or a bounded game action to operate the game.

Drive game input

gui-mcp lists its fixed Semantic Game Action and raw keyboard, pointer, gamepad, and text tool surface. The host rejects an operation when its corresponding capability is not enabled. Stateful holds are owner-scoped, lease-bounded, and released by release_all_game_inputs or automatic cleanup. Actions marked requiresConfirmation require an explicit confirmation argument.

Recording and replay

start_recording / stop_recording

Record Semantic UI and game-input operations executed through MCP and return a schema v2 document.

save_recording / replay_recording

Save under the artifact root and replay an inline, named, or most recent recording.

Sensitive UI and game-input steps store only a secretKey. Replay resolves values from an in-memory secrets map and releases all replay-owned game input during final cleanup. Coordinate fallback is rejected by default.

Visual comparison

compare_screenshot

Compare the latest PNG against an explicitly named OS/renderer baseline. Baseline updates must be requested explicitly.

get_visual_artifacts

Return paths for actual, expected, diff, and comparison manifest artifacts.

Prompting an agent

Example prompttext
Read the Gua UI tree and find the “Start Game” button.
Click it by stable ID, then wait until “Loading...” is visible.
If the operation fails, inspect the logs and report both
the relevant UI tree state and the likely cause.

Read additional observations

When the game advertises observe_v1, gui-mcp exposes get_observe_snapshot, subscribe_observations, poll_observations, and unsubscribe_observations. The game registers values; the client cannot register getters remotely or upgrade its profile. Reconnect or a gap requires a fresh subscription. Registration and subscription.

Query an authorized physics provider

Use get_spatial_info/query_spatial_batch only when the native host explicitly enables its Testing/Debug spatial provider. They are bounded physics queries, separate from World radius searches, and are invalidated by reset or scene/provider changes. Provider setup and query limits.

Discover action value metadata

Explicitly use get_game_input_actions_v2/find_game_input_actions_v2 when semantic_game_input_metadata_v1 is available. valueSchema/examples describe values but do not bypass permission or confirmation. Input metadata guide.