Give AI coding agents a way to verify running games
The coding agent writes the game; Gua provides the runtime UI layer it uses to check the result. Through gui-mcp, the agent can access the Semantic UI Tree, actions, waits, logs, screenshots, recordings, visual comparisons, and test execution.
The AI development loop
- Implement the feature
The AI coding agent changes Godot GDScript/C# or Unity C# and the game UI.
- Launch the game
The Godot GDScript add-on collects Controls. The Unity UPM package automatically collects runtime UI Toolkit, uGUI, and TextMeshPro controls. Both publish through the Gua WebSocket bridge.
- Inspect the Semantic UI Tree
The agent calls
get_ui_treeand identifies a control from its ID, role, text, and state without depending on engine-specific object APIs. - Operate and wait for the result
It sends click, value, selection, scroll, or key actions and waits for the expected UI state.
- Verify from evidence and fix
On failure, it inspects the tree, engine logs, screenshot, or visual diff, changes the implementation, and runs the test again.
Engine integration
| Engine | Runtime UI source | How Gua starts |
|---|---|---|
| Godot 4.7 | Standard Control tree | The GDScript add-on attaches an automatic adapter and starts the bridge. |
| Unity 6 | UI Toolkit, uGUI, and TextMeshPro runtime UI | The UPM package starts automatically before scene load and runs collection after each frame. |
| Both | Semantic UI Tree v2 | gui-mcp uses the same locators, actions, waits, logs, screenshots, Visual, and Recording tools. |
How it maps to Playwright on the web
| Web UI automation | Game UI automation with Gua |
|---|---|
| Browser | Running game |
| DOM | Semantic UI Tree |
| locator | Semantic locator by ID, role, text, and state |
| click / fill / keyboard | Semantic actions such as click / set value / press key |
| expect / wait | GuaAssertions / wait_for_node |
| screenshot / visual test | Gua screenshot / visual baseline comparison |
Frequently asked questions
Can Gua test Godot UI?
Yes. The recommended Godot 4.7 GDScript add-on reflects standard Controls and supports semantic locators, actions, waits, assertions, screenshots, and visual comparison.
Can Gua test Unity UI?
Yes. The Unity 6 UPM package automatically reflects runtime UI Toolkit, uGUI, and TextMeshPro controls on supported Windows x64 Mono builds. The AI uses the same semantic MCP tools as it does with Godot.
Can an AI agent operate a game with Gua?
Yes. An MCP-enabled AI client can use gui-mcp to read the live tree, act on Controls, wait for state, inspect logs and screenshots, and run tests.
Is Gua a Godot or Unity Editor MCP?
No; the roles are complementary. Editor tools operate the development environment, while Gua observes, operates, and verifies the runtime UI of a running Godot or Unity game.
Does Gua locate buttons with image recognition?
No. Primary targeting uses IDs, roles, text, and state published by the game. Screenshots remain available as evidence for rendered output and visual comparison.
Does Gua replace the game engine or coding agent?
No. Gua is the automation layer connecting a game runtime to tests, Inspector, and AI coding agents.