What is a Semantic UI Tree?
Instead of analyzing a rendered image, Gua represents the UI structure and meaning as one frame-consistent snapshot. A consumer can tell that a node is a visible, enabled Start Game button that supports click. This is not a dump of engine scene nodes or GameObjects; adapters translate user-facing controls into one shared protocol.
Information on the complete tree
| Field | Meaning |
|---|---|
screen | The current logical screen, such as title or settings. |
nodes | The currently published UI nodes. parentId expresses hierarchy. |
frameSequence | Increments for every frame published by the adapter. |
revision | Increments only when the screen, nodes, state, bounds, or supported actions change. |
sessionEpoch | Identifies the current test-isolation session so stale snapshots are not reused after reset. |
{
"schemaVersion": 2,
"screen": "title",
"frameSequence": 42,
"revision": 7,
"nodes": [
{
"id": "title/start",
"role": "button",
"label": "Start Game",
"text": "Start Game",
"visible": true,
"enabled": true,
"bounds": { "x": 520, "y": 430, "w": 240, "h": 64 },
"actions": ["click", "focus"]
}
]
}Information on each UI node
id, parentId, and role describe which control this is, what it does, and where it belongs.
label, text, and value describe its name, displayed text, entered value, or slider value.
visible, enabled, focus, hover, pressed, checked, selected, caret, selection, scroll, and range fields describe its current behavior.
bounds uses physical viewport pixels. actions lists the operations structurally supported by that kind of node.
What can you do with it?
Read the current screen, visible controls, text, values, selection, focus, and scroll state.
Combine id, role, name, text, parent scope, visibility, and enabled state to find a control.
Send click, focus, set_value, set_checked, select, scroll, or press_key to a matching node.
Wait until Loading appears, a button becomes enabled, or a checkbox becomes checked instead of relying on a fixed sleep.
How it differs from World and game input
Observe and operate on-screen buttons, labels, dialogs, and input fields.
Observe opted-in doors, enemies, objectives, and checkpoints outside the UI. World v1 is read-only.
Drive gameplay intent such as jump or move.
Operate keyboard, pointer, and gamepad controls close to the physical device model.