IOU, A Game-User Interactivity System (GUI)
Graphical User Interfaces (GUI) are a mandatory in today's utility and entertainment application productions. Games in particular feature GUIs with heavily animated Widgets and unique input device control. When using the term GUI, most of us visualize two dimensional rectangular windows, buttons, menus. However I come to realize that the entire 3D Game-world is a Graphical User Interface and in most cases games incorporate a custom GUI system to support this interactivity.

What I describe herein is a design for Physics-powered Graphical User Interface that consolidates all User and CPU Interactivity: 2D/3D GUI Widgets, 3D Event Triggers, Player Controller, AI Agent Controller into one system. A Game-User Interactivity System, Input/Output & You (IOU). IOU's Design draws upon inspirations from several 2D/3D Drawing & Animation Applications such as Scaleform(Flash), Cinema4D, GIMP, PowerPoint, Movie Maker.

IOU is designed to support:

• 2D / 3D Animated Widgets
• 3D Cameras & Viewports
• Various Input Devices

Interactive Primitives: Input, Output, & User Devices
IOU has 3 Interactive Primitives (C++ Classes): InputDevice, OutputDevice, and UserDevice.

1. InputDevice manages User Input from peripherals such as Keyboard, Joystick, Joypad, Mouse, other. (Key, Button, Axis).
2. OutputDevice manages presentation of Visual Objects and Audio to the user.
3. UserDevice is an aggregate object that couples:
   • InputDevice for contact collision detection (button presses, etc).
   • Body, a Collision Component powered by Physics Engine for high performance bound collision detection/filtering and physics simulation.
   • Sprite, a Visual Component powered by High Performance Graphics Engine. Visual Components: 2D Image & Video, 3D Sprite/Mesh & Light, High Level 3D Entity.

In IOU, contact collision detection from input devices and bounds collision detection powered by Physics are the key mechanism of User-Application interaction and the driving principle behind IOU's design. IOU adopts the Entity Component Model in Engine Framework C++ Classes and Model-Viewer-Controller design pattern in the Script API (S3GXL) implementation, decoupling visual components from the collision component. Collision Detection generate Events, which are used select and execute scripts.


GUI Entities: Canvas, Widget, Pointer
IOU provides 3 GUI Entities (Script API) common to graphical user interfaces: Pointer, Widgets, Canvas . All other Interactive Entities inherit from these GUI Entities in a Interactive-centric Gameworld.

1. Canvas provides a single presentation point consolidates 3D Cameras, 2D Viewports, and Listener.
2. Widget provides a single interaction point for the direct manipulation of a given kind of data (1). A Widget has no specific mechanical operation (Behavior) until assigned, and a Widget can be assigned any Behavior.
3. Pointer, a Widget controlled by a User via InputDevice. Its used to interact with other Widgets. IOU decouples control of the Pointer directly from the mouse allowing it to be controlled other devices to include the keyboard.

Event State Machine (ESM) Model
IOU implements a Finite State Machine model composed of a finite number of states, transitions between those states, and actions. States are determined by Events. An Event is a state change based on Pointer/Widget collision detection, Control Input Detection, and Event Flags. These interactions are the basis for many types of Events in IOU. Hence, the Event State Machine. A Widget can execute a Script (Action) with each Event State. IOUs design revolves around Widget and Control Events.


Event-Driven Scripting
Event-driven Scripting Behaviors, Transitions, Styles, Themes, Schemes & Actions.

In most GUI's the mechanical operation of a Gadget (ie: Button, Slider, etc) is hardcoded. In IOU, this functionality is scripted and referred to as Behaviors. Behaviors define the logic for handling interactions, event states for Widgets and Controls. Games can require Widgets with unusual Behaviors. Scriptable Behaviors allows for many types of traditional and non-traditional Widgets to be created.

Games have very active GUIs complete with sound and animation. It was desired that IOU support sound and animation during events and in between. Transitions define logic for effects executed in between Event Changes. They were originally intended to be used for audio/visual effects during the transition of one event to another, but, can be used other actions.

Styles define Font properties based on Widget Event States and Themes define Border Images and Lines based on Widget Event States. Both Styles and Themes can use Script for visual effect fonts and borders are drawn.


Composite Widgets
Behaviors can also be scripted to use two or more Widgets to operate as a Composite Widget such as a Dropdown Box. These types of Behaviors rely on IOU's Widget Family Tree Data structure in which Parent|Child|Sibling relationships are established between Widgets during creation. In the Parent-Child relationship, the Parent usually influences the Child. Widgets that share the same Parent are Siblings. IOU uses a Link List for both operational and performance purposes.


Image-based Border Themes, Fonts, and SoundFX
IOU supports 2D Image-based Border Themes, 2D Animation Frames, and Fonts using Texture Atlases. The .PNG format is the recommended file format for Images.

IOUs Image-based Border Theme uses built-in drawing functions and image sets and IOU XML Theme Tag provides attributes for all edges and corners.

IOU BgSound Tag and Scripting makes it easy to apply sound effects to Widgets based on Events and Transitions.

XML-based file format for import/export.

Inspired by HTML. A XML/LUA S3GXL Implementation was adopted for the S3GP which includes IOU Widget/Canvas/Input configuration.


Messaging

References


To be continued...