Hello,

I'm currently working deeply on my editor. As things are starting to get more complex, I often end up in a situation where the amount of, lets call it information handling, of my code is not good enough for some editor features. I have, for example, a visual scripting system, much like the one used in Unreal4. In each such visual script, I can have a number of variables, and like in Unity, all variables marked "public" can be seen and modified from outside. I also have an entity component system, where I have a specific component for attaching a visual script to the entity.

This component should show and allow to modify all public variables. As for this, it currently has a map<string, string>, whose values are written to the visual scripts variables, and also serialized. This is how the class looks:

		class Event :
			public ecs::Component<Event>
		{
		public:

			Event(void);
			Event(const std::wstring& stName);
			~Event(void);

			std::wstring stName;
			core::EventInstance* pInstance;
			std::map<std::wstring, std::wstring> mVariables;

			static const ecs::ComponentDeclaration& GenerateDeclaration(void);
		};

Whenever the name of the visual script (here called "event") is set (thats what the stName-variable is for), the component gets notified via callback and is allowed to query the scripts variables and modify the map accordingly. That part works so far.

However, what I don't know how to make work, is what happens when a variable is modified in the visual script. How am I supposed to deliver that information to wherever it is needed in the editor? Don't get me wrong, I know how to transfer such events, via messaging, callbacks, signals... but my question is rather, how I could do this in an efficient and architecturally nice way. The thing is, I don't simply want to add a signal for everything to the class like that:

class EventInstance
{
public:
    
    // ...

    core::Signal<const EventVariable&> SigVariableAdded;
    core::Signal<const std::wstring&> SigVariableRemoved;
    core::Signal<const EventVariable&> SigVariableTypeChanged;

    // ...
}

For the reason being, that I really only every need this for the editor, and the classes here are shared between editor and game. I have a common DLLs that both editor and "player" use, as well as a few plugins that are equally shared. I'm very hasitating to add stuff that is only every needed for that very aspect of "live-feedback" in the editor, which wastes resources and runtime in case of the game, where it is simply not needed. I also don't want to rewrite my classes just for the editor...

So what I'm looking for is basically a way to implement such information exchanging for the editor on top of the already existing classes. To put it in a answerable question: How would/did you implement such a thing in an editor? I'm glad for any suggestion, I have a few things in mind, but I don't want to bias anyone. Note that this is just an example, that thing actually happens in a lot of places, so I hope you can help me out without me having to show too much code, but if you need to see something more of the architecture, let me know.

It is not necessary to give each possible change an identity. Due to the fact that exposed variables of a script can occur in any combination of count, names, actual types, order, and whatnot, a GUI view onto them has to be implemented in a dynamic way. The set of variables is like the data model, and rendering the view means to investigate the set with respect to all aspects mentioned above, so the view needs perhaps to be adapted not only with respect to values but also to its structure.

With this mechanism just a single notification is needed, which is send on any change. It can even be send only at defined moments, e.g. whenever the script completed, is paused, or has ran for some time, so that the notification is not send by the script but by the script processor.

You need not be so restrictive though. You can decide to send some kinds of notifications (e.g. value(s) changed and structure changed) but still for the entire set of variables of a particular script. Or ...

Obviously, retained mode GUI's are not well suited for this kind of solution, simply because the proposed GUI mode reflects a data model state directly instead of a "canned" version. This is a flaw you have to live with in case that you've build your editor GUI with standard OS elements (you probably have). In such a case I suggest to create a mediator that is notified and that manages the view's update accordingly.

Additionally, you can implement a notification center where all notifications are send to, and where notifications are dispatched to observers. This would mean a single pointer and conditional method call for the script, script node, or the script processor (in dependence on how you execute scripts), and the "complex" stuff is out-sourced and build only in case of the editor anyway.

Thanks, that gives me some good ideas about how to progress. However, I'm still lost on a few points...

It is not necessary to give each possible change an identity. Due to the fact that exposed variables of a script can occur in any combination of count, names, actual types, order, and whatnot, a GUI view onto them has to be implemented in a dynamic way. The set of variables is like the data model, and rendering the view means to investigate the set with respect to all aspects mentioned above, so the view needs perhaps to be adapted not only with respect to values but also to its structure.

The problem with this is, that its not just about displaying the actual variables & values of the script itself, but of the script as it is being used on a component, and furthermore being able to serialize the variables of said component. Its actually that link between component & script that is causing most of my troubles, I already have view for the variables of a script in the script editor.

So the thing in my system is, all scripts are stored in a global cache. The component gets a name key to a script, and via lazy initialization requests the script instance when the components update code is run. This does however only happen in "game" mode, or in the actual game itself - obviously I don't want anything updating and running while working in the editor. So what that means is that during "editor" mode, the component has no idea about what actual script belongs to it. But it needs to somehow get informed about the variable changes of a certain script. Do you know any good way to handle this?

Obviously, retained mode GUI's are not well suited for this kind of solution, simply because the proposed GUI mode reflects a data model state directly instead of a "canned" version. This is a flaw you have to live with in case that you've build your editor GUI with standard OS elements (you probably have). In such a case I suggest to create a mediator that is notified and that manages the view's update accordingly.

Well I am rolling the GUI myself, with rendering, event dispatching etc... all in my hand. I obviously implemented it in a retained fashion, but is there anything I could do here to make such things easier, as you hinted at?

Additionally, you can implement a notification center where all notifications are send to, and where notifications are dispatched to observers. This would mean a single pointer and conditional method call for the script, script node, or the script processor (in dependence on how you execute scripts), and the "complex" stuff is out-sourced and build only in case of the editor anyway.

I haven't really dealt with more complex build system, so its not quite clear to me what you mean with "out-sourcing" here? Also, how would I go about only bulding the "complex stuff" in case of the editor? I have a specific editor exe, but the "runtime-libary" is so far the same for game & editor. So are you only talking about the stuff that gets built with the editor-exe, or are you suggestion having some defines for a specific editor-dll?

EDIT: Also, one thing that I forgot to mention, for said reasons I also need a way to store variable values in the component outside of the actual script instance. You know, so I can load and save them independately, etc... how would you do that?

The problem with this is, that its not just about displaying the actual variables & values of the script itself, but of the script as it is being used on a component, and furthermore being able to serialize the variables of said component. Its actually that link between component & script that is causing most of my troubles, I already have view for the variables of a script in the script editor.

So the thing in my system is, all scripts are stored in a global cache. The component gets a name key to a script, and via lazy initialization requests the script instance when the components update code is run. This does however only happen in "game" mode, or in the actual game itself - obviously I don't want anything updating and running while working in the editor. So what that means is that during "editor" mode, the component has no idea about what actual script belongs to it. But it needs to somehow get informed about the variable changes of a certain script. Do you know any good way to handle this?

This confuses me. Could you please describe what should be shown by the view in which situation?

Well I am rolling the GUI myself, with rendering, event dispatching etc... all in my hand. I obviously implemented it in a retained fashion, but is there anything I could do here to make such things easier, as you hinted at?

The value shown by a widget is usually a copy in the private use area of the widget, and it is copied back on some "okay" but dropped with the widget on some "cancel". A GUI like the one we're discussing is intended for tweaking, perhaps even for debugging. Instead of copying a value forth and back, it may get a pointer to the variable and read / write it directly. Whether or not this is easier depends a bit on how the environment works.

I haven't really dealt with more complex build system, so its not quite clear to me what you mean with "out-sourcing" here? Also, how would I go about only bulding the "complex stuff" in case of the editor? I have a specific editor exe, but the "runtime-libary" is so far the same for game & editor. So are you only talking about the stuff that gets built with the editor-exe, or are you suggestion having some defines for a specific editor-dll?

I mean that the notification stuff is as most as possible implemented outside of the objects that handle scripting in the runtime part of the engine. E.g. registering of observers / listeners and dispatching of events can be concentrated in a NotificationCenter objects, so that script nodes / scripts / script processors / script components (whatever it will be in the end) need just a single pointer to the NotificationCenter and invoke it if it is not null.

When the editor starts then it instantiates a NotificationCenter and makes it public to the script management. When the player starts then it does not instantiate the center, and the pointer within the script management is left null. The script management, whenever it instantiates a new script, forwards its current pointer, be it valid or null.

Whether this is done by conditional code, or is done in an editor specific DLL, or is done by static linking only into the editor EXE is a question of taste.

EDIT: Also, one thing that I forgot to mention, for said reasons I also need a way to store variable values in the component outside of the actual script instance. You know, so I can load and save them independately, etc... how would you do that?

It depends on how public variables of a script are managed.

Is there an external Variable object where the belonging script nodes refer to? In that case a centralized management would do.

Or has each script node its own internal storage for that? Then iterating and calling a ScriptNode::presetFrom(std::map) may be used.

FWIW: My own scripting system is, as most parts of my engine, data driven. This means bytecode in this case, which is executed by a script processor. It uses a "blob" at runtime, essentially a bank of registers for timers, variables, and so on. Instantiation is done by cloning a prototype of the blob (not by copying the script bytecode itself). Post-initialization means to override a couple of registers with registers stored elsewhere (the component in your case). This is probably totally different than your way.

This confuses me. Could you please describe what should be shown by the view in which situation?

Certainly, I also attached a screenshot of what the whole things looks like. In the "event"-view window in the upper right corner of the screen, you can see what a script looks like. In the upper right of that window is a list of variables. The selected variable "Text" is marked public. As you can see in the lower left corner, the seleced entity has a component "Event" which references the shown Event, therefore all public variables, in this case only "Text" should be shown - with a value specific to that component. This does work in that case, but only because "Text" was already declared at load time. In case another public variable is added, it should also be shown in the view. Which wouldn't be all that complicated, except that this is the entity/component-view, which per se has nothing to do with the script/event. So the question really is not how to display the variables, but how to draw the connection between script - variables - component. Any good ideas for that?

The value shown by a widget is usually a copy in the private use area of the widget, and it is copied back on some "okay" but dropped with the widget on some "cancel". A GUI like the one we're discussing is intended for tweaking, perhaps even for debugging. Instead of copying a value forth and back, it may get a pointer to the variable and read / write it directly. Whether or not this is easier depends a bit on how the environment works.

Interestingly, the component view I'm talking about is already sort of doing that - every entry shown here has a pointer to the certain variable of the component. This is already required by the component-system for automatic serialization, so it works well in in that area.

onditional code, or is done in an editor specific DLL, or is done by static linking only into the editor EXE is a question of taste.

I mean that the notification stuff is as most as possible implemented outside of the objects that handle scripting in the runtime part of the engine. E.g. registering of observers / listeners and dispatching of events can be concentrated in a NotificationCenter objects, so that script nodes / scripts / script processors / script components (whatever it will be in the end) need just a single pointer to the NotificationCenter and invoke it if it is not null.

When the editor starts then it instantiates a NotificationCenter and makes it public to the script management. When the player starts then it does not instantiate the center, and the pointer within the script management is left null. The script management, whenever it instantiates a new script, forwards its current pointer, be it valid or null.

Whether this is done by conditional code, or is done in an editor specific DLL, or is done by static linking only into the editor EXE is a question of taste.

Ok, that makes sense - a nullptr-check in case of a universal DLL for the game shouldn't be too hard though, especially since there should rarely ever be a change worth of notification on a variable in game mode - I'm still glad to see that this is a plausible technique for handling editor stuff, I also did that for storing load-information for textures, meshes etc... by having a seperate "LoadInfo"-struct that would only be set in editor mode. I always considered this a bit cheap and ugly, but seems its OK after all.

It depends on how public variables of a script are managed.

Is there an external Variable object where the belonging script nodes refer to? In that case a centralized management would do.

Or has each script node its own internal storage for that? Then iterating and calling a ScriptNode::presetFrom(std::map) may be used.

Its basically a variable object that the script instance holds on to, and whose value can be manipulated from outside.

FWIW: My own scripting system is, as most parts of my engine, data driven. This means bytecode in this case, which is executed by a script processor. It uses a "blob" at runtime, essentially a bank of registers for timers, variables, and so on. Instantiation is done by cloning a prototype of the blob (not by copying the script bytecode itself). Post-initialization means to override a couple of registers with registers stored elsewhere (the component in your case). This is probably totally different than your way.

Yeah, I do have a much mor primitive and probably worse approach. Every script node is basically a C++-class, and the code flow is handled via Signals/Slots and direct calls in the class ( "CallOutput(index)", etc...). At least its easy to work with, but I'm sure performance sucks pretty hard, since it requires a lot of "virtual" and jumps all around memory for redirecting the flow, as well as requesting input arguments for nodes, etc... I think I could make it better by directly hard-wiring the connections via pointers, but thats a different topic...

Certainly, I also attached a screenshot of what the whole things looks like. In the "event"-view window in the upper right corner of the screen, you can see what a script looks like. In the upper right of that window is a list of variables. The selected variable "Text" is marked public. As you can see in the lower left corner, the seleced entity has a component "Event" which references the shown Event, therefore all public variables, in this case only "Text" should be shown - with a value specific to that component. This does work in that case, but only because "Text" was already declared at load time. In case another public variable is added, it should also be shown in the view. Which wouldn't be all that complicated, except that this is the entity/component-view, which per se has nothing to do with the script/event. So the question really is not how to display the variables, but how to draw the connection between script - variables - component. Any good ideas for that?

Let's see ... ;)

The editor is instructed to make a script variable public. This action is not reflected by the said view, because the view observes the state of the component, and the fact that another variable is made public is not yet seen by the component. So we need a mechanism that tells the component to add another property to self, matching the new public variable within the script. Right?

(1) Observation.

All script components register self with the notification center for being interested in changes to their script (i.e. the script name can be used as "subject"). The editor tells the notification center of the event "interface changed" with the script's name as subject. All belonging components react accordingly by adding a new property, and sending an event "state changed" with self as subject. The view, itself registered with the notification center, of course, reacts accordingly.

While this is all good w.r.t. OOP, it introduces editor stuff into the components which by itself should be runtime related objects. So this solution mixes responsibilities. I would not do so.

(2) Direct editing.

The editor known of the possible side effects of making a variable public, and works accordingly. It iterates the pool of components / the pool of script components / the pool of script components belonging to that particular script (however you manage the components), and instructs all matching components one by one to add a new property, and it causes the notification center to send a "state changed" event for each component.

In this solution there is no dependency of script components to editor stuff. They are just "under editing" together with their script, and the responsible unit is the editor. The solution is pure and direct.

One may argue that the editor does both editing the script as well as editing the component. If you are of the same meaning, then (3) may be your favorite.

(3) Observation by mediators.

Instead of registering the components themselves as observers as in (1), mediator objects especially for maintaining script components could be used. So adding the property is executed by the mediator onto the script component whenever it detects that an incoming "interface changed" event means a new public variable (send by the editor like in (1)).

As a variant only a single mediator object could be used (which has to iterate all belonging script components, similar as in (2) done by the editor itself). This would require just a single additional object but has the additional advantage that the mediator can be registered with the notification center on start-up, and need not be touched ever again.

Also here the script components do not depend on editor stuff. The mediator is an object that encapsulates the knowledge of side effects of editing the script. As such it is a kind of extension to the editor.

This solution is more flexible than (2) because in case that editing scripts may have side effects also onto other objects (besides script components), you just need another kind of mediator (or perhaps pure observer if the other objects belong to editor stuff). However, it is also a bit more complex and requires a little more extra work, because parts of editing are "outsourced", so to say.

I think that personally I prefer solution (3) with its variant of a single mediator. Whatever you do, your debug build should make a check inside the script components whether their set of variable properties matched the set of public variables.

Ok, that makes sense - a nullptr-check in case of a universal DLL for the game shouldn't be too hard though, especially since there should rarely ever be a change worth of notification on a variable in game mode - I'm still glad to see that this is a plausible technique for handling editor stuff, I also did that for storing load-information for textures, meshes etc... by having a seperate "LoadInfo"-struct that would only be set in editor mode. I always considered this a bit cheap and ugly, but seems its OK after all.

A notification center is there from the beginning or it is never. It doesn't disappear unexpectedly. So plain pointers are sufficient. And using null as "absent" indicator is fine anyway.

Its basically a variable object that the script instance holds on to, and whose value can be manipulated from outside.

So the script holds the bunch of variables which means it is somewhat central (opposed to being distributed to a couple of script nodes), then let the component call the script's presetFrom(map) method with the argument being the map of presets, so that the script can iterate its local map and copy values from the preset map into it.

BTW: From the screenshot you seem me to make a good job :) Keep on.

The editor is instructed to make a script variable public. This action is not reflected by the said view, because the view observes the state of the component, and the fact that another variable is made public is not yet seen by the component. So we need a mechanism that tells the component to add another property to self, matching the new public variable within the script. Right?

Yep, thats it.

Alright, thanks a lot then, option 3 seems desirable. I already have an option for adding an editor-module to my plugins, so that should do it. Then I only need to write the notification-center code for the scripts.

BTW: From the screenshot you seem me to make a good job Keep on.

Thank you, much appreciated :)