36 replies to this topic

[FDsagizi]

Hello!

Last version: http://www.gamedev.n...62#entry4937962

Edited by Andreas Jonsson, 07 June 2012 - 10:44 AM.

[FDsagizi]

The main task when you restart the script, is to restore all the global variables.

I made a simple type Data Recovery Tools, scripting classes, simple classes (POD), strings and arrays.

But I really do not know how restore asTYPEID_OBJHANDLE?

If, handles only refer to global variables - it would be easy enough to find the global index.

But the handle may refer to an array element, or the Property class .
What to do ?

(This is my code, which restores the value of variables)
http://www.everfall....hp?imd9m2nc8io6

struct GVar
{
	GVar(){
		typeId = -1;
		init = false;
	}

	// standard type
	asINT64 valueInt;

	int   typeId;

	// serialize  POD class... 
	std::vector<char> mem;

	// is init
	bool init;

	// debug name
	std::string name;

	// type == string
	std::string txt_val;

	// type == array
	std::vector<GVar> arr;

	void Store(void *ref, int refTypeId)
	{
		asIScriptEngine *en = Script::Manager::me()->engine;
		asIScriptModule *mod = Script::Manager::me()->mod;

		init = true;
		typeId = refTypeId;
		if( typeId & asTYPEID_OBJHANDLE )
		{
			///void *valueObj = *(void**)ref;
			// <img src='http://public.gamedev.net/public/style_emoticons/<#EMO_DIR#>/sad.gif' class='bbc_emoticon' alt=':(' />
		}
		else if( typeId & asTYPEID_MASK_OBJECT && typeId & asTYPEID_SCRIPTOBJECT )
		{
			asIScriptObject *obj = (asIScriptObject *)ref;
			asIObjectType *type = en->GetObjectTypeById( typeId );

			int count = obj->GetPropertyCount();
			for( int i =0; i < count; i++ )
			{	
				int child_id;
				type->GetProperty( i, 0, &child_id );

				GVar v;
				v.Store( obj->GetAddressOfProperty( i ), child_id );
				arr.push_back( v );
			}	
		}
		else
		{

			valueInt = 0;
			int size = en->GetSizeOfPrimitiveType(typeId);
			// if it is standart 
			if( size )
			{
				memcpy(&valueInt, ref, size);
			}
			// class...
			else
			{
				asIObjectType *type = en->GetObjectTypeById( typeId );
				
				std::string name = type->GetName();

				size = type->GetSize();

				if( typeId == string_type_id )
				{
					txt_val = ((CScriptString*)ref)->buffer;
				}
				else if( name == "array" )
				{
					CScriptArray *array = (CScriptArray*)ref;

					int arr_types = array->GetElementTypeId();
					for( int i =0; i < array->GetSize(); i++ ){
						GVar v;
						v.Store( array->At( i ), arr_types );
						arr.push_back( v );
					}
				}
				else
				{
					mem.resize( size );
					memcpy(&mem[0], ref, size);
				}
			}
				
		}
	}

	void Retrieve(void *ref)
	{
		if(!init)
			return;

		asIScriptEngine *engine = Script::Manager::me()->engine;
		asIScriptModule *mod = Script::Manager::me()->mod;

		if( typeId  & asTYPEID_OBJHANDLE )
		{
			//*(void**)ref = valueObj;
			/// <img src='http://public.gamedev.net/public/style_emoticons/<#EMO_DIR#>/sad.gif' class='bbc_emoticon' alt=':(' />
		}
		else if( typeId & asTYPEID_MASK_OBJECT && typeId & asTYPEID_SCRIPTOBJECT )
		{
			asIScriptObject *obj = (asIScriptObject *)ref;

			if( obj->GetPropertyCount() == arr.size() )
				for( int i =0; i < obj->GetPropertyCount(); i++ )
				{
					arr[i].Retrieve( obj->GetAddressOfProperty( i ) );
				}
		}
		else
		{
			// Is the primitive type compatible with the stored value?
			int size = engine->GetSizeOfPrimitiveType( typeId );
			if( size )
				memcpy(ref, &valueInt, size);			
			else
			{
				asIObjectType *type = engine->GetObjectTypeById( typeId );
				std::string name = type->GetName();

				if( typeId == string_type_id )
				{
					((CScriptString*)ref)->buffer = txt_val;
				}
				else if( name == "array" )
				{
					CScriptArray *array = (CScriptArray*)ref;
					array->Resize( arr.size() );

					for( size_t i =0; i < arr.size(); ++i )	{
						GVar &v = arr[i];
						v.Retrieve( array->At(i) );
					}
				}
				else
				{
					// POD class....
					memcpy( ref, &mem[0], mem.size());
				}
			}
		}
	}
};

void Script::Manager::reload( std::string &_file_name )
{	

	builder.setModule( engine, mod );

	std::string file_data = ResourceManager::me()->getFileData( _file_name );

	std::map<std::string, GVar > vars;


	if( file_data.empty() == false )
	{
		for( int i =0; i < mod->GetGlobalVarCount(); i++ )
		{
			const char *name;
			int type_id;
			mod->GetGlobalVar( i, &name, &type_id );

			GVar v;
			v.name = name;
			v.Store( mod->GetAddressOfGlobalVar( i ), type_id );
			vars[ name ] = v;
		}

		builder.AddSectionFromMemory( file_data.c_str() , _file_name.c_str() );
		builder.BuildModule();

		for( int i =0; i < mod->GetGlobalVarCount(); i++ )
		{
			const char *name;
			mod->GetGlobalVar( i, &name );
			
			GVar v = vars[name];
			v.Retrieve( mod->GetAddressOfGlobalVar( i ) );
		}


		Nuligine::Manager::me()->bindAllScripts();
	}
}

[Andreas Jonsson]

To be able to store handles you'll need to implement the techniques used with serialization. I can't go into details here, but basically it is about keeping track of all objects that have already been stored in an map, and when a handle is found it converts the pointer to an index into the map. When loading the objects again the same map is also filled with the pointers of the new objects, and when the handle is loaded it uses the index to get the real pointer from the map.

[FDsagizi]

well, I understand.
How to get a table(map) variables?

[Andreas Jonsson]

You build it during the serialization of the objects, e.g:


Store


- For each object
- - If pointer to object not in array, add it
- - store the object
- for each handle
- - find index to pointer in array
- - store the index



Restore


- For each object
- - load the object
- - store the new pointer in the array
- For each handle
- - load the index
- - set the handle to the pointer from the array






The array can be a simple std::vector<void*>, which will let you map pointer to index and back. Just make sure the array is populated in the same order for both storing and restoring so the indexes are not mixed up.

[FDsagizi]

what you suggested will not work.

If we iterate over the global variables, we can not find the variables themselves.

And the only handles that will not allow them to die!

Or something I did not understand correctly?

// after main() we reload script...
class A
{
	//...
}
class B : A
{
	string txt;
}

string @txt_handle;
A@ handle;

void main()
{
	B b;
	
	@handle = b;
	@txt_handle = b.txt;
}

[Andreas Jonsson]

If you find a handle that points to an object that has not yet been stored, you need to store that object too, and not just the index.

[FDsagizi]

If you find a handle that points to an object that has not yet been stored, you need to store that object too, and not just the index.

Oh yes This is not a trivial task!
Enough to look at this code, we'll see that to restore the handles - will pass away first to find the parent handle. Then just refer to his children.

Otherwise, come so we will copy the 3 object.

class B
{
	string txt;
}
class A
{
	B b;
}

string @txt_h;
A @a_h;
B @b_h;

void main()
{
	A a;
	
	@a_h = a;
	@b_h = a.b;
	@txt_h = a.b.txt;
}

[Andreas Jonsson]

It's not that difficult actually. You just need to write a recursive algorithm.

// pseudo code - don't expect it to compile
vector<void*> storedPointers;

void storeVariables(FILE *f)
{
  storedPointers.resize(0);
  for( int i = 0; i < module->GetGlobVarCount(); i++ )
  {
 	writeVariableData(f); // store the data that identifies the variable, like type, name, etc

 	// store the actual value of the variable
 	int typeId = module->GetGlobVarTypeId(i);
 	if( typeId & asTYPEID_OBJHANDLE )
 	{
    	void *ptr = *(void**)module->GetAddressOfGlobVar(i);
 	}
 	else
 	{
     	.... store the object or primitive data
 	}
  }
}

void storeObject(void *ptr, int typeId, FILE *f)
{
   // Determine if this object has already been stored or not
   int idx = storedPointers.indexof(ptr);
   if( idx != -1 )
   {
  	fputc('r', f); // indicates that we're storing a reference to a previous stored object
  	fwrite(&idx, sizeof(int), f); // store the index of the object
   }
   else
   {
  	// Add the object pointer to the list of stored pointers
  	storedPointers.push_last(ptr);

  	fputc('o', f); // indicate that we're storing the actual object
      
      // Store the actual data of the object
  	if( typeId & asTYPEID_SCRIPTOBJECT )
      {
     	// Enumerate each member of the script object, and call storeObject to the objects
     	...
      }
      else
      {
     	// Store the application registered object
     	...
      }  	
   }   
}

The stored data will be something like this:

string @txt_h	<< global variable txt_h
o                     	<< first time the object is stored (index 0)
""                     	<< empty string
A @a_h         	<< global variable a_h
o                     	<< first time the object is stored (index 1)
B A::b             	<< member A::b
o                     	<< first time the object is stored (index 2)
string B::txt    	<< member B::txt
r                       	<< object is already stored, so just store reference
0                      	<< index 0
B @b_h         	<< global variable b_h
r                      	<< object is already stored, so just store reference
2                     	<< index 2

When reloading the data you do the reverse.

The tricky part is when a handle points to a non-handle member. In this case when loading the handle you will create an object with the data, but when loading the member later on you need to use that object to set the value of the member, and then substitute the pointer in the handle for the actual member.

I suggest you look for tutorials on serialization. The theory is exactly the same whether you serialize scripts or C++ objects, or any other language for that matter.

[FDsagizi]

WitchLord
Thank you very much for your help and tips!!!

I got everything done and everything works!


the same way I did it as a module (as well as all the standard examples) and would like to give you a code that you added it to AngelScript - think to many people it will be useful!



Here is the code

ScriptReloader.h
http://www.everfall....hp?55dmnkzvxuzc
ScriptReloader.cpp
http://www.everfall....hp?3m8vcr75t0ty

ScriptReloader.h

//
// CScriptReloader
//

#ifndef SCRIPTRELOADER_H
#define SCRIPTRELOADER_H

#include <angelscript.h>
#include <vector>
#include <string>

BEGIN_AS_NAMESPACE

class CScriptReloader;

struct CScriptReloaderVariable
{
	CScriptReloaderVariable(){
		init();
	}
	CScriptReloaderVariable( CScriptReloaderVariable *_parent, std::string _name,  void *_ref, int _typeId ) 
	{
		init();

		name = _name;
		reloader = _parent->reloader;
		Store( _ref, _typeId );
	}

	void init(){
		handle_ptr = restore_ptr = NULL;
		type_id = 0;
		is_init = false;
		reloader = NULL;
	}

	// need to get engine, and module
	CScriptReloader *reloader;

	//  if standard type ( int, float )...
	int type_id;

	// if its class, need save type str type - "string" "myClass" then next find by decl
	std::string type_id_str;
	
	// name variable
	std::string name;

	// is initialize
	bool is_init;

	// 'this' pointer to variable
	void *ptr;

	// where handle references
	void *handle_ptr;

	// new address object, ie address the restoration
	void *restore_ptr;

	// serialize data
	std::vector<char> mem;

	// if type == string
	std::string txt_val;

	// Child's, or if type == array, or handle data( if handle not found in global space )
	std::vector<CScriptReloaderVariable> childs;

	// save the object and its children
	void Store(void *ref, int refTypeId );

	//restore the object and its children
	void Restore(void *ref);

	// you first need to save all the objects before you can save references to objects
	void StoreHandle();

	// first restore the objects, then we have their new address, and we can restore the handle.
	void RestoreHandle(); 

	// set type to this var
	void setType(int typeId );

	// if it class, return obj type
	asIObjectType *getType();

	// Get child by name variable
	CScriptReloaderVariable *child( std::string _name )
	{
		for( size_t i =0; i < childs.size(); i++ )
			if( childs[i].name == _name )
				return &childs[i];

		return nullptr;
	}

	// get all ptrs of the child
	void  childsPtr( std::vector<void*> *_ptrs )
	{
		_ptrs->push_back( ptr );

		for( size_t i=0; i < childs.size(); ++i )
			childs[i].childsPtr( _ptrs );
	}

	// find variable by ptr
	CScriptReloaderVariable *findByPtr( void *_ptr )
	{
		if( ptr == _ptr )
			return this;

		for( size_t i =0; i < childs.size(); i++ )
		{
			CScriptReloaderVariable *find = childs[i].findByPtr(_ptr );
			if( find )
				return find;
		}

		return nullptr;
	}

	// find variable by ptr but looking only at those in the references, which will create a new object
	CScriptReloaderVariable *findByPtrInCreated( void *_ptr )
	{
		// if this handle created object
		if( type_id & asTYPEID_OBJHANDLE && childs.size() == 1 )
		{
			if( childs[0].ptr == _ptr )
				return this;
		}

		if(  !(type_id & asTYPEID_OBJHANDLE ) )
		{
			for( size_t i =0; i < childs.size(); i++ )
			{
				CScriptReloaderVariable *find = childs[i].findByPtrInCreated(_ptr );
				if( find )
					return find;
			}
		}

		return nullptr;
	}

	// may be that the two references refer to the same variable. 
	// But this variable is not available in the global list. 
	// According to this reference will be restores it. 
	// And so two links are not created 2 variables, 
	// it is necessary to cancel the creation of one of them.
	void canselDublicate( CScriptReloaderVariable *_from );
};


// This class keeps a list of variables, then restores them after the reboot script.
// But you have to be careful with the change of signature classes, or 
// changing the types of objects. You can remove or add variables, functions, 
// methods. But you can not (yet) to change the variable type. 
//
//You also need to understand that after a reboot you should get a new id 
// FUNCTIONS, or class to call them from C + + code.
class CScriptReloader
{

public:
	
	// then store all the variables
	CScriptReloaderVariable root;

	asIScriptEngine *engine;

	asIScriptModule *mod;

	// Store all global variables, and handles
	void Store()
	{
		root.reloader = this;

		// first store global variables
		for( int i =0; i < mod->GetGlobalVarCount(); i++ )
		{
			const char *name;
			int type_id;
			mod->GetGlobalVar( i, &name, &type_id );
			root.childs.push_back( CScriptReloaderVariable( &root, name,  mod->GetAddressOfGlobalVar( i ), type_id ) );
		}

		// second store handles
		root.StoreHandle();
	}

	// Retrieve all global variables after reload script.
	void Restore()
	{
		// first restore global variables
		int var_count = mod->GetGlobalVarCount();
		for( int i =0; i < var_count; i++ )
		{
			const char *name;
			mod->GetGlobalVar(i, &name );

			CScriptReloaderVariable *v = root.child( name );
			v->Restore( mod->GetAddressOfGlobalVar( i ) );
		}

		// up all handles to new ptr
		root.RestoreHandle();
	}

};


END_AS_NAMESPACE

#endif

ScriptReloader.cpp

#include "stdafx.h"
#include <assert.h>
#include <string.h> // strstr
#include <stdio.h>  // sprintf
#include "scriptreloader.h"
#include "scriptstring.h"
#include "scriptarray.h"

using namespace std;

BEGIN_AS_NAMESPACE


void CScriptReloaderVariable::Store(void *ref, int _typeId)
{
	is_init = true;
	setType( _typeId );
	ptr = ref;

	if( type_id & asTYPEID_OBJHANDLE )
	{
		handle_ptr = *(void**)ref;
	}
	else if( type_id & asTYPEID_SCRIPTOBJECT )
	{
		asIScriptObject *obj = (asIScriptObject *)ref;
		asIObjectType *type = obj->GetObjectType();
		setType(  type->GetTypeId() );

		// Store childs 
		for(int i =0; i < type->GetPropertyCount(); i++ )
		{	
			int child_id;
			const char *child_name;
			type->GetProperty( i, &child_name, &child_id );

			childs.push_back( CScriptReloaderVariable( this, child_name, obj->GetAddressOfProperty( i ), child_id ) );
		}	
	}
	else
	{
		int size = reloader->engine->GetSizeOfPrimitiveType( type_id );
		
		if( size == 0 )
		{			
			if( type_id_str== "string" )
			{
				CScriptString *txt_ref = ((CScriptString*)ref); 
				txt_val = txt_ref->buffer;
			}
			else if( type_id_str == "array" )
			{
				CScriptArray *array = (CScriptArray*)ref;

				for( int i =0; i < array->GetSize(); i++ )
					childs.push_back( CScriptReloaderVariable( this, "", array->At( i ), array->GetElementTypeId() ) );
			}
			else if( getType() )
			{
				size = getType()->GetSize();	
			}
		}

		if( size )
		{
			mem.resize( size );
			memcpy(&mem[0], ref, size);
		}
	}
}

void CScriptReloaderVariable::Restore(void *ref)
{
	if( !this || !is_init || !ref )
		return;

	restore_ptr = ref;

	if( type_id  & asTYPEID_OBJHANDLE )
	{
		// if need create objects
		if( childs.size() == 1 )
		{
			asIObjectType *type = childs[0].getType();

			void *new_obejct = reloader->engine->CreateScriptObject( type->GetTypeId() );

			childs[0].Restore( new_obejct );	
		}
	}
	else if( type_id & asTYPEID_SCRIPTOBJECT )
	{
		asIScriptObject *obj = (asIScriptObject *)ref;
		asIObjectType *type = getType();

		// Retrieve children s
		for( int i =0; i < type->GetPropertyCount() ; i++ )
		{	
			const char *name_property;
			type->GetProperty(i, &name_property );
			
			child( name_property )->Restore( obj->GetAddressOfProperty( i ) );		
		}
	}
	else
	{
		if( mem.size() )
			memcpy( ref, &mem[0], mem.size());
		else
		{
			if( type_id_str == "string" )
			{
				CScriptString *ref_txt = ((CScriptString*)ref);			
				ref_txt->buffer = txt_val;
			}
			else if( type_id_str == "array" )
			{
				CScriptArray *array = (CScriptArray*)ref;
				array->Resize( childs.size() );

				for( size_t i =0; i < childs.size(); ++i )	
					childs[i].Restore( array->At(i) );
			}
		}
	}
}

void CScriptReloaderVariable::canselDublicate( CScriptReloaderVariable *from )
{
	std::vector<void*> ptrs;
	from->childsPtr( &ptrs );

	for( size_t i=0; i < ptrs.size(); ++i )
	{
		CScriptReloaderVariable *find = reloader->root.findByPtrInCreated( ptrs[i] );

		while( find )
		{
			// cancel create object
			find->childs.clear();

			// Find next link to this ptr
			find = reloader->root.findByPtrInCreated( ptrs[i] );
		}
	}
}

void CScriptReloaderVariable::StoreHandle()
{
	// Find to 
	if( handle_ptr )
	{
		CScriptReloaderVariable *handle_to = reloader->root.findByPtr( handle_ptr );
		
		// if handle is not found in global space...
		if( handle_to == nullptr )
		{
			asIObjectType *type = getType();

			CScriptReloaderVariable need_create = CScriptReloaderVariable( this, name, handle_ptr, type->GetTypeId() ); 

			canselDublicate( &need_create );

			childs.push_back( need_create );
		}
	}
	// Childs...
	for( size_t i=0; i < childs.size(); ++i )
		childs[i].StoreHandle();
}

void CScriptReloaderVariable::RestoreHandle()
{
	CScriptReloaderVariable *handle_to = reloader->root.findByPtr( handle_ptr );

	// Restore handle!
	if( restore_ptr && handle_to && handle_to->restore_ptr )
		*(void**)restore_ptr = handle_to->restore_ptr;
	
	// Childs...
	for( size_t i=0; i < childs.size(); ++i )
		childs[i].RestoreHandle();
}


void CScriptReloaderVariable::setType(  int _typeId )
{
	type_id = _typeId;

	asIObjectType *type = reloader->engine->GetObjectTypeById( type_id  );

	if( type )
		type_id_str = type->GetName();
}

asIObjectType *CScriptReloaderVariable::getType()
{
	if( !type_id_str.empty() )
	{
		int new_type_id = reloader->mod->GetTypeIdByDecl( type_id_str.c_str() );
		return reloader->engine->GetObjectTypeById( new_type_id );
	}	

	return nullptr;
}

END_AS_NAMESPACE

If necessary, I can make an example of how to use it!
Although it's easy!

	CScriptReloader reloader;
	
	reloader.engine = engine;
	reloader.mod = mod;

	reloader.Store ();

		// Reload scripts...

	reloader.Restore ();

[Andreas Jonsson]

Thanks a lot for the contribution. I agree that many will likely find it useful.

I'll work on the implementation and make a little more generic so it can be used by everyone, and then add it as an add-on for the library.

[FDsagizi]

Thanks a lot for the contribution. I agree that many will likely find it useful.

I'll work on the implementation and make a little more generic so it can be used by everyone, and then add it as an add-on for the library.

Hello. I would like to know how you plan to make contact with the objects?

According to the index variable or her name?

[Andreas Jonsson]

You mean when reloading the stored data? It has to be by the name, as the index may not be the same.

[FDsagizi]

You mean when reloading the stored data? It has to be by the name, as the index may not be the same.

Yes!

Exemple code to reload

int var; index 0
bool var3; index 1

then store.

new script.

int var; index 0;
float time; index 1...
bool var3;

i think you understand! If store variables by name, with out index, only name - can change signature to global table, and change signature classes

this is a warning against the use of indexes, just in case, but you probably already knew!

[FDsagizi]

Hello, (Sorry for my english )

Good news!!!! It's first update. I find bug, and kill him

script reloader 1.2

ScriptReloader.h
http://www.everfall.com/paste/id.php?ck2u2rz09qiq

ScriptReloader.cpp
http://www.everfall.com/paste/id.php?jrk0n7u8rrsr

use:

asIScriptEngine *engine;
asIScriptModule *mod...;


CScriptReloader reloader;

reloader.Store( mod );

// After reload, need remove script engine - Why is this happening, I do not understand. Probably not an angel script recreates the classes are already there.  On this engine to rebuild.

engine->Release();

// create engine, module, load scripts...	
initialize();

reloader.Restore( mod );

[Andreas Jonsson]

If you know the typeId you can use the engine's AddRefScriptObject method to increase the reference. It will use the registered ADDREF behaviour for the type to call the AddRef().

Thanks for providing the update to the script reloader.

I plan to start incorporating your CScriptReloader into the SDK over the next few days. I'll let you know what adjustments I make to it in the process.

[FDsagizi]

Thanks


So, i update reloader( 1.3 ver ). Add can register user types - need for serialize reference objects...


ScriptReloader.h
http://www.everfall.com/paste/id.php?92o5q49sueoi

ScriptReloader.cpp
http://www.everfall.com/paste/id.php?ymyudighiqeu

[FDsagizi]

well as successes?

[Andreas Jonsson]

I didn't have success in finding the time to work on this yet.

[FDsagizi]

I didn't have success in finding the time to work on this yet.

But now it is difficult to find, if I need help! I'm always happy!