Back to AngelCode

Template containers angelscript addon library release!

AngelCode Affiliates

Started by Sami Vuorela October 14, 2014 09:46 AM

48 comments, last by Wracky 7 years, 6 months ago

Sami Vuorela

955

Author

December 04, 2014 02:12 PM

yo Andreas, I could use some help with the serializings.

I got everything except for handles serializing properly.

Containers containing primitives are serialized by simply copying the whole container to CSerializedValue userdata, like in the example for std::string.

Containers containing script objects are serialized like this:


my_serializer_usertype::store(CSerializedValue* val_root, void* ptr){
 actual_container_type* container = cast_dark_majicks(ptr);
 for(auto it = container->begin(); it != container->end(); it++){
  val_root->m_children.push_back(new CSerializedValue(val_root, "", "", const_cast<void*>(*it), container->astypeid_content));
 }
}
my_serializer_usertype::restore(CSerializedValue* val_root, void* ptr){
 actual_container_type* container = cast_dark_majicks(ptr);
 container->clear();//this releases script objects properly
 for(int i = 0; i < val_root->m_children.size(); i++){
  void* restored_object = container->engine->CreateScriptObject(container->objtype_content);
  val_root->m_children[i]->Restore(restored_object, container->astypeid_content);
  container->push_back(restored_object);
 }
}

seems to work fine

Containers containing handles are serialized like this:


my_serializer_usertype::store(CSerializedValue* val_root, void* ptr){
 actual_container_type* container = cast_dark_majicks(ptr);
 for(auto it = container->begin(); it != container->end(); it++){
  const void* ptr_to_object = *it;
  const void** ptr_to_ptr_to_object = &ptr_to_object;
  val_root->m_children.push_back(new CSerializedValue(val_root, "", "", const_cast<void*>((void*)ptr_to_ptr_to_object), container->astypeid_content));
 }
}
my_serializer_usertype::restore(CSerializedValue* val_root, void* ptr){
 actual_container_type* container = cast_dark_majicks(ptr);
 container->clear();//this releases script objects properly
 for(int i = 0; i < val_root->m_children.size(); i++){
  void* restored_object = nullptr;
  void** ptr_to_ptr_to_object = &restored_object;
  
  val_root->m_children[i]->Restore((void*)ptr_to_ptr_to_object, container->astypeid_content);
  container->push_back(restored_object);
 }
}

this seems to die horribly during CSerializer::Restore with this callstack:


  aatc_debug.exe![thunk]:AngelScript::asCScriptObject::`vcall'{8,{flat}}' }'() C++
> aatc_debug.exe!AngelScript::asCScriptEngine::CallObjectMethod(void * obj, AngelScript::asSSystemFunctionInterface * i, AngelScript::asCScriptFunction * s) Line 4211 C++
  aatc_debug.exe!AngelScript::asCScriptEngine::CallObjectMethod(void * obj, int func) Line 4163 C++
  aatc_debug.exe!AngelScript::asCScriptEngine::ReleaseScriptObject(void * obj, const AngelScript::asIObjectType * type) Line 5209 C++
  aatc_debug.exe!AngelScript::CSerializedValue::RestoreHandles() Line 475 C++
  aatc_debug.exe!AngelScript::CSerializedValue::RestoreHandles() Line 494 C++
  aatc_debug.exe!AngelScript::CSerializedValue::RestoreHandles() Line 494 C++
  aatc_debug.exe!AngelScript::CSerializer::Restore(AngelScript::asIScriptModule * mod) Line 125 C++
  aatc_debug.exe!AngelScript::main_contents() Line 46 C++
  aatc_debug.exe!main() Line 68 C++
  [External Code]

how is this different from the array addon example?

WitchLord

4,834

December 05, 2014 01:59 AM

I think you've overcomplicated it. The CUserType for serializing the CScriptArray add-on looks like this:


struct CArrayType : public CUserType
{
  void Store(CSerializedValue *val, void *ptr)
  {
    CScriptArray *arr = (CScriptArray*)ptr;
 
    for( unsigned int i = 0; i < arr->GetSize(); i++ )
      val->m_children.push_back(new CSerializedValue(val ,"", "", arr->At(i), arr->GetElementTypeId()));
  }
  void Restore(CSerializedValue *val, void *ptr)
  {
    CScriptArray *arr = (CScriptArray*)ptr;
    arr->Resize(asUINT(val->m_children.size()));
 
    for( size_t i = 0; i < val->m_children.size(); ++i )
      val->m_children[i]->Restore(arr->At(asUINT(i)), arr->GetElementTypeId());
  }
};

This code is capable of handling all types of arrays, e.g. array<int>, array<Object>, array<Object@>, etc.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

Sami Vuorela

955

Author

December 06, 2014 12:04 AM

There was a real problem:

Apparently the serializer does two passes: recursive Restore on all the objects and recursive RestoreHandles on all the handles.

Pointers passed to CSerializedValue::Restore must be valid until the RestoreHandles pass.

The scriptarray class allocates once and then restores things in place, so all its pointers to its stored objects will be legit.

Some AATC container's contents cannot be restored in place, for example with associative containers, where the state of the object decides its place in the container. Those container classes require that their contents are restored first, then added to the container once their state is ready for sorting or hashing.

In the previous code I was trying to restore them by passing pointers on the stack to CSerializedValue::Restore and adding them to the container during the restore callbacks, but using that approach the pointers to the restored objects were no longer valid during the RestoreHandles pass.

I solved this problem by restoring all the objects to an array during the restore callbacks, then adding the contents of that array to the actual container in a new pass, after the big call to Serializer::Restore. This approach might cause some problems if the state of the container is important for other objects during the Restore pass, but a bunch of semi-complex tests are working fine.

The arrays that store objects between the passes are currently stored in a global variable, because the CSerializer class has no userdata pointers available for storing things like these.

I did get all the things working mostly perfectly in the end. New experimental branch available here: https://github.com/Sami-Vuorela/aatc/archive/experimental-serializer-support.zip

New readme: https://github.com/Sami-Vuorela/aatc/blob/experimental-serializer-support/EXPERIMENTAL_README.txt

Syntax:


CSerializer* my_serializer = new CSerializer();
aatc_serializer_register(engine, my_serializer);

my_serializer->Store(my_module);


//Do_Things_Requiring_Script_Serialization()


my_serializer->Restore(my_module);
aatc_serializer_cleanup(engine, my_serializer);

delete my_serializer;

You can disable all serializer code in the config if its causing you trouble or if you want faster compilation.

The testing environment for this was quite simple so some problems miiiight have gotten through.

Crazy things like this don't seem to work "vector<vector<SomeObject@>@>".

Do post here and tell me if this thing actually works with whatever you're using it with.

WitchLord

4,834

December 06, 2014 02:28 PM

I'm glad you got it working.

To be honest I don't fully grasp how the serializer works myself. I didn't write it (FDsagizi did) and I've never bothered to analyze it in detail to get a full understanding of its inner workings.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

Sami Vuorela

955

Author

December 08, 2014 04:37 PM

There were some horrifying bugs related to the engine constructing aatc containers without an active script context. They are now fixed.

With those bugs fixed the serializer support can now handle structures of this complexity:



class sertest_Host{
  vector<sertest_World@> worlds;
};

class sertest_World{
  sertest_Host@ host;//test circular reference
  vector<sertest_WorldObject@> objects;

  string name;
};

class sertest_WorldObject{
  sertest_World@ world;//test circular reference

  string name;
  unordered_set<int> values;
  map<string,string> textvalues;
};


sertest_Host@ sertest_host;

Container of objects containing containers of objects containing containers and circular references.

The serializer support still can't handle any handles directly to containers.

Wracky

171

January 03, 2015 12:00 AM

Hi Sami,

Thanks for posting! Looks good!
Do you have any plans for adding support for opIndex to the map and unorderd_map ?

So that maps can be used like myMap[myKey] = myValue ?

I think that would be a nice addition.

Another thing:

While trying to add AACT to our project, I noticed that it didn't play nice with our custom String class.
Sure there is a define in the aatc_config that I can change, but our string doesn't have the same interface as the std::string and this gives some problems in the implementation of the aatc_script_Funcpointer::Set functions.

It was easily to fixed though, and kind of our problem for being boneheads and writing our own string class :P
I see why that's kind of a problem when you can't use asIScriptFunction* instead of names....

Anyway!
Keep up the good work!

http://www.piko3d.net

Sami Vuorela

955

Author

January 03, 2015 11:45 AM

Added opIndex for map and unordered map, does the same thing as map::Find.


T_value& opIndex(const T_key &in)

Its available in the experimental-serializer-support branch. ZIP.

WitchLord

4,834

January 03, 2015 04:08 PM

There should be a difference between Find and opIndex. When opIndex is called with a key that currently doesn't exist in the map, then key should be inserted with a default value, and the reference to that default value should be returned so it can be assigned.

This way the map can be populated with code like this:

map<string, int> m;

m['a'] = 1;

m['b'] = 2;

m['c'] = 3;

Find obviously shouldn't insert new values, since you want to be able to use Find to determine if a key exists or not.

Observe, the const overload for opIndex shouldn't insert the new key/value pair. Instead it should raise an exception or possibly return a reference to the default value.

By the way, I see your Find currently returns a reference to some default value when the key doesn't exist in the map. How would the caller know if the returned value is the default value? Wouldn't it be better if Find returns an iterator instead of a reference to the value directly, similar to how it is done in C++? That way the iterator can be checked for a valid entry or not, before the value is obtained from the iterator.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

Sami Vuorela

955

Author

January 03, 2015 05:25 PM

opIndex now works as described in post 28.

The registration code abuses existing Find and Insert, Find returns a default value if the value is not found.

Registration code:


engine->RegisterObjectMethod(n_container_T, "const T_value& get_opIndex(const T_key &in) const", asMETHODPR(T_container, Find, (void*), const void*), asCALL_THISCALL);
engine->RegisterObjectMethod(n_container_T, "void set_opIndex(const T_key&in,const T_value&in)", asMETHOD(T_container, Insert), asCALL_THISCALL);

Find has an overload where it takes a reference to a bool and writes success yes/no there.

Code showing opIndex_set opIndex_get and find with success:


map<int,int> mymap;


mymap[1] = 11;
mymap[3] = 33;
mymap[2] = 22;


Print("opindex[1] = "+mymap[1]);
Print("opindex[2] = "+mymap[2]);
Print("opindex[3] = "+mymap[3]);


bool success = false;
int findresult = mymap.find(2,success);
if(success){
  Print("findresult = "+findresult);
}


//Prints 11 22 33 and 22

Wracky

171

January 08, 2015 08:03 PM

Sweet!

I'm downloading it now. Thank you so much for the quick response.
I found Andreas' suggestion interesting as well.

Is it possible to make the map behave more like the stl one ?
Can it be extended so that find returns an iterator directly?

If that is possible, maybe functions like begin, end, rbegin, rend etc can also be supported.

In that case, find can return end if an object cannot be found, like stl does.

Thanks again.

http://www.piko3d.net

Template containers angelscript addon library release!

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Template containers angelscript addon library release!

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines