I haven't.
My situation goes into :
"when the objects to be serialized contain pointers to other objects, but when those pointers form a tree with no cycles and no joins. "
He talks about constructing objects recursively in the constructors etc.
Its all good but I still need to figure out which way to go with different file versions.
Save/Load System
Quote:Original post by Evil Steve
I assume Zipster means giving each struct an ID number, and then when you load the save file you'll load a number of structs with IDs. Your code then just needs to pull out the structs with particular IDs. Old code would see unrecognised IDs but ignore them, and new code can load the older IDs and the newer ones too.
Yes, however I would take it a step further and give each individual field a unique ID. Otherwise you end up maintaining a bunch of different structs for each version, with each one containing mostly identical members save for the one or two you added/removed each version.
I recommend using the Bridge pattern in your case, Black Knight. What you are after at the moment is having different loading schemas (per version) requiring latest output in terms of structs, lists etc. So, you can separate implementation from interface for those classes that have changed from version to version and instantiate implementation for those classes based on the version present in the file/memory. A problem still stands where you convert/type-cast interfaces/abstract classes returned after loading. An example would be something like this:
Essentially, it is the same as writing function per version of the class, however this one is more elegant and easier to organize.
Alternatively, you can try using boost' serialization library which allows just for the thing with different class versions. (See http://www.boost.org/doc/libs/release/libs/serialization)
class CMap{ // ...void loadMap(DARendererPtr renderer,const std::string& filename){// open file into pFileint iVersion = CURRENT_VERSION;fread(&iVersion, sizeof(int), 1, pFile);CMapImpl *pMapImpl = CMapImplFactory::GetSingleton().Create(iVersion);pMapImpl->loadMap(this, pFile);}};Essentially, it is the same as writing function per version of the class, however this one is more elegant and easier to organize.
Alternatively, you can try using boost' serialization library which allows just for the thing with different class versions. (See http://www.boost.org/doc/libs/release/libs/serialization)
Author
Quote:Original post by ZipsterQuote:Original post by Evil Steve
I assume Zipster means giving each struct an ID number, and then when you load the save file you'll load a number of structs with IDs. Your code then just needs to pull out the structs with particular IDs. Old code would see unrecognised IDs but ignore them, and new code can load the older IDs and the newer ones too.
Yes, however I would take it a step further and give each individual field a unique ID. Otherwise you end up maintaining a bunch of different structs for each version, with each one containing mostly identical members save for the one or two you added/removed each version.
How would I go about giving each field a unique ID?
struct MapData
{
std::map<int,Vector3> position;
std::map<int,float> waterLevel;
};
Can you give a small example which writes a small structure like that?
Because I don't understand how to use the IDs to read data.It seems that I have to write the ID to the file then the data and when reading I need to read the ID and check it agains something but I dont get what.
And how am I going to manage all the unique IDs.
Author
Well for now ill keep it simple and just write some structures to file :
These are enough to recreate the game objects.I'll read these from the file and then send the data to the constructors of the objects to create them again.
Anyway when I was playing with structures and file a few years ago writing a structure to a file was giving different file size then the size of the structure and when reading back everything was getting back wrong.
Someone told me to use pragma pack(1) or something like that so the structures are fit to 1 byte boundries.
Can someone explain if I reall need it and why?
struct TerrainDataChunk{ int size; int scale; int patchSize; boost::shared_array<float> heightData; boost::shared_array<unsigned char> alphaData;};struct WaterDataChunk{ int size; int scale; STMath::Vector3 position;};struct SkyPlaneDataChunk{ int size; int scale; STMath::Vector3 position; float atmosphereRadius; float tiling; float cloudMove; float currentColor[4];};struct MapDataChunk{ std::string name;};These are enough to recreate the game objects.I'll read these from the file and then send the data to the constructors of the objects to create them again.
Anyway when I was playing with structures and file a few years ago writing a structure to a file was giving different file size then the size of the structure and when reading back everything was getting back wrong.
Someone told me to use pragma pack(1) or something like that so the structures are fit to 1 byte boundries.
Can someone explain if I reall need it and why?
Only the save/load code needs to know about the IDs. The actual data and structures should be completely oblivious to it. Here's some Pseudo-code. There are a few classes I omit implementation for because it should be clear what they do:
From there you can easily add new IDs, and add/remove sections of the code that save or load certain fields. If a field exists then the switch statement catches it. If the ID isn't recognized then that field it might have been removed in a newer version and you can either ignore it or run special code that handles any backwards compatibility. And finally if one of your case blocks doesn't hit (because you're trying to load a new field that doesn't exist in the old data), you can easily add flags to check what was loaded and initialize any un-loaded data to appropriate default values.
struct MyStructType{ int myInt; float myFloat; OtherPODType myPOD; void Save(OStream* stream) const; void Load(IStream* stream);};enum { MYSTRUCT_INT_ID = 0x100, MYSTRUCT_FLOAT_ID, MYSTRUCT_OTHER_POD_ID, /* !!!NEW IDS ALWAYS GO HERE, NEVER CHANGE THE ONES ABOVE AFTER THEY'RE IN USE!!! */};void MyStructType::Save(OStream* stream) const{ stream->WriteIDAndData(MYSTRUCT_INT_ID, myInt); stream->WriteIDAndData(MYSTRUCT_FLOAT_ID, myFloat); stream->WriteIDAndData(MYSTRUCT_OTHER_POD_ID, myPOD);}/* Here's where the magic happens */void MyStructType::Load(IStream* stream){ int id = stream->ReadID(); while(!stream->IsAtEnd()) { switch(id) { case MYSTRUCT_INT_ID: stream->ReadData(myInt); break; case MYSTRUCT_FLOAT_ID: stream->ReadData(myFloat); break; case MYSTRUCT_OTHER_POD_ID: stream->ReadData(myPOD); break; default: Log("Unrecognized ID: %d", id); if(!SaveLoadCompatabilityLayer::HandleOldData(id, stream)) stream->SkipData(); break; } id = stream->ReadID(); }}From there you can easily add new IDs, and add/remove sections of the code that save or load certain fields. If a field exists then the switch statement catches it. If the ID isn't recognized then that field it might have been removed in a newer version and you can either ignore it or run special code that handles any backwards compatibility. And finally if one of your case blocks doesn't hit (because you're trying to load a new field that doesn't exist in the old data), you can easily add flags to check what was loaded and initialize any un-loaded data to appropriate default values.
Author
Hmm how should I nest structures with this?
Here is how my code looks like at the moment :
my other structures are simple they only contain ints floats etc.
But the terrain structure has arrays and another structure inside.
And I guess this method will be slower then just reading the whole struct with one read as its reading all the fields one by one from the file right?
Here is how my code looks like at the moment :
struct TerrainDataChunk{ struct TerrainInfo { int size; int scale; int patchSize; int alphaMapSize; }; TerrainInfo info; boost::shared_array<float> heightData; boost::shared_array<unsigned char> alphaMapData; void save(std::ofstream& file)const; void load(std::ifstream& file);};namespace TerrainDataChunkIDs{ enum types { };}my other structures are simple they only contain ints floats etc.
But the terrain structure has arrays and another structure inside.
And I guess this method will be slower then just reading the whole struct with one read as its reading all the fields one by one from the file right?
Author
I want to ask something about this code :
You are looping until the stream ends so are you opening the file again for each structure ?Because this would only read one structure and read lots of other ids from the file for other structures which are useless for this structure.
Im going for something like this :
But inside the terrainData.load I don't know how to handle the switch because I need a second loop there to read the fields and data.
If i stick everything into one structure it might work :
[Edited by - Black Knight on July 30, 2008 11:26:38 AM]
void MyStructType::Load(IStream* stream){ int id = stream->ReadID(); while(!stream->IsAtEnd()) { switch(id) { case MYSTRUCT_INT_ID: stream->ReadData(myInt); break; case MYSTRUCT_FLOAT_ID: stream->ReadData(myFloat); break; case MYSTRUCT_OTHER_POD_ID: stream->ReadData(myPOD); break; default: Log("Unrecognized ID: %d", id); if(!SaveLoadCompatabilityLayer::HandleOldData(id, stream)) stream->SkipData(); break; } id = stream->ReadID(); }}You are looping until the stream ends so are you opening the file again for each structure ?Because this would only read one structure and read lots of other ids from the file for other structures which are useless for this structure.
Im going for something like this :
void MapDataChunk::load(std::ifstream &file){ unsigned int currentID = 0; //read the first ID from the file file.read((char*)¤tID,4); while(!file.eof()) { switch(currentID) { //read the map name case MapDataChunkIDs::NAME_ID: name = STUtil::ReadStringFromStream(file); break; case MapDataChunkIDs::TERRAIN_ID: terrainData.load(file); break; case MapDataChunkIDs::SKYPLANE_ID: skyPlaneData.load(file); break; case MapDataChunkIDs::WATER_ID: waterData.load(file); break; default: break; } file.read((char*)¤tID,4); }}But inside the terrainData.load I don't know how to handle the switch because I need a second loop there to read the fields and data.
If i stick everything into one structure it might work :
struct MapDataChunkAll{ //map std::string map_name; //terrain int terrain_size; int terrain_scale; int terrain_patchSize; int terrain_alphaMapSize; boost::shared_array<float> terrain_heightData; boost::shared_array<unsigned char> terrain_alphaMapData; //water int water_size; int water_scale; STMath::Vector3 water_position; //sky int skyPlane_size; int skyPlane_scale; STMath::Vector3 skyPlane_position; float skyPlane_atmosphereRadius; float skyPlane_tiling; float skyPlane_cloudMove; float currentColor[4];}namespace MapDataChunkIDs{ enum { NAME_ID = 0x100, TERRAIN_SIZE_ID, TERRAIN_SCALE_ID, TERRAIN_PATCHSIZE_ID, TERRAIN_ALPHAMAPSIZE_ID, ... };}[Edited by - Black Knight on July 30, 2008 11:26:38 AM]
Quote:Original post by Black Knight
I want to ask something about this code :
*** Source Snippet Removed ***
You are looping until the stream ends so are you opening the file again for each structure ?Because this would only read one structure and read lots of other ids from the file for other structures which are useless for this structure.
Yeah I realized this after I had already gone to bed, but you don't really want to loop until the end of stream, only until the end of the block of data for the current object. Since this type of save/load system tends to be very hierarchical, it should be easy to place markers in the file that indicate the beginning and end of objects, for instance by using stream->WriteBeginObject() and stream->WriteEndObject() functions around the calls to write data. At the end of the day your file structure should resemble a tree, where each object can save/load a bunch of sub-objects, where leaves are your raw data.
So your terrain loading code would work the same as your map loading code. The only challenge is writing a serialization system that handles hierarchical storage, but that's not too difficult if you're familiar with trees.
Quote:Original post by Black Knight
I want to ask something about this code :
*** Source Snippet Removed ***
You are looping until the stream ends so are you opening the file again for each structure?
Of course not; there is nothing to close the stream, so it stays open.
Quote:Because this would only read one structure and read lots of other ids from the file for other structures which are useless for this structure.
Each time through the loop, it would read an id, and then check the id, and then read the corresponding structure. What's the problem?
Quote:Im going for something like this :
*** Source Snippet Removed ***
Don't do that. Do this:
void MapDataChunk::load(std::ifstream &file){ unsigned int currentID = 0; // Notice: Use the reading operation directly as the condition for the // loop. The standard library is specially designed to make this work, // and it's a standard idiom of the language. while (file.read((char*)¤tID, 4)) { switch(currentID) { // You almost certainly don't want to just have one data member // of each chunk type. Instead, have a function that creates // a new chunk, and append it to a container. case MapDataChunkIDs::NAME_ID: name = STUtil::ReadStringFromStream(file); break; case MapDataChunkIDs::TERRAIN_ID: terrainData.load(file); break; case MapDataChunkIDs::SKYPLANE_ID: skyPlaneData.load(file); break; case MapDataChunkIDs::WATER_ID: waterData.load(file); break; default: // You really should have better error handling :) break; } }}Quote:
But inside the terrainData.load I don't know how to handle the switch because I need a second loop there to read the fields and data.
Why would there be a switch inside the terrainData.load() function? But anyway, you don't have to do anything special to "handle" anything here. Just pass the stream as you're doing. The called function will start reading at the point where the main load function left off, read one chunk's worth of data, and return; then the main function will automatically start reading where the called function left off (this has to do with the stream being passed by reference, and with how streams work in general), and thus read the next ID. So within the called function, you just read each member of the chunk object, one at a time.
Your file is being assumed to contain a sequence of chunks prefaced by IDs. Is that not the case? If it is, are you sure you actually understand what a stream is?
If i stick everything into one structure it might work :
*** Source Snippet Removed ***
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
