Jump to content
  • Advertisement
Sign in to follow this  
Sunsharior

I'm having doubt about my map loading method and its performance.

This topic is 2101 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I have implemented a map loading logic in this 2d sidescroller i'm making and it work. But recently, i've questionned if what i did was the right way to do it and if it won't screw me in the future.

 

One of the first thing i've coded in my game engin is the map loading logic.

The way i did is read a large text file from disk and store it's content into a string.
For reference the text file is formatted in this way:

 

Header~Tile1Param1;Tile1Param2;Tile1ParamN;~Tile2Param1;Tile2ParamN;~TileNParamN;~

 

as you can see, each cell is seperated by '~' and the cell parameters by ';'. To sort everything i tokenize the whole string into tile then for each tile i retokenize to extract the parameters and finaly feed this tile to the game engine.

 

But the problem is i do several thousands tokenize to get to that end. I've done some test and load time "seems" okay with a map size of half a million tiles (750 tile by 750 tile. Each tile is 16 pixel wide). If i could improve this, i wouldn't have to limit myself to that size.

 

Beside this performance issue, i'm wondering if this is the right approche for the future? What will happens when i will use something else than a txt file? Right now, i'm loading a map when the actor pass throught the door trigger. Should i preload all adjacent map before this time?

 

By the way, just for the curious, this is my tokenize function that i call.

//////////////////////////////////////////////////////////////////////////
//
void CMapManager::Tokenize(vector<string> &tokens, const string &str, char delim)
{
		stringstream ss(str); //convert string to stream
		string item;
		while(getline(ss, item, delim)) {
			tokens.push_back(item); //add token to vector
		}
}

I am very sorry if my english sound terrible and thank you for your time.

Share this post


Link to post
Share on other sites
Advertisement

Man its been a while since I've even thought about Tile engines

 

I cant really contribute to this, as I did ALL of my tile map engines like this:

 

3,3,3,3,3,3,3,3,3

3,0,0,0,1,2,0,0,3

3,8,6,0,1,1,1,0,3

3,0,0,0,0,0,0,0,3

3,3,3,3,3,3,3,3,3

 

and would use a matrix like that, and layers, bottom layer for tile graphic, middle layer for entities, top layer for collision.

 

I kind of like how you've parametrized the maps though, that's very interesting

Share this post


Link to post
Share on other sites

For starters, switch from text to binary :) It will be much faster, since you don't have to go through hundreds of characters, you just read a piece of data and there you are.

I remember seeing people just directly reading into the variables ( ie. foo.read(&mapWidth, sizeof(int)); ) and writing them as they are. This is certainly fast, but I'm not sure if this is good practice - issues could emerge when you copy a map from your computer to another and they happen to use different endianness. Still, I don't know how relevant this problem is, I hope somebody more experienced will answer :)

If it is of concern, you could use some fixed endianness in your files, and swap your byte order if while reading/writing if the running machine uses a different one.

 

In case it's of any help, I've written a simple library which does what I've described in my previous sentence. At start it checks what endianness the machine uses. Then you can create buffers, fill them with ( endian-correct ) data and flush those buffers to some file stream or whereever you wish. At reading you can load the whole file into a buffer and read from it, endianness will be handled. Although you have to tell each buffer what endianness they should use. ( i.e. if the buffer's endianness differs from the system's, it will flip the byte order ). Feel free to use it or just check the code :)
https://github.com/elementbound/binio

Share this post


Link to post
Share on other sites

As the previous user posted, you should really switch to a binary format. That way, you can create a structure for storing a tile information, then it's just a matter of creating an array of this and loading/saving it in one go. It will not only be faster, but will also save a lot of space.

Edited by Vortez

Share this post


Link to post
Share on other sites

Thank you all for the replies.

 

Looks like the binary way would be a needed solution to my problem.

Vortez, is this what they call serialization or am i mistaken ?

 

I currently have a class CTileInfo that contain:

int row
int col
int frontrow
int frontcol
int state
SDL_RendererFlip flip //(can be an int)
SDL_RendererFlip frontflip
string walksound
string script //(script need the same tokenization before they can be processed)
vector<CVector> *vertices

I never done that before, im unsure how to proceed. I'll give google a go.

Share this post


Link to post
Share on other sites

Text files can in many cases compress better than binary equivalents and so (using a ZIP asset file or zlib to compress your source assets) you can actually get faster loading time with text than binary.  To be sure, measure, measure, measure.

I think he means storing the entire map data in binary, like in copy the data instead of read the configure file, interpret it and create the data.
Not actually storing the config in a binary file.
 
edit: No, apparently he doesn't. Forget what I said.
Indeed, it may or may not give you an increased load speed.

Never witnessed a situation where it actually lost performance though.
 
 
 

Thank you all for the replies.
 
Looks like the binary way would be a needed solution to my problem.
Vortez, is this what they call serialization or am i mistaken ?
 
I currently have a class CTileInfo that contain:
int row
int col
int frontrow
int frontcol
int state
SDL_RendererFlip flip //(can be an int)
SDL_RendererFlip frontflip
string walksound
string script //(script need the same tokenization before they can be processed)
vector<CVector> *vertices
I never done that before, im unsure how to proceed. I'll give google a go.

 
I strongly advise against your use of strings.
 
You can use some sort of numeric reference instead (like an std::vector or even an [ int , string ] matrix).
 
This way, you can avoid loading and creating all the same strings multiple times, and also makes your life easier when you want to store the map in a binary format. Whenever you need the string, look it up in your matrix, and you're good to go.
 
Also, avoid the using namespace std statement.
 
I'll cook a little snippet for you in a moment.

Edited by dejaime

Share this post


Link to post
Share on other sites

 

Also, avoid the using namespace std statement.

 

 

Alright thank you for the tip, i'll be sure to remove the using namespace.

Share this post


Link to post
Share on other sites
I want you to take a look at a code I've done comparing two possible approaches to the problem.
As it is too big to post here, directly, I'll link you to a gist in github: here.
It is a relatively simple code in a single file, it should be easy to compile.

This is one output for a large set of "tiles" (results varies slightly from a run to another).
 
Starting test for a [7500, 7500] wide set.
Searching for a big enough memory block (450 mb )
Creating the map manually.
Saving the map to the files.
Will now Load the map from the files.

Total Time for each of the tasks -
    Manual Creation:        1.16 s
    Save to TEXT file:      48.12 s
    Save to BINARY file:    0.61 s
    Load from TEXT file:    22.27 s
    Load from BINARY file:  0.18 s
Total Time:                 72.34 s
I guess there's no mistake after all. I was expecting a big difference, but not so big.
If anyone find where I messed up, please let me know and I'll fix it.
 
A curiosity, the files had 1.4gb in plain text format or 450mb in direct binary format (same as the RAM block), for the 7500² set.

Disclaimer: I don't know what I'm doing nor what I'm talking about. Edited by dejaime

Share this post


Link to post
Share on other sites

Vortez, is this what they call serialization or am i mistaken ?

 

Humm, i never tried serialization, but it's similar. The thing with your example is that, there is too much information irrelevent to saving and loading your file. You don't need the x and y, you need to store it's width and height (to calculate the number of tiles), but since that information is only needed once, you dont need to include it in the structure either. Im not very good to explain things so let me give you an example:

struct CTile {
	int state; //? <-- not sure this should be there, but i dont know what it does...
        int texture_id;   // those complicate things a bit, you will have to make sure they always
	int walksound_id; // get assigned the same id
	int script_id;    
};

struct CTilesMap {
	int Width, Height;
	CTile *pTiles;
};

class CTilesManager
{
public:
	CTilesMap TilesMap;

	bool LoadMap(char *fname);	
	void SaveMap(char *fname);

	bool LoadScripts();
	bool LoadSounds();
...
}


bool CTilesManager::LoadMap(char *fname)
{
	FILE *f = fopen(fname, "rb");
	if(f){
		
		// Read the width and heigh of the map
		fread(TilesMap.Width,  1, sizeof(int), f);
		fread(TilesMap.Height, 1, sizeof(int), f);
	
		// Calculate the number of tiles from that
		int NumTiles = TilesMap.Width * TilesMap.Height;

		// Make sure the tiles aren't already allocated
		if(pTiles)
			return false;

		// Allocate memory to hold the tiles data
		pTiles = new CTile[TilesMap.Width][TilesMap.Height];

		// Load the rest of the file
		fread(pTiles, 1, sizeof(CTile) * NumTiles, f);

		fclose(f);

		return true;
	}

	return false;
}

Of course, this is very sketchy and i had a little trouble writting the code because the code is not mine, it's yours, but it should give you an idea about how to load binary files and structure your code. Also, scripts will need to be loaded using text file, and sound are generally binary, but should't be mixed with the map data imo, so that's why i provided 2 extra loader functions. The vertices could be calculated all at once when your done loading the map and know it's size.

 

If your a beginer it will probably seem overwhelming at first, but as i said, that's code is just to show you the general idea about how I would do it, and i wrote it in 5-10 mins so dont take it too seriously and use whatever method you like.

Edited by Vortez

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!