C - Lua memory issues

Started by
12 comments, last by NicholasLopez 9 years, 8 months ago

Hello,

it has taken me awhile to realize, but I notice that there is a spike in memory any time a lua file is being accessed, and it never goes down. I have read around that after awhile there are garbage collections, but I have yet to encounter any. I switched between my games two levels so many times it went from 4mb usage to 1gb usage. A level is loaded like so:


int parse_level(const char* filename) { 
/* initialize Lua */
	L = lua_open();
/* load Lua base libraries */
	luaL_openlibs(L);
/* load the Lua file */
	if(luaL_loadfile( L, filename) || lua_pcall(L, 0, 1, 0)) {
		printf("ERROR: something is wrong with the file!\n");
	}
/* parse information */
	/* negative indexing means (lua_gettop(L) - n) + 1
		push table to -1 */
		lua_pushvalue(L, -1);
		/*	grab the lvl_width and lvl_height */
			lvl_width = get_number(L, "width");
			lvl_height = get_number(L, "height");
		/* grab the lvl_name */
			lua_getfield(L, -1, "properties");
			lvl_name = get_string(L, "name");
			lua_pop(L, 1);
	/* before we go to finish parsing, set up the level */
		level_back2 = malloc(sizeof(int) * lvl_width * lvl_height);
		level_back1 = malloc(sizeof(int) * lvl_width * lvl_height);
		level_fore = malloc(sizeof(int) * lvl_width * lvl_height);
	/* finish parsing by analyzing the tile layers and objects */
		lua_getfield(L, -1, "layers");
	/* acquire the tiles placements */
		/* BACKGROUND LAYER 2 */
		lua_rawgeti(L, -1, 1);
		lua_getfield(L, -1, "data"); 
		int i; 
		for( i = 0; i < lvl_width * lvl_height; i++ ) { 
			lua_rawgeti(L, -1, i + 1); 
			level_back2[i] = lua_tonumber(L, -1); 
			lua_pop(L, 1); 
		}
		lua_pop(L, 2);
		/* BACKGROUND LAYER 1 */
		lua_rawgeti(L, -1, 2);
		lua_getfield(L, -1, "data"); 
		for( i = 0; i < lvl_width * lvl_height; i++ ) { 
			lua_rawgeti(L, -1, i + 1); 
			level_back1[i] = lua_tonumber(L, -1); 
			lua_pop(L, 1); 
		}
		lua_pop(L, 2);
		/* FOREGROUND LAYER */
		lua_rawgeti(L, -1, 3);
		lua_getfield(L, -1, "data"); 
		for( i = 0; i < lvl_width * lvl_height; i++ ) { 
			lua_rawgeti(L, -1, i + 1); 
			level_fore[i] = lua_tonumber(L, -1); 
			lua_pop(L, 1); 
		}
		lua_pop(L, 2);
	/* acquire objects */
		lua_rawgeti(L, -1, 4);
		lua_getfield(L, -1, "objects"); 
		/* get the number of objects in the table */
		int t = lua_objlen( L, -1 );
		total_objects = t;
		/* allocate the number of objects */
		alloc_obj( t );
		/* process each object */
		for( i = 0; i < t; i++ ) { 
			lua_rawgeti(L, -1, i + 1); 
			/* print object information to console */
				if( print_info == true ) { 
					printf("--OBJECT--\n");
					print_string(L, "name");	print_string(L, "type");
					print_number(L, "x");		print_number(L, "y");
					print_number(L, "width");	print_number(L, "height");
					printf("----------\n");
				} 
			/* get object information */
				char *obj_name = get_string(L, "name");
				char *obj_type = get_string(L, "type");
				int obj_x = get_number(L, "x");
				int obj_y = get_number(L, "y");
				int obj_w = get_number(L, "width")/2;
				int obj_h = get_number(L, "height");
			/* create object with information */
				create_obj( i, obj_name, obj_type, obj_x, obj_y, obj_w, obj_h );
			lua_pop(L, 1); 
		} 
		lua_pop(L, 3);
/* close Lua */
	lua_close(L); 
	L = NULL;
} 

If you need to know what kind of lua file it reads, make a level in Tiled Map Editor, then go to file -> export as -> Lua. The levels are along the lines of:


return {
	properties = {
		...
	},
	tilesets = {
		...
	},
	layers = {
		...
	}
}

I also have a object script in my game, just one right now, for the player.


Player = {}

function Player:new()
	local object = {
		...
	}
	setmetatable(object, { __index = Player })
	return object
end

function obj_load()
	p = Player:new()
	p.ySpeed = p.max_vel
	return p
end

function proc()
	...
end

When the object is created in the parse_level() function, it opens lua with it's own lua state (for example, every object has an ID, which accesses it's stuff from a struct; so it would be obj[ID].L) and pcalls it, and then


lua_getglobal( obj[ID].L, "obj_load" );
lua_call( obj[ID].L, 0, 1 );
obj[ID].script_loaded = true;

and that's that. In my object function which runs in a loop, it operates the object of the ID it is given, and it sits around doing:


lua_getglobal( obj[ID].L, "proc" );
lua_call( obj[ID].L, 0, 1 );

So you may or may not notice any memory issues that I have, but I am still going to go into more detail. If those last two code lines I posted are not commented out, then the game increases in memory regularly; but if they are commented out, then the game increases in memory only when changing levels (because that's when a lua file is ran, but is closed). So I have pinpointed my memory issue to lua, I just do not know where or how to fix. Any ideas? Thanks

Here is the results.txt I got from Dr.Memory [attachment=22812:drmemory_results_7-26-2014.txt]

EDIT:

I saw this recently: http://stackoverflow.com/questions/9671793/limiting-a-lua-scripts-memory-usage however while the code for the function did compile, I never could figure out how to incorporate it and have the memory limited or that second part of code he posted. So if you tell me that this is what lua does, then is there a way to limit lua's memory usage? I would like to limit it.

Advertisement

Are you freeing those malloc'd stuff anywhere?


level_back2 = malloc(sizeof(int) * lvl_width * lvl_height);
level_back1 = malloc(sizeof(int) * lvl_width * lvl_height);
level_fore = malloc(sizeof(int) * lvl_width * lvl_height);

Derp

I forgot to mention, I have a change level:


int change_level( const char* filename ) { 
/* clear any current level assets */
	/* level information */
	lvl_name = "";
	lvl_width = 0;
	lvl_height = 0;
	/* camera coordinates */
	camera_x = 0;
	camera_y = 0;
	/* tiles */
	free(level_back2);
	free(level_back1);
	free(level_fore);
	/* objects */
	dealloc_obj();
	player_exists = false;
/* parse new level */
	parse_level( filename );
/* tell user level has been changed */
	printf( "level has been changed to: %s\n", lvl_name );
} 

It is always called first when changing levels. Those are the only three instances of malloc when it comes to lua and levels. Like I said. If the player script is running, the memory increases by the second; if the player script is not running, the memory increases only when the level is changed (because a lua script is being loaded, but is unloaded at the end).

Function `create_obj` creates many of the leaks. I don't see it in the code given.

From the call stack, it looks like create_obj contains another lua_open (a.k.a. luaL_newstate) call, which creates a Lua state that is not freed:


# 9 lua5.1.dll!luaL_newstate     +0xc      (0x1000ffed <lua5.1.dll+0xffed>)
#10 create_obj                    [D:\programming\test/object.c:82]


Also, the debug symbols seem to be missing from the traces. It doesn't seem right that luaL_newstate would be at the top of the stack, after luaM_realloc_.

not sure what you mean about luaL_newstate or luaM_realloc, but this is create_obj


/* This process will create our object */
int create_obj( int ID, char *name, char *type, int x, int y, int w, int h ) { 
/* information */
	obj[ID].name = name;
	obj[ID].type = type;
/* beginning offset */
	obj[ID].x = x;
	obj[ID].y = y;
/* dimensions */
	obj[ID].w = w;
	obj[ID].h = h;
/* other information */
	obj[ID].dir = 1;
/* check to see if object is player */
	if( strcmp( obj[ID].type, "playerobj" ) == 0 ) {
		if( player_exists == false ) {
			/* center the camera in on the player */
				camera_x = obj[ID].x - camera_w/2;
				camera_y = obj[ID].y - camera_h/2;
			player_exists = true;
		} else { /* warn the user that there are two instances of playerobj */
			printf("WARNING: there is more than one instance of playerobj,");
			printf("first one takes priority!\n");
		} 
	} 
/* initialize Lua */
	obj[ID].L = lua_open();
/* load Lua base libraries */
	luaL_openlibs(obj[ID].L);
/* register lua functions */
	lua_register(obj[ID].L, "dt", getdt_lua);
	lua_register(obj[ID].L, "control", control_lua);
	lua_register(obj[ID].L, "set_hitbox", sethitbox_lua);
	lua_register(obj[ID].L, "collision", collision_lua);
	lua_register(obj[ID].L, "get_offset", getoffset_lua);
	lua_register(obj[ID].L, "render", render_lua);
/* load script file based on type */
	if( luaL_loadfile( obj[ID].L, obj_script( ID )) || lua_pcall(obj[ID].L, 0, 0, 0) ) { 
		/*printf("ERROR: lua file for object type: [%s], could not be loaded!\n", obj[ID].type);*/
		obj[ID].script_loaded = false;
	} else { 
/* set object ID */
		lua_pushnumber( obj[ID].L, ID );
		lua_setglobal( obj[ID].L, "ID" );
/* run load function in script */
		lua_getglobal( obj[ID].L, "obj_load" );
		lua_call( obj[ID].L, 0, 1 );
		obj[ID].script_loaded = true;
	} 
/* set up sprite */
	if( strcmp( obj[ID].type, "playerobj" ) == 0 ) { 
		obj[ID].image = new_surface( 32, 32 );
	} 
} 

If I have all the object stuff commented out of the parse level. Then my games memory usage is about 3.8mb, and after changing levels a bunch it goes between 3.8 and 4.6mb. That's not too bad. But now you have helped me pinpoint the issue further. Thanks on that. Though if I closed lua in that process, then the game would crash, because another process reads the script. Any ideas?


obj[ID].L = lua_open();

^^^ This is the Lua object that is leaked.

It's a bit hard to explain how to proceed from here, I've never found a way to make Lua work the way I want myself... but what I can surely say is - try not to create too many Lua states, reuse one state for many objects instead. Instead of globally registering everything, you should create a table for each object and add everything as fields there.

So just use one Lua_State *L for everything? Is that what you are saying? Like define in my main.c Lua_State *L , and in my global.h extern Lua_State *L

Yes, that is what I'm saying. And that indeed is one of the ways to do it.

Okay. If I replace every instance of obj[ID].L with L, the game crashes, do I need to change some things around now? My object process itself is still commented out, so it is only crashing on the create_obj function I pasted earlier.

EDIT:

So after changing the levels repeatedly, I am starting to get a memory rise again (no objects in this instance; it takes quite a few levels changes for it go up 1 mb). I debugged it again with dr.memory, and changed levels til the program itself closed (crashed?). Here is the Dr.Memory file: [attachment=22821:drmemory_results_7-27-2014.txt]

EDIT 2:

After further looking, it must still be within the object creation. Where, i'm not sure at this point.

EDIT 3:

I narrowed it down to two possible things, either strcmp increases memory, or I need to change around an SDL_Surface.

So I think I solved the memory issue on just changing levels (maybe). However I am still confused with just using one lua_State for everything.

Just coming back to this, you did say use one lua state for everything. I'm not entirely sure how to do that, because I said in the last post, the game crashes if I just make everything L. Because there parse level function is like this:


int parse_level() { 
	L = lua_open()
	luaL_openlibs(L);
	if(luaL_loadfile( L, filename) || lua_pcall(L, 0, 1, 0)) {
		printf("ERROR: something is wrong with the file!\n");
	}
	...parse level...
	...parse objects...
		create_obj( ID, x, y );
	lua_close(L); 
} 

and an object is created like so:


int create_obj( int ID ) { 
	...set object info...
	obj[ID].L = lua_open();
	luaL_openlibs(L);
	...register lua functions...
	if( luaL_loadfile( L, script) || lua_pcall(L, 0, 0, 0) )
		printf("ERROR: lua file for object type: [%s], could not be loaded!\n", obj[ID].type);
} 

and from there, there is another function which loops and runs the level, and in it calls the object function


int object( int ID ) { 
	lua_getglobal( L, "proc" );
	lua_call( L, 0, 1 );
} 

so any ideas?

This topic is closed to new replies.

Advertisement