Jump to content

  • Log In with Google      Sign In   
  • Create Account

We're offering banner ads on our site from just $5!

1. Details HERE. 2. GDNet+ Subscriptions HERE. 3. Ad upload HERE.


C - Lua memory issues


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
13 replies to this topic

#1 DrNicholas   Members   -  Reputation: 130

Like
0Likes
Like

Posted 26 July 2014 - 11:06 AM

Hello,

 

it has taken me awhile to realize, but I notice that there is a spike in memory any time a lua file is being accessed, and it never goes down. I have read around that after awhile there are garbage collections, but I have yet to encounter any. I switched between my games two levels so many times it went from 4mb usage to 1gb usage. A level is loaded like so:

int parse_level(const char* filename) { 
/* initialize Lua */
	L = lua_open();
/* load Lua base libraries */
	luaL_openlibs(L);
/* load the Lua file */
	if(luaL_loadfile( L, filename) || lua_pcall(L, 0, 1, 0)) {
		printf("ERROR: something is wrong with the file!\n");
	}
/* parse information */
	/* negative indexing means (lua_gettop(L) - n) + 1
		push table to -1 */
		lua_pushvalue(L, -1);
		/*	grab the lvl_width and lvl_height */
			lvl_width = get_number(L, "width");
			lvl_height = get_number(L, "height");
		/* grab the lvl_name */
			lua_getfield(L, -1, "properties");
			lvl_name = get_string(L, "name");
			lua_pop(L, 1);
	/* before we go to finish parsing, set up the level */
		level_back2 = malloc(sizeof(int) * lvl_width * lvl_height);
		level_back1 = malloc(sizeof(int) * lvl_width * lvl_height);
		level_fore = malloc(sizeof(int) * lvl_width * lvl_height);
	/* finish parsing by analyzing the tile layers and objects */
		lua_getfield(L, -1, "layers");
	/* acquire the tiles placements */
		/* BACKGROUND LAYER 2 */
		lua_rawgeti(L, -1, 1);
		lua_getfield(L, -1, "data"); 
		int i; 
		for( i = 0; i < lvl_width * lvl_height; i++ ) { 
			lua_rawgeti(L, -1, i + 1); 
			level_back2[i] = lua_tonumber(L, -1); 
			lua_pop(L, 1); 
		}
		lua_pop(L, 2);
		/* BACKGROUND LAYER 1 */
		lua_rawgeti(L, -1, 2);
		lua_getfield(L, -1, "data"); 
		for( i = 0; i < lvl_width * lvl_height; i++ ) { 
			lua_rawgeti(L, -1, i + 1); 
			level_back1[i] = lua_tonumber(L, -1); 
			lua_pop(L, 1); 
		}
		lua_pop(L, 2);
		/* FOREGROUND LAYER */
		lua_rawgeti(L, -1, 3);
		lua_getfield(L, -1, "data"); 
		for( i = 0; i < lvl_width * lvl_height; i++ ) { 
			lua_rawgeti(L, -1, i + 1); 
			level_fore[i] = lua_tonumber(L, -1); 
			lua_pop(L, 1); 
		}
		lua_pop(L, 2);
	/* acquire objects */
		lua_rawgeti(L, -1, 4);
		lua_getfield(L, -1, "objects"); 
		/* get the number of objects in the table */
		int t = lua_objlen( L, -1 );
		total_objects = t;
		/* allocate the number of objects */
		alloc_obj( t );
		/* process each object */
		for( i = 0; i < t; i++ ) { 
			lua_rawgeti(L, -1, i + 1); 
			/* print object information to console */
				if( print_info == true ) { 
					printf("--OBJECT--\n");
					print_string(L, "name");	print_string(L, "type");
					print_number(L, "x");		print_number(L, "y");
					print_number(L, "width");	print_number(L, "height");
					printf("----------\n");
				} 
			/* get object information */
				char *obj_name = get_string(L, "name");
				char *obj_type = get_string(L, "type");
				int obj_x = get_number(L, "x");
				int obj_y = get_number(L, "y");
				int obj_w = get_number(L, "width")/2;
				int obj_h = get_number(L, "height");
			/* create object with information */
				create_obj( i, obj_name, obj_type, obj_x, obj_y, obj_w, obj_h );
			lua_pop(L, 1); 
		} 
		lua_pop(L, 3);
/* close Lua */
	lua_close(L); 
	L = NULL;
} 

If you need to know what kind of lua file it reads, make a level in Tiled Map Editor, then go to file -> export as -> Lua. The levels are along the lines of:

return {
	properties = {
		...
	},
	tilesets = {
		...
	},
	layers = {
		...
	}
}

I also have a object script in my game, just one right now, for the player.

Player = {}

function Player:new()
	local object = {
		...
	}
	setmetatable(object, { __index = Player })
	return object
end

function obj_load()
	p = Player:new()
	p.ySpeed = p.max_vel
	return p
end

function proc()
	...
end

When the object is created in the parse_level() function, it opens lua with it's own lua state (for example, every object has an ID, which accesses it's stuff from a struct; so it would be obj[ID].L) and pcalls it, and then

lua_getglobal( obj[ID].L, "obj_load" );
lua_call( obj[ID].L, 0, 1 );
obj[ID].script_loaded = true;

and that's that. In my object function which runs in a loop, it operates the object of the ID it is given, and it sits around doing:

lua_getglobal( obj[ID].L, "proc" );
lua_call( obj[ID].L, 0, 1 );

So you may or may not notice any memory issues that I have, but I am still going to go into more detail. If those last two code lines I posted are not commented out, then the game increases in memory regularly; but if they are commented out, then the game increases in memory only when changing levels (because that's when a lua file is ran, but is closed). So I have pinpointed my memory issue to lua, I just do not know where or how to fix. Any ideas? Thanks

 

Here is the results.txt I got from Dr.Memory Attached File  drmemory_results_7-26-2014.txt   68.6KB   30 downloads

 

EDIT:

I saw this recently: http://stackoverflow.com/questions/9671793/limiting-a-lua-scripts-memory-usage however while the code for the function did compile, I never could figure out how to incorporate it and have the memory limited or that second part of code he posted. So if you tell me that this is what lua does, then is there a way to limit lua's memory usage? I would like to limit it.


Edited by DrNicholas, 26 July 2014 - 07:30 PM.


Sponsor:

#2 Sponji   Members   -  Reputation: 1314

Like
0Likes
Like

Posted 27 July 2014 - 12:09 AM

Are you freeing those malloc'd stuff anywhere?

level_back2 = malloc(sizeof(int) * lvl_width * lvl_height);
level_back1 = malloc(sizeof(int) * lvl_width * lvl_height);
level_fore = malloc(sizeof(int) * lvl_width * lvl_height);

Derp

#3 DrNicholas   Members   -  Reputation: 130

Like
0Likes
Like

Posted 27 July 2014 - 12:21 AM

I forgot to mention,  I have a change level:

int change_level( const char* filename ) { 
/* clear any current level assets */
	/* level information */
	lvl_name = "";
	lvl_width = 0;
	lvl_height = 0;
	/* camera coordinates */
	camera_x = 0;
	camera_y = 0;
	/* tiles */
	free(level_back2);
	free(level_back1);
	free(level_fore);
	/* objects */
	dealloc_obj();
	player_exists = false;
/* parse new level */
	parse_level( filename );
/* tell user level has been changed */
	printf( "level has been changed to: %s\n", lvl_name );
} 

It is always called first when changing levels. Those are the only three instances of malloc when it comes to lua and levels. Like I said. If the player script is running, the memory increases by the second; if the player script is not running, the memory increases only when the level is changed (because a lua script is being loaded, but is unloaded at the end).



#4 snake5   Members   -  Reputation: 247

Like
0Likes
Like

Posted 27 July 2014 - 03:51 AM

Function `create_obj` creates many of the leaks. I don't see it in the code given.
 
From the call stack, it looks like create_obj contains another lua_open (a.k.a. luaL_newstate) call, which creates a Lua state that is not freed:

# 9 lua5.1.dll!luaL_newstate     +0xc      (0x1000ffed <lua5.1.dll+0xffed>)
#10 create_obj                    [D:\programming\test/object.c:82]

 
Also, the debug symbols seem to be missing from the traces. It doesn't seem right that luaL_newstate would be at the top of the stack, after luaM_realloc_.



#5 DrNicholas   Members   -  Reputation: 130

Like
0Likes
Like

Posted 27 July 2014 - 09:08 AM

not sure what you mean about luaL_newstate or luaM_realloc, but this is create_obj

/* This process will create our object */
int create_obj( int ID, char *name, char *type, int x, int y, int w, int h ) { 
/* information */
	obj[ID].name = name;
	obj[ID].type = type;
/* beginning offset */
	obj[ID].x = x;
	obj[ID].y = y;
/* dimensions */
	obj[ID].w = w;
	obj[ID].h = h;
/* other information */
	obj[ID].dir = 1;
/* check to see if object is player */
	if( strcmp( obj[ID].type, "playerobj" ) == 0 ) {
		if( player_exists == false ) {
			/* center the camera in on the player */
				camera_x = obj[ID].x - camera_w/2;
				camera_y = obj[ID].y - camera_h/2;
			player_exists = true;
		} else { /* warn the user that there are two instances of playerobj */
			printf("WARNING: there is more than one instance of playerobj,");
			printf("first one takes priority!\n");
		} 
	} 
/* initialize Lua */
	obj[ID].L = lua_open();
/* load Lua base libraries */
	luaL_openlibs(obj[ID].L);
/* register lua functions */
	lua_register(obj[ID].L, "dt", getdt_lua);
	lua_register(obj[ID].L, "control", control_lua);
	lua_register(obj[ID].L, "set_hitbox", sethitbox_lua);
	lua_register(obj[ID].L, "collision", collision_lua);
	lua_register(obj[ID].L, "get_offset", getoffset_lua);
	lua_register(obj[ID].L, "render", render_lua);
/* load script file based on type */
	if( luaL_loadfile( obj[ID].L, obj_script( ID )) || lua_pcall(obj[ID].L, 0, 0, 0) ) { 
		/*printf("ERROR: lua file for object type: [%s], could not be loaded!\n", obj[ID].type);*/
		obj[ID].script_loaded = false;
	} else { 
/* set object ID */
		lua_pushnumber( obj[ID].L, ID );
		lua_setglobal( obj[ID].L, "ID" );
/* run load function in script */
		lua_getglobal( obj[ID].L, "obj_load" );
		lua_call( obj[ID].L, 0, 1 );
		obj[ID].script_loaded = true;
	} 
/* set up sprite */
	if( strcmp( obj[ID].type, "playerobj" ) == 0 ) { 
		obj[ID].image = new_surface( 32, 32 );
	} 
} 

If I have all the object stuff commented out of the parse level. Then my games memory usage is about 3.8mb, and after changing levels a bunch it goes between 3.8 and 4.6mb. That's not too bad. But now you have helped me pinpoint the issue further. Thanks on that. Though if I closed lua in that process, then the game would crash, because another process reads the script. Any ideas?



#6 snake5   Members   -  Reputation: 247

Like
0Likes
Like

Posted 27 July 2014 - 10:49 AM

obj[ID].L = lua_open();

^^^ This is the Lua object that is leaked.

 

It's a bit hard to explain how to proceed from here, I've never found a way to make Lua work the way I want myself... but what I can surely say is - try not to create too many Lua states, reuse one state for many objects instead. Instead of globally registering everything, you should create a table for each object and add everything as fields there.



#7 DrNicholas   Members   -  Reputation: 130

Like
0Likes
Like

Posted 27 July 2014 - 12:19 PM

So just use one Lua_State *L for everything? Is that what you are saying? Like define in my main.c  Lua_State *L , and in my global.h extern Lua_State *L


Edited by DrNicholas, 27 July 2014 - 02:03 PM.


#8 snake5   Members   -  Reputation: 247

Like
0Likes
Like

Posted 27 July 2014 - 03:07 PM

Yes, that is what I'm saying. And that indeed is one of the ways to do it.



#9 DrNicholas   Members   -  Reputation: 130

Like
0Likes
Like

Posted 27 July 2014 - 03:37 PM

Okay. If I replace every instance of obj[ID].L with L, the game crashes, do I need to change some things around now? My object process itself is still commented out, so it is only crashing on the create_obj function I pasted earlier.

 

EDIT:

So after changing the levels repeatedly, I am starting to get a memory rise again (no objects in this instance; it takes quite a few levels changes for it go up 1 mb). I debugged it again with dr.memory, and changed levels til the program itself closed (crashed?). Here is the Dr.Memory file: Attached File  drmemory_results_7-27-2014.txt   20.39KB   10 downloads

 

EDIT 2:

After further looking, it must still be within the object creation. Where, i'm not sure at this point.

 

EDIT 3:

I narrowed it down to two possible things, either strcmp increases memory, or I need to change around an SDL_Surface.

 

So I think I solved the memory issue on just changing levels (maybe). However I am still confused with just using one lua_State for everything.


Edited by DrNicholas, 27 July 2014 - 08:17 PM.


#10 DrNicholas   Members   -  Reputation: 130

Like
0Likes
Like

Posted 29 July 2014 - 12:36 PM

Just coming back to this, you did say use one lua state for everything. I'm not entirely sure how to do that, because I said in the last post, the game crashes if I just make everything L. Because there parse level function is like this:

int parse_level() { 
	L = lua_open()
	luaL_openlibs(L);
	if(luaL_loadfile( L, filename) || lua_pcall(L, 0, 1, 0)) {
		printf("ERROR: something is wrong with the file!\n");
	}
	...parse level...
	...parse objects...
		create_obj( ID, x, y );
	lua_close(L); 
} 

and an object is created like so:

int create_obj( int ID ) { 
	...set object info...
	obj[ID].L = lua_open();
	luaL_openlibs(L);
	...register lua functions...
	if( luaL_loadfile( L, script) || lua_pcall(L, 0, 0, 0) )
		printf("ERROR: lua file for object type: [%s], could not be loaded!\n", obj[ID].type);
} 

and from there, there is another function which loops and runs the level, and in it calls the object function

int object( int ID ) { 
	lua_getglobal( L, "proc" );
	lua_call( L, 0, 1 );
} 

so any ideas?



#11 snake5   Members   -  Reputation: 247

Like
0Likes
Like

Posted 30 July 2014 - 01:11 AM

the game crashes if I just make everything L

 
There could be many reasons behind the crash. It's probably one of the last unaddressable access errors in the log. I see invalid printf usage (which probably comes from `lvl_name = get_string(L, "name");`, which could've been freed somewhere) and issues with SDL (though I don't see any SDL code there).
 

I'm not entirely sure how to do that

 
Use tables. Store object-specific data in tables, which could then be stored in a global [object_id -> object_table] table. I'm rusty with my Lua API skills as well as I don't quite understand your code so I can't give the exact code for this. But I can say that it involves lua_getfield, lua_setfield, lua_getglobal and lua_setglobal, most of which were already used in your code.


Edited by snake5, 30 July 2014 - 01:12 AM.


#12 DrNicholas   Members   -  Reputation: 130

Like
0Likes
Like

Posted 30 July 2014 - 09:51 AM

get_string is like this:

char *get_string(lua_State *L, const char *key) { 
	/*	assumes there's a table at -1
		this puts the table[key] value at -1 on the stack */
	lua_getfield(L, -1, key); /* -- (top) table[key] table ... */
	/* get the value at -1 as a string */
	const char* result = lua_tostring(L, -1);    
	/* remove it from the stack */
	lua_pop(L, 1);	
	/* return the value */
	return (char *)result;
} 

and I know it's not an SDL issue, and there are no memory leakages right now. However that is because I commented out the code inside the object functions that pertain to lua. Don't understand what you mean with tables. Isn't there a way to copy the script into a c side variable, and than just have that read from/cleared when it's done?



#13 snake5   Members   -  Reputation: 247

Like
0Likes
Like

Posted 31 July 2014 - 12:51 AM

The problem is not in get_string, it's in the global variable you assign a string with unknown lifetime to. As soon as that string is freed (lua_close at the latest, possibly before that), the variable becomes invalid, it's what people call a dangling pointer.

 

 

 

Don't understand what you mean with tables.

 

Tables. Tables. Tables. Tables. (click on each link to learn something). At this point it seems like there's no place for not understanding things anymore - if you still don't understand, I suggest practicing with something easier first. If you want to solve the problem, I expect you to know something about Lua tables and be able to look up the things you don't know yourself.

 

 

 

Isn't there a way to copy the script into a c side variable, and than just have that read from/cleared when it's done?

 

Sure, there is. Compiling the script every time you want to run it will probably be slower than you expect, though.



#14 DrNicholas   Members   -  Reputation: 130

Like
0Likes
Like

Posted 31 July 2014 - 08:49 AM

-- the object
Player = {}

-- ID is set upon object load
ID = nil

-- define the object
function Player:new()
	-- define our parameters here
	local object = {
		x = 0,
		y = 0,
		w = 16,
		h = 32,
		dir = 1,
		xSpeed = 0,
		ySpeed = 0,
		gravity = 30,
		max_vel = 160,
		onground = true,
		jumping = false,
		falling = false,
		jump_held = false,
	}
	setmetatable(object, { __index = Player })
	return object
end

I suppose I can change this first part around again. I understand how to do things, but not the vocabulary they are given. So do I just call the tables creation, and then stuff all of it's contents into a variable in C, and then run the script in my loop process with that table being placed in?






Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS