6 replies to this topic

[O-san]

Hello!

I'm developing a game which uses lua for scripting. I am at a point in development where I want to have save files/states and I am wondering what's the best solution. My scripts usually have a table where most of the variables reside, I use no coroutines in my scripts. Is it a good idea to somehow iterate through the lua stack and save the variables to file?

I have searched around a bit and found a library called pluto but I am not very confident that it will work with the latest lua build (5.2). Moreover it seems to have issues with 64bit systems.

Any thoughts on lua stack saving is greatly appreciated.

Edited by O-san, 13 June 2012 - 05:14 AM.

[Ashaman73]

Hello!

I'm developing a game which uses lua for scripting. I am at a point in development where I want to have save files/states and I am wondering what's the best solution. My scripts usually have a local table where most of the variables reside, I use no coroutines in my scripts. Is it a good idea to somehow iterate through the lua stack and save the variables to file?

I have searched around a bit and found a library called pluto but I am not very confident that it will work with the latest lua build (5.2). Moreover it seems to have issues with 64bit systems.

Any thoughts on lua stack saving is greatly appreciated.

Exists your game state really only in lua ? What about references (user data) to external (i.e. C++/C) data ?

I use a simple serialization=>deserialization=>post processing mechanism to save my game state (mostly lua) and reconnect it with engine related data (C++) after reloading it. An other consideration is, that you should take care of game related data which will change quite often during development. I.e. some item data or combat tables which changes frequently, saving this with the lua stack will render saved game useless, and you need saved games once you reach a late development stage.

An other approach, though more expensive, is to export/import your game state. This is similar to de-/serialization, but much more loosly (and error prone), think of a kind of simple script format which is needed to load your game and insert all entites etc. into the game (de-/serialization is more like an 1:1 mapping). This way frequent changes to the game or game related meta data can be managed really easily.

Edited by Ashaman73, 13 June 2012 - 04:59 AM.

[O-san]

Exists your game state really only in lua ? What about references (user data) to external (i.e. C++/C) data ?

I got some of my functionality scripted, not all of it by any means. My NPCs can have a behaviour script associated with them. These scripts can read variables from the "engine" and also set new values to these variables in some cases (through lua_register C funcitons). An NPC might have unique variables too, only available in lua. Such as if the player has talked or interacted in someway with the NPC.

Could you describe your serialization a bit, it sounds like something I am leaning towards.

I haven't written a game save system before so this is all quite new to me.

[Ashaman73]

The basic idea behind my serialization approach is, to assign an unique id to each object (=lua table) and use this unique id to save associations between objects. To archieve this, I've started to assign an unique id to each object at creation time:

-- global data
uid_counter = 0

uid_map = {}

createNewObject = function()
  uid_counter = uid_counter + 1
  local object = {}
  object.uid = uid_counter
  uid_map[uid_counter] = object

  return object
end

Then ,when you write your data to a file, exchange any references to tables by its uid, mark each uid with a special char ,i.e. '#', to distinquish it from other variables.

serializeObject = function( _obj)
  write("new_object={")

  for key,value in pairs(_obj) do
	-- is this a table ?
	if type(value)=="table" then
	  -- is this a object ?
	  if value.uid~=nil then
		-- save reference only
	   write("  "..key.."= [[#"..value.uid.."]],")
	 else
		-- just a table, write values as table, ignore nested tables/objects
	   ..
	 else
	   -- just normal attribute, save it according to it type
	   if type(value)=="string" then
		 write("  "..key.."= \""..value.."\",")
		...
	 end
  end
  -- close object
  write("}")
end

We only create objects and hold them in a map which prevents garbage collection, either delete them directly(bad) or clean them up (i.e. just before saving game). Ensure that you really have a complete object net, that is, you need some kind of root object.

cleanup  = function()
  local is_referenced = {}
  for uid,object in pairs(uid_map) do
	  for key,value in pairs(object) do
		if type(value)=="table" and value.uid~=nil then
		  is_referenced[value.uid] == true
		end
	  end
  end

  -- remove all objects, which are no longer references and is not the root object
  for uid,object in pairs(uid_map) do
	if root_uid~=uid and is_referenced[value.uid]==nil then
		-- remove
		uid_map[uid] = nil
	end
  end
end

Now we want to serialize the whole object data structure

serializeAll = function()
  -- clean up first
  cleanup()

  -- serialize them now
  for uid,object in pairs(uid_map) do
	serializeObject(object)
  end
end

You now should have a file like this one:

new_object =
{
  uid = 23,
  test = "Hello World",
  attr = 10,
}
new_object =
{
  uid = 25,
  my_ref = [[#23]],
  other_attr = "2928",
}

To deserizialize it (aka loading), just write a new_object method and execute the doFile lua function.

new_object = function(_input)
  -- save it to uid map
  uid_map[_input.uid] = _input
end

-- once you have read it, you need to reconnect the data
postprocess = function()
  local max_uid = 0
  for uid,object in pairs(uid_map) do
	  for key,value in pairs(object) do
		if type(value)=="string" and string.find(value,"#")~=nil then
		  -- extract uid (home work)
		  local ref_uid = ...
		  --reconnect with table
		  object[key]=uid_map[ref_uid]
		end
	  end
	  max_uid = math.max(uid,max_uid)
  end

  -- init counter
  uid_counter = max_uid
end

That is the basic idea, it is really simple, but need some work:
1. You can compress uids to keep them in a certain range.
2. You can use fixed uid ranges which represent meta-data (i.e. combat tables etc.) and will not be serialized (though the will be automatically reconnected).
3. You can expand this to use C/C++ engine objects (just assign uids to these objects too and write a seraziation mechanism to write them back too).
4. Add some error checking.

Edited by Ashaman73, 14 June 2012 - 01:06 AM.

[O-san]

Thanks for the comprehensive reply! I think I got most of it but one thing that puzzles me is how the resulting file new_object={...}new_object=... can call the new_object method. The contents looks to me like a table construction and not a function call to new_object(input).

[Ashaman73]

Oopps, sorry, these are chunks, not tables. You can define a chunk reader function anywhere and then use dofile on a file including some chunks which will call the chunk reader function with the given data as table. Example:

-- declare chunk reader function, could be any function anywhere
new_chunk = function(_input)
  print("New chunk")
  for k,v in pairs(_input) do
   print("== "..tostring(k).."="..tostring(v))
  end
end

-- a chunk could be build like : <chunk reader function>{ ...content...}
new_chunk
{
  first = "hello",
  second = 21,
}
new_chunk
{
  third = "world",
  forth = 42,
}

Edited by Ashaman73, 15 June 2012 - 04:05 AM.

[O-san]

Oops, now I see the omitted =, that makes it a chunk.. doh! The little things!

I must be missing something, that does not compute for me. My code looks like...

savefile.txt:

newObject=
{
  id = 1,
}
newObject=
{
  id = 2,
}

My script got these few lines:

newObject= function(input)
  print("New object")
  for k,v in pairs(input) do
   print("== "..tostring(k).."="..tostring(v))
  end
end

dofile("savefile.txt")

As soon as I dofile("savefile.txt") my newObject function type becomes a table type and I am missing why it should not. Thanks for taking your time!

Edited by O-san, 18 June 2012 - 03:14 AM.