Hello,

Im working on running Lua scripts from within my program written in C++. The way I am currently handling resources, I register the scripts to a memory buffer, and then use luaL_loadbuffer to load the scripts. However i'm a bit stuck in trying to figure out a way to replicate luas "require()" behavior for including other files when I am not actually using files.

Im thinking that I may have to register a c++ function with the lua state prior to loading scripts, that will pass a filename that I can then call luaL_loadbuffer on the representative buffer. However, then I believe that I will have to load all the scripts in the same Lua state which I would imagine is a pretty horrible idea, and calling this method multiple times on the same file will probably make things explode.

Note that I have an extremely primitive knowledge of Lua, so I little comprehension of how the Stack/States work so im not sure if what im thinking is even possible (and probably makes no sense/is a bunch of crap).

So basically i'm wondering if anybody has any experience with this type of problem (doesn't necessarily have to be Lua) or any insight on what direction to go in.

Thank You!

Lua require() uses a table called package.loaders to store functions to load modules.

The function take a file name string as parameter, and return a function (like the result of luaL_loadbuffer ) if it succeeded and a error string if failed.

require() will call these functions one by one to find a match.

you can get more info from

http://pgl.yoyo.org/luai/i/package.loaders

it will be difficult to implement such loader functions if you don't know how to write c functions for Lua

you should have a look at chapter 26 of Programming in Lua

http://www.lua.org/pil/26.html

Thanks for the Response Azure,

The Problem is that for my usage, individual lua files do not exist, but rather I am loading them into memory buffers from a generic container file (compressed assets folder). As such, it seems that require will always fail since the file its looking for doesnt exist.

packages.preload looks promising. I will take a closer look tommorow, but do you by chance know if its required to be loaded into the same lua_State?

Likewise I would like to avoid having to have generic lua files be compiled into separate C libraries.

Thanks!

As such, it seems that require will always fail since the file its looking for doesnt exist.

When looking for a module, require calls each of these searchers in ascending order, with the module name (the argument given to require) as its sole parameter.

You make sure that your searcher is used first, by placing it first in the table (in ascending order)... Maybe there's also a way to remove all other searchers from the table?

Note: I found an old thread with an example of a loader implementation which might help you: http://www.gamedev.net/topic/517836-solved-lua--how-to-redirect-require-to-load-from-packed-file/

Some shorter example

int MyLoader(lua_State *L){
	//load the first parameter
	//which is the name of the file, or whatever string identifier for a resource that you passed in with require()
	string filename = lua_tostring(state,1);
	
	if( exist_in_container(filename)){
		buffer = get_data_from_container(filename);
		
		//load the buffer
		luaL_loadbuffer(L,buffer.data,buffer.size, filename.c_str());
		
		return 1;
	}else{
	
		lua_pushstring(L, "Error: file not found");
		return 1;
	}
}

// when you initialize your lua state...

//locate the package.loaders table
lua_getfield(state,LUA_GLOBALSINDEX,"package");
lua_getfield(state,-1,"loaders");
lua_remove(state,-2);


//for convienice, we just replace the first one with our loader
//package.loaders[1] = MyLoader
lua_pushinteger(state,1);
lua_pushcfunction(state,MyLoader);
lua_rawset(state,-3);
//balance the stack.
lua_pop(state,1);

Although I'll still recommend you to try to understand the Lua stack and interfacing with C.

Or you can make it with lua itself.

Assuming you have native function that can load binary blob of data by its name:

loadfile = function(name, mode, ...)
    local file = load_file(name) -- native function returning data blob
    if file then
        -- :as_chunk() method will do "lua_loadbuffer" and set environment if asked
        return file:as_chunk(name, ...)
    end
    return nil, "cannot open '"..name.."': No such file or directory"
end

dofile = function(name, ...)
    local chunk, err = loadfile(name)
    if not chunk then error(err, 2) end
    return chunk(...)
end

local function vfs_loader(modname, file)
    local chunk, err = file:as_chunk(modname)
    if not chunk then error(err,2) end
    return chunk()
end

local function vfs_searcher(fname)
    local file = load_file(gsub(fname,"%.","/")..".lua")
    if file then
        return vfs_loader, file
    end
end

-- not just insert our searcher, but drop all the rest to disable
-- loading of external files
package.searchers = { package.loaders[1], vfs_searcher }

It's for lua 5.2, package searching slightly changed there.

Note that there's 2 searchers. First loader is lua's own, searching modules in cache. Second searcher will look for files from your asset manager.

Thanks for all the Responses!

AzureBlade, that example code was exactly what I needed, and I was able to adapt it to get it to work for my needs.

It didnt even occur to me that you could add your own function to package.searchers. (5.2)

I will definitely need to familiarize myself with the lua stack more.

Thanks for the Help Everyone!