Jump to content
  • Advertisement
Sign in to follow this  
theflamingskunk

[Solved] Lua "require" other files when loaded from memory

This topic is 1501 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello,

 

Im working on running Lua scripts from within my program written in C++. The way I am currently handling resources, I register the scripts to a memory buffer, and then use luaL_loadbuffer to load the scripts. However i'm a bit stuck in trying to figure out a way to replicate luas "require()" behavior for including other files when I am not actually using files.

 

Im thinking that I may have to register a c++ function with the lua state prior to loading scripts, that will pass a filename that I can then call luaL_loadbuffer on the representative buffer. However, then I believe that I will have to load all the scripts in the same Lua state which I would imagine is a pretty horrible idea, and calling this method multiple times on the same file will probably make things explode.

 

Note that I have an extremely primitive knowledge of Lua, so I little comprehension of how the Stack/States work so im not sure if what im thinking is even possible (and probably makes no sense/is a bunch of crap).

 

So basically i'm wondering if anybody has any experience with this type of problem (doesn't necessarily have to be Lua) or any insight on what direction to go in.

 

Thank You!

Share this post


Link to post
Share on other sites
Advertisement

Lua require() uses a table called package.loaders to store functions to load modules.

The function take a file name string as parameter, and return a function (like the result of luaL_loadbuffer ) if it succeeded and a error string if failed.

require() will call these functions one by one to find a match.

 

you can get more info from

http://pgl.yoyo.org/luai/i/package.loaders

 

it will be difficult to implement such loader functions if you don't know how to write c functions for Lua

you should have a look at chapter 26 of Programming in Lua

http://www.lua.org/pil/26.html

Share this post


Link to post
Share on other sites

Thanks for the Response Azure,

 

The Problem is that for my usage, individual lua files do not exist, but rather I am loading them into memory buffers from a generic container file (compressed assets folder). As such, it seems that require will always fail since the file its looking for doesnt exist.

 

packages.preload looks promising. I will take a closer look tommorow, but do you by chance know if its required to be loaded into the same lua_State?

 

Likewise I would like to avoid having to have generic lua files be compiled into separate C libraries.

 

Thanks!

Share this post


Link to post
Share on other sites

As such, it seems that require will always fail since the file its looking for doesnt exist.

 

When looking for a module, require calls each of these searchers in ascending order, with the module name (the argument given to require) as its sole parameter.

You make sure that your searcher is used first, by placing it first in the table (in ascending order)... Maybe there's also a way to remove all other searchers from the table?

 

Note: I found an old thread with an example of a loader implementation which might help you: http://www.gamedev.net/topic/517836-solved-lua--how-to-redirect-require-to-load-from-packed-file/

Edited by tonemgub

Share this post


Link to post
Share on other sites

It's probably not the most useful thing for you to look at (it's not meant to be tutorial code!), but my code to replace the package loader is here:
https://code.google.com/p/eight/source/browse/src/lua/lua.cpp
 
As long as your custom file system (packed asset system) can load a required lua file on demand and end up calling luaL_loadbuffer(L, bytes, numBytes, filename); then you should be ok.

I will take a closer look tommorow, but do you by chance know if its required to be loaded into the same lua_State?

I don't understand. When one lua_State requires you to load some code, why would you load it into a different lua_State?

Share this post


Link to post
Share on other sites

Some shorter example

int MyLoader(lua_State *L){
	//load the first parameter
	//which is the name of the file, or whatever string identifier for a resource that you passed in with require()
	string filename = lua_tostring(state,1);
	
	if( exist_in_container(filename)){
		buffer = get_data_from_container(filename);
		
		//load the buffer
		luaL_loadbuffer(L,buffer.data,buffer.size, filename.c_str());
		
		return 1;
	}else{
	
		lua_pushstring(L, "Error: file not found");
		return 1;
	}
}

// when you initialize your lua state...

//locate the package.loaders table
lua_getfield(state,LUA_GLOBALSINDEX,"package");
lua_getfield(state,-1,"loaders");
lua_remove(state,-2);


//for convienice, we just replace the first one with our loader
//package.loaders[1] = MyLoader
lua_pushinteger(state,1);
lua_pushcfunction(state,MyLoader);
lua_rawset(state,-3);
//balance the stack.
lua_pop(state,1);

Although I'll still recommend you to try to understand the Lua stack and interfacing with C.

Share this post


Link to post
Share on other sites

Or you can make it with lua itself.

Assuming you have native function that can load binary blob of data by its name:

loadfile = function(name, mode, ...)
    local file = load_file(name) -- native function returning data blob
    if file then
        -- :as_chunk() method will do "lua_loadbuffer" and set environment if asked
        return file:as_chunk(name, ...)
    end
    return nil, "cannot open '"..name.."': No such file or directory"
end

dofile = function(name, ...)
    local chunk, err = loadfile(name)
    if not chunk then error(err, 2) end
    return chunk(...)
end

local function vfs_loader(modname, file)
    local chunk, err = file:as_chunk(modname)
    if not chunk then error(err,2) end
    return chunk()
end

local function vfs_searcher(fname)
    local file = load_file(gsub(fname,"%.","/")..".lua")
    if file then
        return vfs_loader, file
    end
end

-- not just insert our searcher, but drop all the rest to disable
-- loading of external files
package.searchers = { package.loaders[1], vfs_searcher }

It's for lua 5.2, package searching slightly changed there.

Note that there's 2 searchers. First loader is lua's own, searching modules in cache. Second searcher will look for files from your asset manager.

Share this post


Link to post
Share on other sites

Thanks for all the Responses!

 

AzureBlade, that example code was exactly what I needed, and I was able to adapt it to get it to work for my needs.

 

It didnt even occur to me that you could add your own function to package.searchers. (5.2)

 

I will definitely need to familiarize myself with the lua stack more.

 

Thanks for the Help Everyone!

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!