# [Solved] Lua "require" other files when loaded from memory

This topic is 1233 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Hello,

Im working on running Lua scripts from within my program written in C++. The way I am currently handling resources, I register the scripts to a memory buffer, and then use luaL_loadbuffer to load the scripts. However i'm a bit stuck in trying to figure out a way to replicate luas "require()" behavior for including other files when I am not actually using files.

Im thinking that I may have to register a c++ function with the lua state prior to loading scripts, that will pass a filename that I can then call luaL_loadbuffer on the representative buffer. However, then I believe that I will have to load all the scripts in the same Lua state which I would imagine is a pretty horrible idea, and calling this method multiple times on the same file will probably make things explode.

Note that I have an extremely primitive knowledge of Lua, so I little comprehension of how the Stack/States work so im not sure if what im thinking is even possible (and probably makes no sense/is a bunch of crap).

So basically i'm wondering if anybody has any experience with this type of problem (doesn't necessarily have to be Lua) or any insight on what direction to go in.

Thank You!

##### Share on other sites

Lua require() uses a table called package.loaders to store functions to load modules.

The function take a file name string as parameter, and return a function (like the result of luaL_loadbuffer ) if it succeeded and a error string if failed.

require() will call these functions one by one to find a match.

it will be difficult to implement such loader functions if you don't know how to write c functions for Lua

you should have a look at chapter 26 of Programming in Lua

http://www.lua.org/pil/26.html

##### Share on other sites

Thanks for the Response Azure,

The Problem is that for my usage, individual lua files do not exist, but rather I am loading them into memory buffers from a generic container file (compressed assets folder). As such, it seems that require will always fail since the file its looking for doesnt exist.

packages.preload looks promising. I will take a closer look tommorow, but do you by chance know if its required to be loaded into the same lua_State?

Likewise I would like to avoid having to have generic lua files be compiled into separate C libraries.

Thanks!

##### Share on other sites

As such, it seems that require will always fail since the file its looking for doesnt exist.

When looking for a module, require calls each of these searchers in ascending order, with the module name (the argument given to require) as its sole parameter.

You make sure that your searcher is used first, by placing it first in the table (in ascending order)... Maybe there's also a way to remove all other searchers from the table?

Note: I found an old thread with an example of a loader implementation which might help you: http://www.gamedev.net/topic/517836-solved-lua--how-to-redirect-require-to-load-from-packed-file/

Edited by tonemgub

##### Share on other sites

It's probably not the most useful thing for you to look at (it's not meant to be tutorial code!), but my code to replace the package loader is here:

As long as your custom file system (packed asset system) can load a required lua file on demand and end up calling luaL_loadbuffer(L, bytes, numBytes, filename); then you should be ok.

I will take a closer look tommorow, but do you by chance know if its required to be loaded into the same lua_State?

I don't understand. When one lua_State requires you to load some code, why would you load it into a different lua_State?

##### Share on other sites

Some shorter example

int MyLoader(lua_State *L){
//load the first parameter
//which is the name of the file, or whatever string identifier for a resource that you passed in with require()
string filename = lua_tostring(state,1);

if( exist_in_container(filename)){
buffer = get_data_from_container(filename);

return 1;
}else{

return 1;
}
}

// when you initialize your lua state...

//locate the package.loaders table
lua_getfield(state,LUA_GLOBALSINDEX,"package");
lua_remove(state,-2);

//for convienice, we just replace the first one with our loader
lua_pushinteger(state,1);
lua_rawset(state,-3);
//balance the stack.
lua_pop(state,1);



Although I'll still recommend you to try to understand the Lua stack and interfacing with C.

##### Share on other sites

Or you can make it with lua itself.

Assuming you have native function that can load binary blob of data by its name:

loadfile = function(name, mode, ...)
local file = load_file(name) -- native function returning data blob
if file then
-- :as_chunk() method will do "lua_loadbuffer" and set environment if asked
return file:as_chunk(name, ...)
end
return nil, "cannot open '"..name.."': No such file or directory"
end

dofile = function(name, ...)
local chunk, err = loadfile(name)
if not chunk then error(err, 2) end
return chunk(...)
end

local function vfs_loader(modname, file)
local chunk, err = file:as_chunk(modname)
if not chunk then error(err,2) end
return chunk()
end

local function vfs_searcher(fname)
local file = load_file(gsub(fname,"%.","/")..".lua")
if file then
end
end

-- not just insert our searcher, but drop all the rest to disable
package.searchers = { package.loaders[1], vfs_searcher }



It's for lua 5.2, package searching slightly changed there.

Note that there's 2 searchers. First loader is lua's own, searching modules in cache. Second searcher will look for files from your asset manager.

##### Share on other sites

Thanks for all the Responses!

AzureBlade, that example code was exactly what I needed, and I was able to adapt it to get it to work for my needs.

It didnt even occur to me that you could add your own function to package.searchers. (5.2)

I will definitely need to familiarize myself with the lua stack more.

Thanks for the Help Everyone!

• 10
• 14
• 11
• 10
• 11