embedding multiple lua instances in a multiple threaded program

Question

I have a program with 4 threads.

Within each thread, I do a luaL_newstate();

Each thread only has access to it's own lua instance.

Is there anything I need to worry about? [I.e. is there some hidden state that all lua instances share behind my back?]

Thanks!

Answer 1

No, that should work just fine. All interpreter state is self contained in each Lua instance. I would even say that is the preferred way to use Lua with multiple threads and/or processes.

If you find that you do need to communicate between Lua states eventually, then it is best to serialize the data and pass it using the C API. I recommend reading the "Exploring Lua for Concurrent Programming" whitepaper. It introduces a method of using multiple Lua processes with message passing for inter-process communication.

Answer 2

Creating a single lua_State per thread is a good solution to having multiple threads of Lua execution. However, those states are very separated. In particular, it is difficult to safely communicate between them since the Lua API is only thread-safe as long as each lua_State is accessed from a single thread at a time. (Well, unless lua_lock and lua_unlock are implemented as a suitable mutex, which they are not in the default builds of the lua core.)

If that level of isolation is not acceptable, then you need to investigate one of the several mechanisms for allowing Lua instances to play well with others in a threaded process.

My favorite choice is Lua Lanes which provides for multiple threads along with a mechanism for passing messages and sharing values between them in a thread-safe way. Values of most Lua types (including userdata with a little C side support from the library that uses it) can be safely and efficiently passed from one lane to another.

Other mechanisms exist, and a good starting point for most of them is at the Lua user's wiki page on MultiTaksing.

Answer 3

You're good as long as you don't try to pass values between Lua instances without converting them to C first. For example, it will be nearly impossible to share a mutable table among instances.

What you ask sounds easy to do but not necessarily any more useful than simply having multiple processes running, each with its own Lua and its own address space.