Avoiding string copying in Lua

Question

Say I have a C program which wants to call a very simple Lua function with two strings (let's say two comma separated lists, returning true if the lists intersect at all, false if not).

The obvious way to do this is to push them onto the stack with lua_pushstring, which works fine, however, from the doc it looks like lua_pushstring but makes a copy of the string for Lua to work with.

That means that to cross over to the Lua function is going to require two string copies which I could avoid by rewriting the Lua function in C. Is there any way to arrange things so that the existing C strings could be reused on the Lua side for the sake of performance (or would the strcpy cost pale into insignificance anyway)?

From my investigation so far (my first couple of hours looking seriously at Lua), lite userdata seems like the sort of thing I want, but in the form of a string.

Answer 1

No. You cannot forbid Lua making a copy of the string when you call lua_pushstring().

Reason: Unless, the internal garbage collector would not be able to free unused memory (like your 2 input strings).

Even if you use the light user data functionality (which would be an overkill in this case), you would have to use lua_pushstring() later, when the Lua program asks for the string.

Answer 2

Hmm.. You could certainly write some C functions so that the work is being done on the C side, but as the other answer points out, you might get stuck pushing the string or sections of it in anyways.

Of note: Lua only stores strings once when they are brought in. i.e.: if I have 1 string containing "The quick brown fox jumps over the lazy dog" and I push it into Lua, and there are no other string objects that contain that string, it will make a new copy of it. If, on the other hand, I've already inserted it, you'll just get a pointer to that first string so equality checks are pretty cheap. Importing those strings can be a little expensive, I would guess, if this is done at a high frequency, however comparisons, again, are cheap.

I would try profiling what you're implementing and see if the performance is up to your expectations or not.

As the Lua Performance Tips document (which I recommend reading if you are thinking about maximizing performance with Lua), the two programming maxims related to optimizing are:

Rule #1: Don’t do it.

Rule #2: Don’t do it yet. (for experts only)