How do I make a nasty C++ program scriptable with Python and/or Lua?

Question

I'm confronted with the task of making a C++ app scriptable by users. The app has been in development for several years with no one wasting a thought on this before. It contains all sorts of niceties like multithreading, template wizardry and multiple inheritance. As the scripting language, Python is preferred, but Lua might be accepted if it is significantly easier to implement.

Question 1

From what I have learned so far, there are broadly speaking two ways to integrate Python/Lua with C++ : "extending" and "embedding".

In this case, it looks like I need both. The scripting language need access to objects, methods and data from the app but needs to be called by the app once the user has written the script - without restarting anything.

How is this usually done in the real world?

Question 2

There seems to be a bewildering array of of manual solutions and binding generators out there, all of them less than perfect.

• swig, pyste, Py++, ctypes, Boost.Python sip, PyCXX, pybindgen, robin, (Cython/Pyrex, Weave)
• CppLua, Diluculum, Luabind, Luabridge, LuaCpp, Luna/LunaWrapper, MLuaBind, MultiScript, OOLua, SLB, Sweet Lua, lux (this list from the lua wiki)
• CPB, tolua, tolua++, toLuaxx, luna and again swig

Most commments on these found on the web are a little out of date. For example, swig is said to be difficult in non-trivial cases and to generate incomprehensible code. OTOH, it has recently gone to v2.0.

Some of the above use pygccxml to let gcc analyze the C++ code and then genarate the binding. I find this idea appealing, as gcc probably understands the code better than i do :-). Does this work well?

Testing them all might easily cost me half of the time allocated for the whole project.

So, which ones do you recommend?

Answer 1

I wouldn't recommend swig as it's hard to get it to generate satisfactory binding in complex situations: been there, done that. I had to write a horrible script that "parsed" the original C++ code to generate some acceptable C++ code that swig could chew and generate acceptable bindings. So, in general: avoid ANY solution that relies on parsing the original C++ program.

Between Lua and Python: I have found Lua MUCH, MUCH better documented and more cleanly implemented. Python has a GIL (global lock), whereas with Lua, you can have an interpreter instance in each thread, for example. So, if you can choose, I'd recommend Lua. It is smaller language, easier to comprehend, easier to embed (much cleaner and smaller API, with excellent documentation). I have used luabind for a small project of mine and found it easy to use.

Answer 2

My experience may not be much, but I figure it's at least worth what you paid for it ;)

I've done some basic "hello world" python modules, and I couldn't really get into swig - it seemed like a lot of overhead for what I was doing. Of course it's also possible that it's just the right amount for your needs.

Answer 3

Regarding Question 1 - yes, you need to do both.

You would expose your applications scripting interface to an embedded interpreter, which then runs the user script in question.

The Python manuals section on embedding includes a section Extending Embedded Python and there are tutorials for similar things for Lua, see e.g. this article. Of course there are probably easier ways to be found when you have decided with what scripting language and binding mechanism you want to go.

Answer 4

Try Boost::Python, it has somewhat of a learning curve associated with it but it is the best tool for the job in my view, we have a huge real time system and developed the scripting library for the QA in Boost::Python.

Answer 5

Some of the above use pygccxml to let gcc analyze the C++ code and then genarate the binding. I find this idea appealing, as gcc probably understands the code better than i do :-). Does this work well?

A similar approach is used in the Lua Qt binding - lqt. It uses cpptoxml to generate an XML file containing all classes and methods with parameters, and generated binding C++ code according to that.

It works really well, the generator is able to bind almost all Qt classes and methods, including virtual methods, overloaded operators, selected template classes, signal/slot mechanism etc.

Although it was created specially for Qt, it has a "noqt" mode, in which it can serve as a generic binder, unfortunately I have no experience with it, so I cannot tell how much work it would be to bind your code.