Creating a module system (dynamic loading) in C

Question

How would one go about loading compiled C code at run time, and then calling functions within it? Not like simply calling exec().

EDIT: The the program loading the module is in C.

Answer 1

In Linux/UNIX you can use the POSIX dlopen / dlsym / dlerror / dlclose functions to dynamically open shared libraries and access the symbols (including functions) they provide, see the man page for details.

Answer 2

There is a DIY approach. While the method (and possibility) of doing this varies from system to system, the general idea is to open a file, read the contents of the file into memory, make said memory executable, initialise a function pointer to a valid position within this memory, and there you are.

Of course this is assuming that it's just executable code - quite unlikely. The code probably requires data to be loaded into RAM too, and may require space for global/static variables. You could load this all yourself, but you'd need to go into the executable code and adjust all the memory references in it.

Most operating systems allow dynamic linking, which does all this for you.

Answer 3

Dynamic languages like Perl do this all the time. The Perl interpreter is written in C, and many Perl modules are partially written in C. When those modules are required, the compiled C components are dynamically loaded on the fly. As noted in another answer, the mechanism for storing those modules is DLLs on windows, and shared libraries (.so files) on UNIX. I believe the call for loading a shared library on UNIX is dlopen(). You can probably find pointers for how to accomplish this on UNIX by starting with the documentation for that call. For Windows, you would need to research DLLs and learn how to load them dynamically at runtime. [Or possibly go through the Cygwin UNIX emulation layer, which would probably allow you to use the same calls on Windows as you would on UNIX, but I wouldn't recommend that unless you're already using and compiling against Cygwin.]

Note that this is different from just linking against a shared library. If you know ahead of time exactly what code you will call, you can build against a shared library and the build will be "dynamically linked" to that library; without any special handling from you the routines from the library will be loaded into memory only when and if your program actually calls them. But you can't do that if you're planning to write something capable of loading any arbitrary object code, code that you can't identify now, at build time, but are instead waiting to be selected somehow at run time. For that you'll have to use dlopen() and its Windows cousins.

You might look at the way Perl or other dynamic languages do this to see some real examples. The Perl library responsible for this kind of dynamic loading is DynaLoader; it has both a Perl and a C component, I believe. I'm certain that other dynamic languages like Python have something similar which you might rather look at; and Parrot, the virtual machine for the unreleased Perl 6, surely has a mechanism for doing this as well (or will in the future).

For that matter, Java accomplishes this through its JNI (Java Native Interface) interface, so you could probably look at the source code for OpenJDK to see how Java accomplishes this on both UNIX and Windows.

Answer 4

Yeah, dlopen/uselib is what I was looking for, thanks.

Answer 5

dlopen is the way to go. Here are a few examples:

Loading a plugin with dlopen:

#include        <dlfcn.h>
...
int
main (const int argc, const char *argv[])
{

  char *plugin_name;
  char file_name[80];
  void *plugin;
  ...
  plugin = dlopen (file_name, RTLD_NOW);
  if (!plugin)
  {
     fatal ("Cannot load %s: %s", plugin_name, dlerror ());
  }

Compiling the above:

% cc  -ldl -o program program.o

Then, assuming this API for the plugins:

/* The functions we will find in the plugin */
typedef void (*init_f) ();
init_f init;
typedef int (*query_f) ();
query_f query;

Finding the address of init() in the plugin:

init = dlsym (plugin, "init");
result = dlerror ();
if (result)
{
   fatal ("Cannot find init in %s: %s", plugin_name, result);
}
init ();

With the other function, query(), which returns a value:

query = dlsym (plugin, "query");
result = dlerror ();
if (result)
{
    fatal ("Cannot find query in %s: %s", plugin_name, result);
}
printf ("Result of plugin %s is %d\n", plugin_name, query ());

You can retrieve the complete example on line.

Answer 6

See this question has been answered but thought others interested in this topic may appreciate a cross platform example from an old plugin based application. The example works on win32 or linux, and seaches for and calls a function called 'constructor' in the dynamically loaded .so or .dll specified in the file argument. The example is in c++ but the procedures should be the same for c.

//firstly the includes
#if !defined WIN32
   #include <dlfcn.h>
   #include <sys/types.h>
#else
   #include <windows.h>
#endif

//define the plugin's constructor function type named PConst
typedef tcnplugin* (*PConst)(tcnplugin*,tcnplugin*,HANDLE);

//loads a single specified tcnplugin,allmychildren[0] = null plugin
int tcnplugin::loadplugin(char *file) {
    tcnplugin *hpi;
#if defined WIN32               //Load library windows style
    HINSTANCE hplugin=LoadLibrary(file);
    if (hplugin != NULL) {
            PConst pinconstruct = (PConst)GetProcAddress(hplugin,"construct");
#else                                   //Load it nix style
    void * hplugin=dlopen(file,RTLD_NOW);
    if (hplugin != NULL) {
            PConst pinconstruct = (PConst)dlsym(hplugin,"construct");
#endif   
            if (pinconstruct != NULL) { //Try to call constructor function in dynamically loaded file, which returns a pointer to an instance of the plugin's class
                    hpi = pinconstruct(this, this, hstdout);
            } else {
                    piprintf("Cannot find constructor export in plugin!\n");
                    return 0;
            }
    } else {
            piprintf("Cannot open plugin!\n");
#if !defined WIN32
            perror(dlerror());
#endif
            return 0;
    }
    return addchild(hpi); //add pointer to plugin's class to our list of plugins
}

Might also be mentioned that if the module who's functions you wish to call is written in c++ your must declare the function with extern "C" such as:

extern "C" pcparport * construct(tcnplugin *tcnptr,tcnplugin *parent) {
    return new pcparport(tcnptr,parent,"PCPARPORT",0,1);
}

Answer 7

Also you can look at cpluff. It is a plugin management library on pure c.

Answer 8

Under Windows, this is how I do it:

• Generate code (in C because it's easy to find compilers, and library requirements are minimal)
• spawn a job to compile/link it into a DLL
• load it with LoadLibrary
• get function pointers with GetProcAddress

The generate/compile/link steps generally take less than a second.