Guides for implementing a foreign function interface

Question

Right now I'm working on a scripting language that doesn't yet have a FFI. I'd like to know what's the most convenient way to get it in, assuming that I'd like to write it like cool geeks do - I'd like to write the FFI in the scripting language itself.

The programming language I need to interface is C. So for basics I know that libdl.so is my best friend. Obviously it's not the only thing I'm going to need but the most important of them.

I only have slight ideas about what else do I need for it. I'd like to get similar behavior from the FFI as what python ctypes has.

What should I need to know in order to get this far? I know there's some serious magic with data structures I'll need to deal with. How do I manage it so that I could do most of that serious magic in the scripting language itself? I'd have use from such magic for much more than just the foreign function interface. For instance I might want to pass C-like binary data to files.

Answer 1

I think an appropiate answer requires a detailed essay.

Basically, there should be wrappers for the library loading and symbol searching facilities provided by the host OS. If the core datatypes of your language are internally represented with a single C data structure, then a requirement can be placed on the library developers that the parameters and the return type of the exported C functions should be objects of that data structure. This will make data exchange simpler to implement. If your language has some form of pattern expressions and first class functions, then the signature of C functions might be written in patterns and the library searched for a function matching an equivalent signature. Here is some pseudocode of a C function and its usage in script:

/* arith.dll */
/* A sample C function callable from the scripting language. */

#include "my_script.h" // Data structures used by the script interpreter.

My_Script_Object* add(My_Script_Object* num1, My_Script_Object* num2)
{
   int a = My_Script_Object_To_Int(num1);
   int b = My_Script_Object_To_Int(num2);
   return Int_To_My_Script_Object(a + b);
}

/* End of arith.dll */

// Script using the dll
clib = open_library("arith.dll");

// if it has first-class functions
add_func = clib.find([add int int]);
if (cfunc != null)
{
   sum = add_func(10, 20);
   print(sum);
}

// otherwise
print(clib.call("add", 10 20));

It is not possible to discuss more implementation details here. Note that we haven't said anything about garbage collection, etc.

The sources available at the following links may help you move further:

http://common-lisp.net/project/cffi/ http://www.nongnu.org/cinvoke/

Answer 2

Check out http://sourceware.org/libffi/

Remember the calling conventions are going to be different on different architectures, i.e. what order function variables are popped onto the stack. I don't know about writing it in your own scripting language, I do know that Java JNI uses libffi.