New at Python: GLPK not building properly / Python ImportError

Question

This is a beginner question, and a follow-up to this one, where I was pointed to GLPK.

I'm trying to get PyGLPK, a Python binding for the GNU Linear Programming Kit up and running, but no matter what I do, I can't seem to build and install GLPK so that Python finds it correctly. This comes after running ./configure, make, and sudo make install on the GLPK libraries, and following the instructions for PyGLPK.

Specifically, here is the error I get:

>>> import glpk  
Traceback (most recent call last):  
File "<stdin>", line 1, in <module>  
ImportError:    dlopen(/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-    packages/glpk.so, 2): Symbol not found: __glp_lpx_print_ips
   Referenced from:     /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/glpk.so
   Expected in: dynamic lookup

I assume that something isn't linking to somewhere else, and that it probably has something to do with paths and environment variables. However, here's where my abilities in the shell fail, and I'm at a loss over what to do next.

Edits

1. I'm able to run the GLPK Solver (glpsol) from the command line, so I know that it works, at least in theory.

2. At one point I tried using MacPorts to install a version of GLPK. I've since uninstalled this version, albeit using MacPorts.

3. Here's the result of using otool -L, which apparently is the OS X answer to ldd:

   /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/glpk.so:   
   /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.3.11)
   

Again, there is probably a simple answer to this, but I haven't had any luck with Google using the terminology I know.

Answer 1

The problem isn't really specific to Python. The glpk module is an extension module, a C shared library that Python loads. That C shared library has a dependency on the GLPK C library that it wraps; loading the extension module should load the GLPK C library so that the extension module can reference symbols from the GLPK C library, like __glp_lpx_print_ips. Apparently, something there is failing. It could be one of several things:

1. The glpk.so extension module may not be linked against the GLPK C library at all. That would mean it was built without the -l argument necessary to link against the GLPK library, which means the problem is in the build procedure for the glpk extension module. You can tell whether glpk.so depends on the C library by using the ldd tool. For example:

   % ldd /usr/lib/python2.6/lib-dynload/gdbm.so 
   linux-gate.so.1 =>  (0xb77bb000)
   libgdbm.so.3 => /usr/lib/libgdbm.so.3 (0xb7799000)
   libpthread.so.0 => /lib/i686/cmov/libpthread.so.0 (0xb7780000)
   libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7639000)
   /lib/ld-linux.so.2 (0xb77bc000)
   

   That shows my gdbm extension module is linked against the libgdbm shared library (as well as those other shared libraries.)

2. The glpk.so extension module may be linked against the GLPK C library correctly, but the dynamic linker may not be able to find the C library. Usually this produces a different warning (about not being able to find the C library) but this may not happen on MacOS. You can see this by again using the ldd tool: it'll list the dependency but not the actual file that is loaded (it'll say "not found" instead.)

   This is usually caused by not installing the C library, or by installing it somewhere the dynamic linker doesn't know to look. Unfortunately I don't know how MacOS X does its library lookup, and how you're supposed to modify the paths it scans. (On most UNIX systems you'd edit /etc/ld.so.conf or a file in /etc/ld.so.conf.d/, or run ldconfig -m.)

3. The glpk.so extension module may be linked correctly, and the dynamic linker may find the module correctly, but the GLPK extension module may not define this symbol after all. That could be a bug in GLPK, or it could be because the dynamic linker is finding a different GLPK C library (a different version, or one that is built differently), or it could be because the GLPK C library was compiled differently than the glpk.so extension module was. That's a little hard to diagnose, though, as it means digging into the actual C library symbols and the header files used during compilation.

I would guess, all things considered, the problem is #2. It's the most common problem, especially when installing in /usr/local, which is what ./configure without a --prefix argument usually does.

Answer 2

Solved the problem!

Thomas Wouter's first suggestion was closest to the mark: the glpk.so module wasn't linked to the C library at all. The reason was that make built the original GLPK libraries using gcc4.2 and specifying a 64-bit architecture , while Python's distutils module insisted on building PyGLPK's source code using gcc-4.0 with a 32-bit architecture.

Since I couldn't figure out how to add compiler flags to distutils, I just rebuilt the GLPK libraries forcing distutils compiler flags. This was what finally worked.

This seems to be an issue with OS X 10.6. ./configure scripts query the system architecture, which by default, I think, is x86_64, even though Python 2.6 plays best with 32-bit binaries.