Mac 10.6 Universal Binary scipy: cephes/specfun "_aswfa_" symbol not found

Question

Hi folks,

I can't get scipy to function in 32 bit mode when compiled as a i386/x86_64 universal binary, and executed on my 64 bit 10.6.2 MacPro1,1.

My python setup

With the help of this answer, I built a 32/64 bit intel universal binary of python 2.6.4 with the intention of using the arch command to select between the architectures. (I managed to make some universal binaries of a few libraries I wanted using lipo.) That all works. I then installed scipy according to the instructions on hyperjeff's article, only with more up-to-date numpy (1.4.0) and skipping the bit about moving numpy aside briefly during the installation of scipy.

Now, everything except scipy seems to be working as far as I can tell, and I can indeed select between 32 and 64 bit mode using arch -i386 python and arch -x86_64 python.

The error

Scipy complains in 32 bit mode:

$ arch -x86_64 python -c "import scipy.interpolate; print 'success'"
success
$ arch -i386 python -c "import scipy.interpolate; print 'success'"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/interpolate/__init__.py", line 7, in <module>
    from interpolate import *
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/interpolate/interpolate.py", line 13, in <module>
    import scipy.special as spec
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/__init__.py", line 8, in <module>
    from basic import *
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/basic.py", line 8, in <module>
    from _cephes import *
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/_cephes.so, 2): Symbol not found: _aswfa_
  Referenced from: /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/_cephes.so
  Expected in: flat namespace
 in /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/_cephes.so

Attempt at tracking down the problem

It looks like scipy.interpolate imports something called _cephes, which looks for a symbol called _aswfa_ but can't find it in 32 bit mode. Browsing through scipy's source, I find an ASWFA subroutine in specfun.f. The only scipy product file with a similar name is specfun.so, but both that and _cephes.so appear to be universal binaries:

$ cd /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/
$ file _cephes.so specfun.so 
_cephes.so: Mach-O universal binary with 2 architectures
_cephes.so (for architecture i386): Mach-O bundle i386
_cephes.so (for architecture x86_64): Mach-O 64-bit bundle x86_64
specfun.so: Mach-O universal binary with 2 architectures
specfun.so (for architecture i386): Mach-O bundle i386
specfun.so (for architecture x86_64): Mach-O 64-bit bundle x86_64

Ho hum. I'm stuck. Things I may try but haven't figured out how yet include compiling specfun.so myself manually, somehow.

I would imagine that scipy isn't broken for all 32 bit machines, so I guess something is wrong with the way I've installed it, but I can't figure out what.

I don't really expect a full answer given my fairly unique (?) setup, but if anyone has any clues that might point me in the right direction, they'd be greatly appreciated.

(edit) More details to address questions:

I'm using gfortran (GNU Fortran from GCC 4.2.1 Apple Inc. build 5646).

Python 2.6.4 was installed more-or-less like so:

cd /tmp
curl -O http://www.python.org/ftp/python/2.6.4/Python-2.6.4.tar.bz2
tar xf Python-2.6.4.tar.bz2
cd Python-2.6.4
# Now replace buggy pythonw.c file with one that supports the "arch" command:
curl http://bugs.python.org/file14949/pythonw.c | sed s/2.7/2.6/ > Mac/Tools/pythonw.c    
./configure --enable-framework=/Library/Frameworks --enable-universalsdk=/ --with-universal-archs=intel
make -j4
sudo make frameworkinstall

Scipy 0.7.1 was installed pretty much as described as here, but it boils down to a simple sudo python setup.py install.

It would indeed appear that the symbol is undefined in the i386 architecture if you look at the _cephes library with nm, as suggested by David Cournapeau:

$ nm -arch x86_64 /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/_cephes.so | grep _aswfa_
00000000000d4950 T _aswfa_
000000000011e4b0 d _oblate_aswfa_data
000000000011e510 d _oblate_aswfa_nocv_data
(snip)
$ nm -arch i386 /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/_cephes.so | grep _aswfa_
         U _aswfa_
0002e96c d _oblate_aswfa_data
0002e99c d _oblate_aswfa_nocv_data
(snip)

however, I can't yet explain its absence.

Answer 1

How did you install scipy, for which python version, and with which fortran compiler ?

You may also want to check that the missing symbol is indeed in both archs (I don't remember off-hand where the function is, but you should be able to find ti by yourself pretty easily using a combination of nm/otool).

Answer 2

Have you tried using scipy compiled using macports?

sudo port install scipy +universal

(of course you must have the rest of the chain, python, py26-numpycompiled with the same option)

I get:

$ arch -x86_64 /opt/local/bin/python -c "import scipy.interpolate; print 'success'"
success

$ arch -i386 /opt/local/bin/python -c "import scipy.interpolate; print 'success'"
success

you may then use the setting and knowledge that the macports maintainers used to make your own compilation.

Answer 3

I stumbled across this and am having exactly the same issue. One thing I am seeing is that the makefile in scipy/scipy/special/Makefile is all sorts of messed up. There is /usr/local/include/python2.1 hardcoded in, of course python2.1 is old and I don't have that directory. Is this a useful observation on this issue? I would also like to see it solved.