SCons: Copy external libraries to build output directory

Question

We're using SCons + swtoolkit for our build system on Mac, Linux, and Windows. We have a svn repository that contains all of our external library dependencies, the path to which is located in the environment variable EXTERNAL_SDKS.

In each of our target SConscripts, I'd like to find a method that will look up what libraries located under the EXTERNAL_SDKS path the target is linked to and copy it into the build output folder when the target itself is built and placed there.

I have found one method of doing so using some added on components in swtoolkit, but it slows down the parsing of the sconscripts a ton (15+ seconds on mac, 1 minute+ on windows!!).

Does anyone know of an efficient way of doing this?

Answer 1

I found an answer via Randall Spangler, the dev at Google that created swtoolkit. Thus this answer is specific to using swtoolkit with SCons.

Previously we were scanning our targets for dependencies, then determining what external libraries to copy from that dependency scan. This is what was causing the severe slowdown.

swtoolkit has an env.Publish() method that registers targets so that they can be used as dependencies for other targets. Once the external libraries have been published, they'll automatically be copied into the build output folder via the ReplicatePublished() call that is used within swtoolkit when a target is built.

He gave the following sample code:

thirdparty_libs = []
for dir in env.SubstList2('$THIRDPARTY_LIB_DIRS'):
  thirdparty_libs += env.Glob(dir + '/lib*.dylib')
  thirdparty_libs += env.Glob(dir + '/lib*.a')

import os
for lib in thirdparty_libs:
  name_parts = os.path.splitext(lib.name)
  if name_parts[1] == '.dylib':
    # TODO: Need to publish 'libfoo.dylib' or 'libfoo.a' as both
    # 'libfoo' and 'foo'.  Need to clean up sconscripts to remove 'lib' prefix
    # from all libs for mac, linux.
    lib_basename = name_parts[0]
    env.Publish(lib_basename, 'run', lib)
    env.Publish(lib_basename[3:], 'run', lib)

We modified this to suit our needs and placed it in the scripts that configure use of our external libraries. For example, in our BoostSettings module, this finds and publishes all of the boost libraries. If one of them is needed by a target, it is automatically copied into the build output folder.