How can I make setuptools ignore subversion inventory?

Question

When packaging a Python package with a setup.py that uses the setuptools:

from setuptools import setup
...

the source distribution created by:

python setup.py sdist

not only includes, as usual, the files specified in MANIFEST.in, but it also, gratuitously, includes all of the files that Subversion lists as being version controlled beneath the package directory. This is vastly annoying. Not only does it make it difficult to exercise any sort of explicit control over what files get distributed with my package, but it means that when I build my package following an "svn export" instead of an "svn checkout", the contents of my package might be quite different, since without the .svn metadata setuptools will make different choices about what to include.

My question: how can I turn off this terrible behavior, so that "setuptools" treats my project the same way whether I'm using Subversion, or version control it's never heard of, or a bare tree created with "svn export" that I've created at the end of my project to make sure it builds cleanly somewhere besides my working directory?

The best I have managed so far is an ugly monkey-patch:

from setuptools.command import sdist
del sdist.finders[:]

But this is Python, not the jungle, so of course I want a better solution that involves no monkeys at all. How can I tame setuptools, turn off its magic, and have it behave sensibly by looking at the visible, predictable rules in my MANIFEST.py instead?

Answer 1

Probably the answer is in your setup.py. Do you use find_packages? This function by default uses the VCS (e.g. subversion, hg, ...). If you don't like it, just write a different Python function which collects only the things you want.

Answer 2

You probably want something like this:

from distutils.core import setup

def packages():
    import os

    packages = []

    for path, dirs, files in os.walk("yourprogram"):
        if ".svn" in dirs:
            dirs.remove(".svn")

        if "__init__.py" in files:
            packages.append(path.replace(os.sep, "."))

    return packages

setup(
    # name, version, description, etc...

    packages = packages(),

    # pacakge_data, data_files, etc...
)

Answer 3

I know you know much of this, Brandon, but I'll try to give as a complete answer as I can (although I'm no setuptools gury) for the benefit of others.

The problem here is that setuptools itself involves quite a lot of black magick, including using an entry point called setuptools.file_finders where you can add plugins to find files to include. I am, however, at a complete loss as to how REMOVE plugins from it...

• Quick workaround: svn export your package to a temporary directory and run the setup.py from there. That means you have no svn, so the svn finder finds no files to include. :)

• Longer workaround: Do you really need setuptools? Setuptools have a lot of features, so the answer is likely yes, but mainly those features are depdenencies (so your dependencies get installed by easy_install), namespace packages (foo.bar), and entry points. Namespace packages can actually be created without setuptools as well. But if you use none of these you might actually get away with just using distutils.

• Ugly workaround: The monkeypatch you gave to sdist in your question, which simply makes the plugin not have any finders, and exit quickly.

So as you see, this answer, although as complete as I can make it, is still embarrassingly incomplete. I can't actually answer your question, though I think the answer is "You can't".

Answer 4

I would argue that the default sdist behavior is correct. When you are building a source distribution, I would expect it to contain everything that is checked into Subversion. Of course it would be nice to be able to override it cleanly in special circumstances.

Compare sdist to bdist_egg; I bet only the files that are specified explicitly get included.

I did a simple test with three files, all in svn. Empty dummy.lkj and foobar.py and with setup.py looking like this:

import setuptools
setuptools.setup(name='foobar', version='0.1', py_modules=['foobar'])

sdist creates a tarball that includes dummy.lkj. bdist_egg creates an egg that does not include dummy.lkj.

Answer 5

Create a MANIFEST.in file with:

recursive-exclude .
# other MANIFEST.in commands go here
# to explicitly include whatever files you want

See http://docs.python.org/distutils/commandref.html#sdist-cmd for the MANIFEST.in syntax.