Workflow for maintaining different versions of codebase for different versions of Python

Question

I'm developing an open source application called GarlicSim.

Up to now I've been developing it only for Python 2.6. It seems not to work on any other version.

I decided it's important to produce versions of it that will support other versions of Python. I'm thinking I'll make a version for 2.5, 3.1 and maybe 2.4.

So I have several questions:

1. What would be a good way to organize the folder structure of my repo to include these different versions?
2. What would be a good way to 'merge' changes I do in one version of the code to other versions? I know how to do merges in my SCM (which is git), but these are folders that are all in the same repo, and I want to do a merge between them. There is of course the option of having a repo for each version, but I think it's not a good idea.

Does anyone have any suggestions?

Answer 1

I would try to maintain one branch to cover all the python 2.4-2.6

The differences are not so great, after all if you have to write a bunch of extra code for 2.4 to do something that is easy in 2.6, it will be less work for you in the long run to use the 2.4 version for 2.5 and 2.6.

Python 3 should have a different branch, you should still try to keep as much code in common as you can.

Answer 2

If your code is not overly dependent on the run-time performance in exception handlers, you might even get away without having a separate branch for Py3. I've managed to keep one version of pyparsing for all of my Py2.x versions, although I've had to stick with a "lowest common denominator" approach, meaning that I have to forego using some constructs like generator expressions, and to your point, context managers. I use dicts in place of sets, and all my generator expressions get wrapped as list comprehensions, so they will still work going back to Python 2.3. I have a block at the top of my code that takes care of a number of 2vs3 issues (contributed by pyparsing user Robert A Clark):

_PY3K = sys.version_info[0] > 2
if _PY3K:
    _MAX_INT = sys.maxsize
    basestring = str
    unichr = chr
    unicode = str
    _str2dict = set
    alphas = string.ascii_lowercase + string.ascii_uppercase
else:
    _MAX_INT = sys.maxint
    range = xrange
    def _str2dict(strg):
        return dict( [(c,0) for c in strg] )
    alphas = string.lowercase + string.uppercase

The biggest difficulty I've had has been with the incompatible syntax for catching exceptions, that was introduced in Py3, changing from

except exceptiontype,varname:

to

except exceptionType as varname:

Of course, if you don't really need the exception variable, you can just write:

except exceptionType:

and this will work on Py2 or Py3. But if you need to access the exception, you can still come up with a cross-version compatible syntax as:

except exceptionType:
    exceptionvar = sys.exc_info()[1]

This has a minor run-time penalty, which makes this unusable in some places in pyparsing, so I still have to maintain separate Py2 and Py3 versions. For source merging, I use the utility WinMerge, which I find very good for keeping directories of source code in synch.

So even though I keep two versions of my code, some of these unification techniques help me to keep the differences down to the absolute incompatible minimum.

Answer 3

You need separate branches for separate versions only in the rarest of cases. You mention context managers, and they are great, and it would suck not to use them, and you are right. But for Python 2.4 you will have to not use them. So that will suck. So therefore, if you want to support Python 2.4 you'll have to write a version without context managers. But that one will work under Python 2.6 too, so there is no point in having separate versions there.

As for Python 3, having a separate branch there is a solution, but generally not the best one. For Python 3 support there is something called 2to3 which will convert your Python 2 code to Python 3 code. It's not perfect, so quite often you will have to modify the Python 2 code to generate nice Python 3 code, but the Python 2 code has a tendency to become better as a result anyway.

With Distribute (a maintained fork of setuptools) you can make this conversation automatically during install. That way you don't have to have a separate branch even for Python 3. See http://bitbucket.org/tarek/distribute/src/tip/docs/python3.txt for the docs on that.

As Paul McGuire writes it is even possible to support Python 3 and Python 2 with the same code without using 2to3, but I would not recommend it if you want to support anything else than 2.6 and 3.x. You get too much of this ugly special hacks. With 2.6 there is enough forwards compatibility with Python 3 to make it possible to write decent looking code and support both Python 2.6 and 3.x, but not Python 2.5 and 3.x.

Answer 4

I eventually decided to have 4 different forks for my project, for 2.4, 2.5, 2.6 and 3.1. My main priority is 2.6 and I don't want to compromise the elegance of that code for the sake of 2.4. So the ugly compatibility hacks will be on the lower versions, not the higher versions.