How can I move my Python2.6 site-packeges into Python2.7?

Question

I just ran an update on ArchLinux which gave me Python3 and Python2.7.

Before this update, I was using Python2.6. The modules I have installed reside in /usr/lib/python2.6/site-package. I now want to use Python2.7 and remove Python2.6.

How can I move my Python2.6 modules into Python2.7 ?

Is it as simple as doing mv /usr/lib/python2.6/site-packages/* /usr/lib/python2.7/site-packages ?

Answer 1

Not a complete answer: It is not as simple as a mv. The files are byte compiled into .pyc files which are specific to python versions. So at the very least you'd have to regenerate the .pyc files. (Removing them should be sufficient, too.) Regenerating can be done using compileall.py.

Most distributions offer a saner way to upgrade Python modules than manual fiddling like this, so maybe someone can else can give the Arch specific part of the answer?

Answer 2

The clean way would be re-installing. However, for many if not most of pure python packages the mv approach would work

Answer 3

Your question is really, "How can I get the packages I have in python 2.6 into my [new] python 2.7 configuration? Would copying the files work?"

I would recommend installing the packages into 2.7 the same way you did your 2.6 packages. I would not recommend you copy the files.

Reasonable ways to install the files are:

1. easy_install

   Get easy_install like this: wget http://python-distribute.org/distribute_setup.py && sudo python ./distribute_setup.py

2. pip install

   Get pip like this: sudo easy_install pip

3. apt-get install
4. wget and untar

Answer 4

You might want to 'easy_install yolk', which can be invoked as 'yolk -l' to give you an easy-to-read list of all the installed packages.

Answer 5

Try something like this:

#!/usr/bin/env python

import os
import os.path
import subprocess
import sys
import tempfile

def distributions(path):
    # Extrapolate from paths which distributions presently exist.
    (parent, child) = os.path.split(path)
    while child is not '' and not child.startswith('python'):
        (parent, child) = os.path.split(parent)
    if len(child) > 0:
        rel = os.path.relpath(path, os.path.join(parent, child))
        ret = []
        for distro in os.listdir(parent):
            if distro.startswith('python'):
                dir = os.path.join(os.path.join(parent, distro), rel)
                if os.path.isdir(dir):
                    ret.append((distro, dir))
        ret.sort()
        return ret
    return []

def packages(dir):
    return [pkg.split('-')[0] for pkg in os.listdir(dir)]

def migrate(old, new):
    print 'moving all packages found in ' + old[0] + ' (' + old[1] + ') to ' + new[0] + ' (' + new[1] + ')'
    f = tempfile.TemporaryFile(mode = 'w+')
    result = subprocess.call(
      ['which', 'easy_install'], stdout = f, stderr = subprocess.PIPE)
    f.seek(0)
    easy_install = f.readline().strip()
    f.close()
    if os.path.isfile(easy_install):
        pkgs = packages(old[1])
        success = []
        failure = []
        for pkg in pkgs:
            # Invoke easy_install on the package
            print 'installing "' + pkg + '" for ' + new[0]
            result = subprocess.call(
              [new[0], easy_install, pkg])
            if result != 0:
                failure.append(pkg)
                print 'failed'
            else:
                success.append(pkg)
                print 'success'
        print str(len(success)) + ' of ' + str(len(pkgs)) + ' succeeded'
    else:
        print 'Unable to locate easy_install script'

if __name__ == '__main__':
    package_path = sys.path[-1]
    distros = distributions(package_path)
    migrate(distros[0], distros[1])
    list(package_path)